2024-05-08T05:05:14.332 INFO:root:teuthology version: 0.0.1.dev271+g1ae7ad8 2024-05-08T05:05:14.332 DEBUG:teuthology.run:Teuthology command: teuthology --name vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi --owner scheduled_vshankar@teuthology --description fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} --archive /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230 --verbose -- /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/orig.config.yaml 2024-05-08T05:05:14.391 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T05:05:14.476 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230 branch: wip-57676-3 description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} email: vshankar@redhat.com first_in_suite: false job_id: '7697230' kernel: client: sha1: distro flavor: default kdb: true sha1: 806ec36b21129f113ce356c007dd59694d340b37 ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule name: vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-57676-3 ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_export_ephemeral_random_max: 0.1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore compression mode: aggressive bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a subvols: create: 2 subvol_options: --namespace-isolated --size 25000000000 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount - mds.dir_split install: ceph: flavor: default sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=legacy - nowsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 - scontext=system_u:system_r:getty_t:s0 - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-57676-3 sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a owner: scheduled_vshankar@teuthology priority: 80 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 5998 sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a sleep_before_teardown: 0 subset: 915/1024 suite: fs:workload suite_branch: wip-57676-3 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi059.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxOpKdV1TjaPaKkqoGEbojT1DCA9/rVJ3OYAIby07Lt1F2pSljnQuJaR6T5fdVf8awv2B975UjP1JS9PBCw2Vs= smithi096.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHp98TPHGBnoEafOr/SWnSdcJ117YwpfWCFV+7hvZKD2rXIIi+3dQhUhfz+pVmnPZRhl2G4RdcN9WCjEixajB4k= smithi143.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFWvTIGL6cfZtQW3R+0IHv9h9OwH3M/Yp1O1IaO5/unBOXHgHexsrDDEqIfkIwWDOqI78H8QSddeJaycip4rrp4= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate false - ceph fs subvolumegroup pin cephfs qa random 0.10 - check-counter: {} - exec: mon.a: - ceph config set mgr mgr/snap_schedule/log_level debug - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - suites/fsx.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 1ae7ad82388e92a475afff437d49054826c019a1 timestamp: 2024-05-08_04:00:39 tube: smithi user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-05-08T05:05:14.477 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa; will attempt to use it 2024-05-08T05:05:14.478 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/qa/tasks 2024-05-08T05:05:14.478 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-05-08T05:05:14.479 INFO:teuthology.task.internal:Checking packages... 2024-05-08T05:05:14.502 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a' 2024-05-08T05:05:14.502 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-05-08T05:05:14.502 INFO:teuthology.packaging:ref: None 2024-05-08T05:05:14.502 INFO:teuthology.packaging:tag: None 2024-05-08T05:05:14.502 INFO:teuthology.packaging:branch: wip-57676-3 2024-05-08T05:05:14.502 INFO:teuthology.packaging:sha1: bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:05:14.503 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-57676-3 2024-05-08T05:05:14.741 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3538.gbee299f4 2024-05-08T05:05:14.743 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-05-08T05:05:14.749 INFO:teuthology.task.internal:no buildpackages task found 2024-05-08T05:05:14.749 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-05-08T05:05:14.783 INFO:teuthology.task.internal:Saving configuration 2024-05-08T05:05:14.804 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-05-08T05:05:14.810 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-05-08T05:05:14.833 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi059.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:00:21.145144', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNxOpKdV1TjaPaKkqoGEbojT1DCA9/rVJ3OYAIby07Lt1F2pSljnQuJaR6T5fdVf8awv2B975UjP1JS9PBCw2Vs='} 2024-05-08T05:05:14.855 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi096.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:00:21.148263', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBHp98TPHGBnoEafOr/SWnSdcJ117YwpfWCFV+7hvZKD2rXIIi+3dQhUhfz+pVmnPZRhl2G4RdcN9WCjEixajB4k='} 2024-05-08T05:05:14.878 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi143.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-05-08 05:00:21.146906', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBFWvTIGL6cfZtQW3R+0IHv9h9OwH3M/Yp1O1IaO5/unBOXHgHexsrDDEqIfkIwWDOqI78H8QSddeJaycip4rrp4='} 2024-05-08T05:05:14.878 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-05-08T05:05:14.885 INFO:teuthology.task.internal:roles: ubuntu@smithi059.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-05-08T05:05:14.885 INFO:teuthology.task.internal:roles: ubuntu@smithi096.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-05-08T05:05:14.885 INFO:teuthology.task.internal:roles: ubuntu@smithi143.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-05-08T05:05:14.885 INFO:teuthology.run_tasks:Running task console_log... 2024-05-08T05:05:14.989 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f02c64d70d0>, signals=[15]) 2024-05-08T05:05:14.989 INFO:teuthology.run_tasks:Running task internal.connect... 2024-05-08T05:05:14.995 INFO:teuthology.task.internal:Opening connections... 2024-05-08T05:05:14.996 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:05:14.997 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:05:15.073 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi096.front.sepia.ceph.com 2024-05-08T05:05:15.074 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:05:15.148 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi143.front.sepia.ceph.com 2024-05-08T05:05:15.149 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:05:15.222 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-05-08T05:05:15.229 DEBUG:teuthology.orchestra.run.smithi059:> uname -m 2024-05-08T05:05:15.247 INFO:teuthology.orchestra.run.smithi059.stdout:x86_64 2024-05-08T05:05:15.247 DEBUG:teuthology.orchestra.run.smithi059:> cat /etc/os-release 2024-05-08T05:05:15.305 INFO:teuthology.orchestra.run.smithi059.stdout:NAME="CentOS Stream" 2024-05-08T05:05:15.305 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION="9" 2024-05-08T05:05:15.305 INFO:teuthology.orchestra.run.smithi059.stdout:ID="centos" 2024-05-08T05:05:15.305 INFO:teuthology.orchestra.run.smithi059.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:05:15.305 INFO:teuthology.orchestra.run.smithi059.stdout:VERSION_ID="9" 2024-05-08T05:05:15.305 INFO:teuthology.orchestra.run.smithi059.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:05:15.305 INFO:teuthology.orchestra.run.smithi059.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:05:15.306 INFO:teuthology.orchestra.run.smithi059.stdout:ANSI_COLOR="0;31" 2024-05-08T05:05:15.306 INFO:teuthology.orchestra.run.smithi059.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:05:15.306 INFO:teuthology.orchestra.run.smithi059.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:05:15.306 INFO:teuthology.orchestra.run.smithi059.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:05:15.306 INFO:teuthology.orchestra.run.smithi059.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:05:15.306 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:05:15.306 INFO:teuthology.orchestra.run.smithi059.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:05:15.307 INFO:teuthology.lock.ops:Updating smithi059.front.sepia.ceph.com on lock server 2024-05-08T05:05:15.327 DEBUG:teuthology.orchestra.run.smithi096:> uname -m 2024-05-08T05:05:15.344 INFO:teuthology.orchestra.run.smithi096.stdout:x86_64 2024-05-08T05:05:15.345 DEBUG:teuthology.orchestra.run.smithi096:> cat /etc/os-release 2024-05-08T05:05:15.401 INFO:teuthology.orchestra.run.smithi096.stdout:NAME="CentOS Stream" 2024-05-08T05:05:15.401 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION="9" 2024-05-08T05:05:15.401 INFO:teuthology.orchestra.run.smithi096.stdout:ID="centos" 2024-05-08T05:05:15.401 INFO:teuthology.orchestra.run.smithi096.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:05:15.401 INFO:teuthology.orchestra.run.smithi096.stdout:VERSION_ID="9" 2024-05-08T05:05:15.401 INFO:teuthology.orchestra.run.smithi096.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:05:15.402 INFO:teuthology.orchestra.run.smithi096.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:05:15.402 INFO:teuthology.orchestra.run.smithi096.stdout:ANSI_COLOR="0;31" 2024-05-08T05:05:15.402 INFO:teuthology.orchestra.run.smithi096.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:05:15.402 INFO:teuthology.orchestra.run.smithi096.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:05:15.402 INFO:teuthology.orchestra.run.smithi096.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:05:15.402 INFO:teuthology.orchestra.run.smithi096.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:05:15.402 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:05:15.402 INFO:teuthology.orchestra.run.smithi096.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:05:15.402 INFO:teuthology.lock.ops:Updating smithi096.front.sepia.ceph.com on lock server 2024-05-08T05:05:15.506 DEBUG:teuthology.orchestra.run.smithi143:> uname -m 2024-05-08T05:05:15.523 INFO:teuthology.orchestra.run.smithi143.stdout:x86_64 2024-05-08T05:05:15.523 DEBUG:teuthology.orchestra.run.smithi143:> cat /etc/os-release 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:NAME="CentOS Stream" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION="9" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:ID="centos" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:ID_LIKE="rhel fedora" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:VERSION_ID="9" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:PLATFORM_ID="platform:el9" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:PRETTY_NAME="CentOS Stream 9" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:ANSI_COLOR="0;31" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:LOGO="fedora-logo-icon" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-05-08T05:05:15.580 INFO:teuthology.orchestra.run.smithi143.stdout:HOME_URL="https://centos.org/" 2024-05-08T05:05:15.581 INFO:teuthology.orchestra.run.smithi143.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-05-08T05:05:15.581 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-05-08T05:05:15.581 INFO:teuthology.orchestra.run.smithi143.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-05-08T05:05:15.581 INFO:teuthology.lock.ops:Updating smithi143.front.sepia.ceph.com on lock server 2024-05-08T05:05:15.610 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-05-08T05:05:15.621 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-05-08T05:05:15.650 INFO:teuthology.task.internal:Checking for old test directory... 2024-05-08T05:05:15.650 DEBUG:teuthology.orchestra.run.smithi059:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:05:15.652 DEBUG:teuthology.orchestra.run.smithi096:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:05:15.655 DEBUG:teuthology.orchestra.run.smithi143:> test '!' -e /home/ubuntu/cephtest 2024-05-08T05:05:15.671 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-05-08T05:05:15.731 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-05-08T05:05:15.731 DEBUG:teuthology.orchestra.run.smithi059:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:05:15.733 DEBUG:teuthology.orchestra.run.smithi096:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:05:15.735 DEBUG:teuthology.orchestra.run.smithi143:> test -z $(ls -A /var/lib/ceph) 2024-05-08T05:05:15.757 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-05-08T05:05:15.829 INFO:teuthology.run_tasks:Running task kernel... 2024-05-08T05:05:15.843 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'flavor': 'default', 'kdb': True, 'sha1': '806ec36b21129f113ce356c007dd59694d340b37'} 2024-05-08T05:05:15.844 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-05-08T05:05:15.844 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-05-08T05:05:15.844 DEBUG:teuthology.orchestra.run.smithi059:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:05:15.861 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:05:15.861 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-05-08T05:05:15.919 INFO:teuthology.orchestra.run.smithi059.stdout:5.14.0-437.el9.x86_64 2024-05-08T05:05:15.919 INFO:teuthology.task.kernel:Running kernel on smithi059: 5.14.0-437.el9.x86_64 2024-05-08T05:05:15.919 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum install -y kernel 2024-05-08T05:05:19.313 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 4.6 MB/s | 8.1 MB 00:01 2024-05-08T05:05:23.180 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 7.0 MB/s | 19 MB 00:02 2024-05-08T05:05:28.389 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 4.1 MB/s | 6.2 MB 00:01 2024-05-08T05:05:29.989 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 16 kB 00:00 2024-05-08T05:05:31.468 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 16 MB/s | 21 MB 00:01 2024-05-08T05:05:36.833 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 32 kB/s | 1.7 kB 00:00 2024-05-08T05:05:37.044 INFO:teuthology.orchestra.run.smithi059.stdout:Last metadata expiration check: 0:00:01 ago on Wed 08 May 2024 05:05:36 AM UTC. 2024-05-08T05:05:38.922 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-05-08T05:05:38.923 INFO:teuthology.orchestra.run.smithi059.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-05-08T05:05:39.007 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T05:05:39.024 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T05:05:39.024 INFO:teuthology.orchestra.run.smithi059.stdout: Package Architecture Version Repository Size 2024-05-08T05:05:39.024 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T05:05:39.024 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T05:05:39.024 INFO:teuthology.orchestra.run.smithi059.stdout: kernel x86_64 5.14.0-444.el9 baseos 484 k 2024-05-08T05:05:39.024 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T05:05:39.024 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-core x86_64 5.14.0-444.el9 baseos 15 M 2024-05-08T05:05:39.024 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules x86_64 5.14.0-444.el9 baseos 34 M 2024-05-08T05:05:39.024 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules-core x86_64 5.14.0-444.el9 baseos 28 M 2024-05-08T05:05:39.025 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:05:39.025 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T05:05:39.025 INFO:teuthology.orchestra.run.smithi059.stdout:================================================================================ 2024-05-08T05:05:39.025 INFO:teuthology.orchestra.run.smithi059.stdout:Install 4 Packages 2024-05-08T05:05:39.025 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:05:39.025 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 78 M 2024-05-08T05:05:39.026 INFO:teuthology.orchestra.run.smithi059.stdout:Installed size: 123 M 2024-05-08T05:05:39.026 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T05:05:40.301 INFO:teuthology.orchestra.run.smithi059.stdout:(1/4): kernel-5.14.0-444.el9.x86_64.rpm 420 kB/s | 484 kB 00:01 2024-05-08T05:05:41.151 INFO:teuthology.orchestra.run.smithi059.stdout:(2/4): kernel-core-5.14.0-444.el9.x86_64.rpm 7.7 MB/s | 15 MB 00:02 2024-05-08T05:05:42.226 INFO:teuthology.orchestra.run.smithi059.stdout:(3/4): kernel-modules-5.14.0-444.el9.x86_64.rpm 11 MB/s | 34 MB 00:03 2024-05-08T05:05:43.218 INFO:teuthology.orchestra.run.smithi059.stdout:(4/4): kernel-modules-core-5.14.0-444.el9.x86_6 9.6 MB/s | 28 MB 00:02 2024-05-08T05:05:43.218 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:05:43.218 INFO:teuthology.orchestra.run.smithi059.stdout:Total 19 MB/s | 78 MB 00:04 2024-05-08T05:05:43.924 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T05:05:44.305 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T05:05:44.305 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T05:05:45.277 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T05:05:45.277 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T05:05:47.809 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T05:05:48.434 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-modules-core-5.14.0-444.el9.x86_64 1/4 2024-05-08T05:05:48.518 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:05:51.345 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:05:51.479 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:05:56.550 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:05:56.590 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:06:01.699 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:06:32.654 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:06:32.695 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-modules-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:06:33.058 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: kernel-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:06:33.059 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-5.14.0-444.el9.x86_64 1/4 2024-05-08T05:06:33.059 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-core-5.14.0-444.el9.x86_64 2/4 2024-05-08T05:06:33.059 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-modules-5.14.0-444.el9.x86_64 3/4 2024-05-08T05:06:33.848 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : kernel-modules-core-5.14.0-444.el9.x86_64 4/4 2024-05-08T05:06:33.849 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:06:33.849 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T05:06:33.849 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-5.14.0-444.el9.x86_64 2024-05-08T05:06:33.849 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-core-5.14.0-444.el9.x86_64 2024-05-08T05:06:33.849 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules-5.14.0-444.el9.x86_64 2024-05-08T05:06:33.849 INFO:teuthology.orchestra.run.smithi059.stdout: kernel-modules-core-5.14.0-444.el9.x86_64 2024-05-08T05:06:33.849 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:06:33.849 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T05:06:34.169 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T05:06:34.257 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T05:06:34.257 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T05:06:34.257 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-437.el9.x86_64 Expected: 5.14.0-444.el9.x86_64 2024-05-08T05:06:34.257 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-05-08T05:06:34.258 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-05-08T05:06:34.258 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-05-08T05:06:34.258 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi059.front.sepia.ceph.com, path=None, version=distro) 2024-05-08T05:06:34.258 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q kernel | sort -rV | head -n 1 2024-05-08T05:06:34.308 INFO:teuthology.orchestra.run.smithi059.stdout:kernel-5.14.0-444.el9.x86_64 2024-05-08T05:06:34.309 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-444.el9.x86_64 2024-05-08T05:06:34.309 DEBUG:teuthology.orchestra.run.smithi059:> sudo rpm -qi grub2-tools 2024-05-08T05:06:34.350 INFO:teuthology.orchestra.run.smithi059.stdout:Name : grub2-tools 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Epoch : 1 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Version : 2.06 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Release : 68.el9 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Architecture: x86_64 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Install Date: Fri 19 Apr 2024 09:32:39 PM UTC 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Group : Unspecified 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Size : 8264254 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:License : GPLv3+ 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Signature : RSA/SHA256, Mon 07 Aug 2023 01:57:16 PM UTC, Key ID 05b555b38483c65d 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Source RPM : grub2-2.06-68.el9.src.rpm 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Build Date : Wed 02 Aug 2023 03:15:48 PM UTC 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Packager : builder@centos.org 2024-05-08T05:06:34.351 INFO:teuthology.orchestra.run.smithi059.stdout:Vendor : CentOS 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout:URL : http://www.gnu.org/software/grub/ 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout:Summary : Support tools for GRUB. 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout:Description : 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout:customizable bootloader with modular architecture. It supports a rich 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout:variety of kernel formats, file systems, computer architectures and 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout:hardware devices. 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:06:34.352 INFO:teuthology.orchestra.run.smithi059.stdout:This subpackage provides tools for support of all platforms. 2024-05-08T05:06:34.353 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-05-08T05:06:34.353 INFO:teuthology.task.kernel:Updating grub on smithi059 to boot 5.14.0-444.el9.x86_64 2024-05-08T05:06:34.353 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-05-08T05:06:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:Generating grub configuration file ... 2024-05-08T05:06:36.486 INFO:teuthology.orchestra.run.smithi059.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-05-08T05:06:36.521 INFO:teuthology.orchestra.run.smithi059.stderr:done 2024-05-08T05:06:36.523 DEBUG:teuthology.orchestra.run.smithi059:> mktemp 2024-05-08T05:06:36.579 INFO:teuthology.orchestra.run.smithi059.stdout:/tmp/tmp.brOM5zLS1D 2024-05-08T05:06:36.579 DEBUG:teuthology.orchestra.run.smithi059:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.brOM5zLS1D 2024-05-08T05:06:36.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0666 /tmp/tmp.brOM5zLS1D 2024-05-08T05:06:36.788 DEBUG:teuthology.orchestra.remote:smithi059:/tmp/tmp.brOM5zLS1D is 6KB 2024-05-08T05:06:36.838 DEBUG:teuthology.orchestra.run.smithi059:> rm -fr /tmp/tmp.brOM5zLS1D 2024-05-08T05:06:36.852 DEBUG:teuthology.orchestra.run.smithi059:> sudo /bin/ls /boot/loader/entries || true 2024-05-08T05:06:36.914 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-0-rescue.conf 2024-05-08T05:06:36.915 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-239.el9.x86_64.conf 2024-05-08T05:06:36.915 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-437.el9.x86_64.conf 2024-05-08T05:06:36.915 INFO:teuthology.orchestra.run.smithi059.stdout:7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64.conf 2024-05-08T05:06:36.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo grub2-set-default 7fc8828e06d0482694bcf4a447d13cb6-5.14.0-444.el9.x86_64 2024-05-08T05:06:37.060 DEBUG:teuthology.orchestra.run.smithi059:> sudo shutdown -r now 2024-05-08T05:07:07.063 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-05-08T05:07:07.063 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T05:07:07.064 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:07:25.598 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.59 2024-05-08T05:07:34.600 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T05:07:34.601 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:08:34.604 DEBUG:teuthology.orchestra.remote:timed out 2024-05-08T05:08:46.606 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T05:08:46.607 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:08:46.940 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-05-08T05:08:47.280 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T05:08:47.281 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-444.el9.x86_64"... 2024-05-08T05:08:47.281 DEBUG:teuthology.orchestra.run.smithi059:> uname -r 2024-05-08T05:08:47.338 INFO:teuthology.orchestra.run.smithi059.stdout:5.14.0-444.el9.x86_64 2024-05-08T05:08:47.339 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-444.el9.x86_64 vs 5.14.0-444.el9.x86_64 2024-05-08T05:08:47.339 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-05-08T05:08:47.339 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-05-08T05:08:48.339 DEBUG:teuthology.parallel:result is None 2024-05-08T05:08:48.340 INFO:teuthology.run_tasks:Running task internal.base... 2024-05-08T05:08:48.347 INFO:teuthology.task.internal:Creating test directory... 2024-05-08T05:08:48.348 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:08:48.350 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:08:48.353 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-05-08T05:08:48.370 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-05-08T05:08:48.398 INFO:teuthology.run_tasks:Running task internal.archive... 2024-05-08T05:08:48.434 INFO:teuthology.task.internal:Creating archive directory... 2024-05-08T05:08:48.434 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:08:48.436 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:08:48.438 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-05-08T05:08:48.551 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-05-08T05:08:48.558 INFO:teuthology.task.internal:Enabling coredump saving... 2024-05-08T05:08:48.558 DEBUG:teuthology.orchestra.run.smithi096:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:08:48.574 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:08:48.575 DEBUG:teuthology.orchestra.run.smithi143:> test -f /run/.containerenv -o -f /.dockerenv 2024-05-08T05:08:48.591 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:08:48.591 DEBUG:teuthology.orchestra.run.smithi059:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:08:48.594 DEBUG:teuthology.orchestra.run.smithi096:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:08:48.617 DEBUG:teuthology.orchestra.run.smithi143:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-05-08T05:08:48.657 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:08:48.663 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:08:48.669 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:08:48.687 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:08:48.700 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:08:48.708 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-05-08T05:08:48.710 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-05-08T05:08:48.717 INFO:teuthology.task.internal:Configuring sudo... 2024-05-08T05:08:48.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:08:48.719 DEBUG:teuthology.orchestra.run.smithi096:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:08:48.731 DEBUG:teuthology.orchestra.run.smithi143:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-05-08T05:08:48.767 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-05-08T05:08:48.786 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-05-08T05:08:48.786 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:08:48.789 DEBUG:teuthology.orchestra.run.smithi096:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:08:48.797 DEBUG:teuthology.orchestra.run.smithi143:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-05-08T05:08:48.823 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:08:48.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:08:48.943 DEBUG:teuthology.orchestra.run.smithi059:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:08:49.027 DEBUG:teuthology.orchestra.run.smithi059:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:08:49.072 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:08:49.072 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:08:49.140 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:08:49.180 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:08:49.235 DEBUG:teuthology.orchestra.run.smithi096:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:08:49.318 DEBUG:teuthology.orchestra.run.smithi096:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:08:49.359 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:08:49.359 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:08:49.425 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:08:49.470 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-05-08T05:08:49.522 DEBUG:teuthology.orchestra.run.smithi143:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:08:49.604 DEBUG:teuthology.orchestra.run.smithi143:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-05-08T05:08:49.647 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:08:49.647 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-05-08T05:08:49.713 DEBUG:teuthology.orchestra.run.smithi059:> sudo service rsyslog restart 2024-05-08T05:08:49.716 DEBUG:teuthology.orchestra.run.smithi096:> sudo service rsyslog restart 2024-05-08T05:08:49.718 DEBUG:teuthology.orchestra.run.smithi143:> sudo service rsyslog restart 2024-05-08T05:08:49.756 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:08:49.762 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:08:49.793 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:08:50.281 INFO:teuthology.run_tasks:Running task internal.timer... 2024-05-08T05:08:50.289 INFO:teuthology.task.internal:Starting timer... 2024-05-08T05:08:50.289 INFO:teuthology.run_tasks:Running task pcp... 2024-05-08T05:08:50.337 INFO:teuthology.run_tasks:Running task selinux... 2024-05-08T05:08:50.369 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0', 'scontext=system_u:system_r:getty_t:s0', 'scontext=system_u:system_r:getty_t:s0']} 2024-05-08T05:08:50.369 DEBUG:teuthology.orchestra.run.smithi059:> sudo service auditd rotate 2024-05-08T05:08:50.428 INFO:teuthology.orchestra.run.smithi059.stdout:Rotating logs: 2024-05-08T05:08:50.430 DEBUG:teuthology.orchestra.run.smithi096:> sudo service auditd rotate 2024-05-08T05:08:50.494 INFO:teuthology.orchestra.run.smithi096.stdout:Rotating logs: 2024-05-08T05:08:50.497 DEBUG:teuthology.orchestra.run.smithi143:> sudo service auditd rotate 2024-05-08T05:08:50.554 INFO:teuthology.orchestra.run.smithi143.stdout:Rotating logs: 2024-05-08T05:08:50.556 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-05-08T05:08:50.556 DEBUG:teuthology.orchestra.run.smithi059:> /usr/sbin/getenforce 2024-05-08T05:08:50.580 INFO:teuthology.orchestra.run.smithi059.stdout:Permissive 2024-05-08T05:08:50.580 DEBUG:teuthology.orchestra.run.smithi096:> /usr/sbin/getenforce 2024-05-08T05:08:50.612 INFO:teuthology.orchestra.run.smithi096.stdout:Permissive 2024-05-08T05:08:50.612 DEBUG:teuthology.orchestra.run.smithi143:> /usr/sbin/getenforce 2024-05-08T05:08:50.635 INFO:teuthology.orchestra.run.smithi143.stdout:Permissive 2024-05-08T05:08:50.635 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi059.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi096.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi143.front.sepia.ceph.com': 'permissive'} 2024-05-08T05:08:50.635 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:08:50.664 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:08:50.665 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:08:50.690 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:08:50.691 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:08:50.716 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:08:50.717 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-05-08T05:08:50.717 DEBUG:teuthology.orchestra.run.smithi059:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:08:50.745 DEBUG:teuthology.orchestra.run.smithi096:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:08:50.773 DEBUG:teuthology.orchestra.run.smithi143:> sudo /usr/sbin/setenforce permissive 2024-05-08T05:08:50.797 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-05-08T05:08:50.807 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-05-08T05:08:50.812 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-05-08T05:08:50.934 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-05-08T05:08:50.949 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-05-08T05:08:50.950 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi059.front.sepia.ceph.com,smithi096.front.sepia.ceph.com,smithi143.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-05-08T05:14:04.403 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi059.front.sepia.ceph.com'), Remote(name='ubuntu@smithi096.front.sepia.ceph.com'), Remote(name='ubuntu@smithi143.front.sepia.ceph.com')] 2024-05-08T05:14:04.405 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T05:14:04.406 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi059.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:14:04.488 DEBUG:teuthology.orchestra.run.smithi059:> true 2024-05-08T05:14:04.566 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi059.front.sepia.ceph.com' 2024-05-08T05:14:04.566 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-05-08T05:14:04.566 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi096.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:14:04.642 DEBUG:teuthology.orchestra.run.smithi096:> true 2024-05-08T05:14:04.716 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi096.front.sepia.ceph.com' 2024-05-08T05:14:04.717 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-05-08T05:14:04.717 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi143.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-05-08T05:14:04.799 DEBUG:teuthology.orchestra.run.smithi143:> true 2024-05-08T05:14:04.881 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi143.front.sepia.ceph.com' 2024-05-08T05:14:04.882 INFO:teuthology.run_tasks:Running task clock... 2024-05-08T05:14:04.893 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-05-08T05:14:04.893 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:14:04.893 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:14:04.895 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:14:04.896 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:14:04.898 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-05-08T05:14:04.898 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:14:04.923 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:14:04.925 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:14:04.936 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:14:04.940 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:14:04.950 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-05-08T05:14:04.958 INFO:teuthology.orchestra.run.smithi096.stderr:sudo: ntpd: command not found 2024-05-08T05:14:04.964 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-05-08T05:14:04.966 INFO:teuthology.orchestra.run.smithi059.stderr:sudo: ntpd: command not found 2024-05-08T05:14:04.968 INFO:teuthology.orchestra.run.smithi096.stdout:506 Cannot talk to daemon 2024-05-08T05:14:04.978 INFO:teuthology.orchestra.run.smithi059.stdout:506 Cannot talk to daemon 2024-05-08T05:14:04.981 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:14:04.986 INFO:teuthology.orchestra.run.smithi143.stderr:sudo: ntpd: command not found 2024-05-08T05:14:04.992 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:14:04.994 INFO:teuthology.orchestra.run.smithi096.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:14:04.996 INFO:teuthology.orchestra.run.smithi143.stdout:506 Cannot talk to daemon 2024-05-08T05:14:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:14:05.010 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-05-08T05:14:05.022 INFO:teuthology.orchestra.run.smithi143.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-05-08T05:14:05.051 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-05-08T05:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.055 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.067 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:14:05.071 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:14:05.071 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-05-08T05:14:05.071 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.072 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.072 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.072 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.079 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:14:05.082 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:14:05.082 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-05-08T05:14:05.082 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.082 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.082 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.082 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:14:05.083 INFO:teuthology.run_tasks:Running task install... 2024-05-08T05:14:05.092 DEBUG:teuthology.task.install:project ceph 2024-05-08T05:14:05.092 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'}} 2024-05-08T05:14:05.092 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'} 2024-05-08T05:14:05.092 INFO:teuthology.task.install:Using flavor: default 2024-05-08T05:14:05.099 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-05-08T05:14:05.099 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-05-08T05:14:05.100 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:14:05.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:14:05.103 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:14:05.103 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:14:05.104 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'tag': None, 'wait_for_package': False} 2024-05-08T05:14:05.104 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:14:05.328 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:14:05.328 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:14:05.332 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:14:05.332 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:14:05.343 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/ 2024-05-08T05:14:05.343 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3538.gbee299f4 2024-05-08T05:14:05.504 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:14:05.504 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:14:05.504 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:14:05.513 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:14:05.513 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:14:05.513 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:14:05.533 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:14:05.533 DEBUG:teuthology.orchestra.run.smithi059:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:14:05.537 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:14:05.538 DEBUG:teuthology.orchestra.run.smithi096:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:14:05.601 DEBUG:teuthology.orchestra.run.smithi059:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:14:05.602 DEBUG:teuthology.orchestra.run.smithi096:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:14:05.675 DEBUG:teuthology.orchestra.run.smithi096:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:14:05.678 DEBUG:teuthology.orchestra.run.smithi059:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:14:05.688 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-05-08T05:14:05.688 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:14:05.688 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-05-08T05:14:05.714 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-05-08T05:14:05.714 DEBUG:teuthology.orchestra.run.smithi143:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-05-08T05:14:05.738 INFO:teuthology.orchestra.run.smithi096.stdout:check_obsoletes = 1 2024-05-08T05:14:05.740 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum clean all 2024-05-08T05:14:05.744 INFO:teuthology.orchestra.run.smithi059.stdout:check_obsoletes = 1 2024-05-08T05:14:05.746 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum clean all 2024-05-08T05:14:05.779 DEBUG:teuthology.orchestra.run.smithi143:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-05-08T05:14:05.852 DEBUG:teuthology.orchestra.run.smithi143:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-05-08T05:14:05.915 INFO:teuthology.orchestra.run.smithi143.stdout:check_obsoletes = 1 2024-05-08T05:14:05.916 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum clean all 2024-05-08T05:14:05.969 INFO:teuthology.orchestra.run.smithi096.stdout:54 files removed 2024-05-08T05:14:05.987 INFO:teuthology.orchestra.run.smithi059.stdout:54 files removed 2024-05-08T05:14:05.996 DEBUG:teuthology.orchestra.run.smithi096:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:14:06.014 DEBUG:teuthology.orchestra.run.smithi059:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:14:06.142 INFO:teuthology.orchestra.run.smithi143.stdout:54 files removed 2024-05-08T05:14:06.169 DEBUG:teuthology.orchestra.run.smithi143:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-05-08T05:14:06.664 INFO:teuthology.orchestra.run.smithi096.stdout:ceph packages for x86_64 207 kB/s | 84 kB 00:00 2024-05-08T05:14:06.774 INFO:teuthology.orchestra.run.smithi059.stdout:ceph packages for x86_64 178 kB/s | 84 kB 00:00 2024-05-08T05:14:07.073 INFO:teuthology.orchestra.run.smithi143.stdout:ceph packages for x86_64 130 kB/s | 84 kB 00:00 2024-05-08T05:14:07.222 INFO:teuthology.orchestra.run.smithi096.stdout:ceph noarch packages 23 kB/s | 12 kB 00:00 2024-05-08T05:14:07.508 INFO:teuthology.orchestra.run.smithi143.stdout:ceph noarch packages 30 kB/s | 12 kB 00:00 2024-05-08T05:14:07.883 INFO:teuthology.orchestra.run.smithi143.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-05-08T05:14:07.898 INFO:teuthology.orchestra.run.smithi096.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-05-08T05:14:08.323 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - BaseOS 21 MB/s | 8.1 MB 00:00 2024-05-08T05:14:08.575 INFO:teuthology.orchestra.run.smithi059.stdout:ceph noarch packages 6.7 kB/s | 12 kB 00:01 2024-05-08T05:14:08.942 INFO:teuthology.orchestra.run.smithi059.stdout:ceph source packages 6.0 kB/s | 1.9 kB 00:00 2024-05-08T05:14:09.841 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - BaseOS 4.2 MB/s | 8.1 MB 00:01 2024-05-08T05:14:10.066 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - AppStream 29 MB/s | 19 MB 00:00 2024-05-08T05:14:11.117 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - BaseOS 3.8 MB/s | 8.1 MB 00:02 2024-05-08T05:14:11.651 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - AppStream 26 MB/s | 19 MB 00:00 2024-05-08T05:14:12.851 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - AppStream 31 MB/s | 19 MB 00:00 2024-05-08T05:14:14.317 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - CRB 8.8 MB/s | 6.2 MB 00:00 2024-05-08T05:14:15.752 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - CRB 12 MB/s | 6.2 MB 00:00 2024-05-08T05:14:15.784 INFO:teuthology.orchestra.run.smithi096.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-05-08T05:14:16.919 INFO:teuthology.orchestra.run.smithi096.stdout:Extra Packages for Enterprise Linux 21 MB/s | 21 MB 00:00 2024-05-08T05:14:17.586 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - CRB 5.8 MB/s | 6.2 MB 00:01 2024-05-08T05:14:19.103 INFO:teuthology.orchestra.run.smithi059.stdout:CentOS Stream 9 - Extras packages 54 kB/s | 16 kB 00:00 2024-05-08T05:14:20.813 INFO:teuthology.orchestra.run.smithi059.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-05-08T05:14:22.035 INFO:teuthology.orchestra.run.smithi096.stdout:lab-extras 41 kB/s | 1.7 kB 00:00 2024-05-08T05:14:24.295 INFO:teuthology.orchestra.run.smithi096.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:14:24.295 INFO:teuthology.orchestra.run.smithi096.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:14:24.306 INFO:teuthology.orchestra.run.smithi096.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:14:24.307 INFO:teuthology.orchestra.run.smithi096.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:14:24.308 INFO:teuthology.orchestra.run.smithi096.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:14:24.308 INFO:teuthology.orchestra.run.smithi096.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:14:24.310 INFO:teuthology.orchestra.run.smithi096.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:14:24.312 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:14:24.313 INFO:teuthology.orchestra.run.smithi096.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:14:24.374 INFO:teuthology.orchestra.run.smithi096.stdout:Dependencies resolved. 2024-05-08T05:14:24.393 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-05-08T05:14:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: Package Arch Version Repository Size 2024-05-08T05:14:24.393 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-05-08T05:14:24.393 INFO:teuthology.orchestra.run.smithi096.stdout:Installing: 2024-05-08T05:14:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:14:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:14:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:14:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:14:24.393 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:14:24.394 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:14:24.395 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrading: 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:14:24.396 INFO:teuthology.orchestra.run.smithi096.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout:Installing dependencies: 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:14:24.397 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:14:24.398 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:14:24.399 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:14:24.400 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:14:24.401 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:14:24.402 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:14:24.403 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:14:24.404 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:14:24.405 INFO:teuthology.orchestra.run.smithi096.stdout:Installing weak dependencies: 2024-05-08T05:14:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:14:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:14:24.405 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction Summary 2024-05-08T05:14:24.405 INFO:teuthology.orchestra.run.smithi096.stdout:======================================================================================= 2024-05-08T05:14:24.405 INFO:teuthology.orchestra.run.smithi096.stdout:Install 148 Packages 2024-05-08T05:14:24.405 INFO:teuthology.orchestra.run.smithi096.stdout:Upgrade 8 Packages 2024-05-08T05:14:24.405 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:14:24.406 INFO:teuthology.orchestra.run.smithi096.stdout:Total download size: 226 M 2024-05-08T05:14:24.406 INFO:teuthology.orchestra.run.smithi096.stdout:Downloading Packages: 2024-05-08T05:14:25.469 INFO:teuthology.orchestra.run.smithi143.stdout:CentOS Stream 9 - Extras packages 1.9 kB/s | 16 kB 00:08 2024-05-08T05:14:25.745 INFO:teuthology.orchestra.run.smithi096.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 13 kB/s | 6.6 kB 00:00 2024-05-08T05:14:26.296 INFO:teuthology.orchestra.run.smithi059.stdout:lab-extras 39 kB/s | 1.7 kB 00:00 2024-05-08T05:14:26.345 INFO:teuthology.orchestra.run.smithi096.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 1.4 MB/s | 890 kB 00:00 2024-05-08T05:14:26.453 INFO:teuthology.orchestra.run.smithi096.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 1.3 MB/s | 145 kB 00:00 2024-05-08T05:14:27.046 INFO:teuthology.orchestra.run.smithi096.stdout:(4/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 4.0 MB/s | 2.4 MB 00:00 2024-05-08T05:14:27.414 INFO:teuthology.orchestra.run.smithi143.stdout:Extra Packages for Enterprise Linux 12 MB/s | 21 MB 00:01 2024-05-08T05:14:27.429 INFO:teuthology.orchestra.run.smithi096.stdout:(5/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 3.9 MB/s | 1.5 MB 00:00 2024-05-08T05:14:28.063 INFO:teuthology.orchestra.run.smithi096.stdout:(6/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 1.9 MB/s | 5.4 MB 00:02 2024-05-08T05:14:28.471 INFO:teuthology.orchestra.run.smithi096.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 4.5 MB/s | 4.7 MB 00:01 2024-05-08T05:14:28.716 INFO:teuthology.orchestra.run.smithi059.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:14:28.717 INFO:teuthology.orchestra.run.smithi059.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:14:28.728 INFO:teuthology.orchestra.run.smithi059.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:14:28.729 INFO:teuthology.orchestra.run.smithi059.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:14:28.730 INFO:teuthology.orchestra.run.smithi059.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:14:28.730 INFO:teuthology.orchestra.run.smithi059.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:14:28.732 INFO:teuthology.orchestra.run.smithi059.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:14:28.734 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:14:28.735 INFO:teuthology.orchestra.run.smithi059.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:14:28.800 INFO:teuthology.orchestra.run.smithi059.stdout:Dependencies resolved. 2024-05-08T05:14:28.817 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-08T05:14:28.817 INFO:teuthology.orchestra.run.smithi059.stdout: Package Arch Version Repository Size 2024-05-08T05:14:28.817 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-08T05:14:28.817 INFO:teuthology.orchestra.run.smithi059.stdout:Installing: 2024-05-08T05:14:28.817 INFO:teuthology.orchestra.run.smithi059.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:14:28.817 INFO:teuthology.orchestra.run.smithi059.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:14:28.817 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:14:28.817 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:14:28.818 INFO:teuthology.orchestra.run.smithi059.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:14:28.819 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrading: 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout:Installing dependencies: 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:14:28.820 INFO:teuthology.orchestra.run.smithi059.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:14:28.821 INFO:teuthology.orchestra.run.smithi059.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:14:28.822 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:14:28.823 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:14:28.824 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:14:28.825 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:14:28.826 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:14:28.827 INFO:teuthology.orchestra.run.smithi059.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout:Installing weak dependencies: 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction Summary 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout:======================================================================================= 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout:Install 148 Packages 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout:Upgrade 8 Packages 2024-05-08T05:14:28.828 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:14:28.829 INFO:teuthology.orchestra.run.smithi059.stdout:Total download size: 226 M 2024-05-08T05:14:28.829 INFO:teuthology.orchestra.run.smithi059.stdout:Downloading Packages: 2024-05-08T05:14:29.865 INFO:teuthology.orchestra.run.smithi059.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 15 kB/s | 6.6 kB 00:00 2024-05-08T05:14:30.715 INFO:teuthology.orchestra.run.smithi059.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 1.0 MB/s | 890 kB 00:00 2024-05-08T05:14:30.823 INFO:teuthology.orchestra.run.smithi059.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 1.3 MB/s | 145 kB 00:00 2024-05-08T05:14:30.904 INFO:teuthology.orchestra.run.smithi096.stdout:(8/156): ceph-radosgw-19.0.0-3538.gbee299f4.el9 4.4 MB/s | 11 MB 00:02 2024-05-08T05:14:31.029 INFO:teuthology.orchestra.run.smithi096.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 202 kB/s | 25 kB 00:00 2024-05-08T05:14:31.282 INFO:teuthology.orchestra.run.smithi059.stdout:(4/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 5.2 MB/s | 2.4 MB 00:00 2024-05-08T05:14:31.666 INFO:teuthology.orchestra.run.smithi059.stdout:(5/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 2.4 MB/s | 5.4 MB 00:02 2024-05-08T05:14:31.699 INFO:teuthology.orchestra.run.smithi059.stdout:(6/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 3.6 MB/s | 1.5 MB 00:00 2024-05-08T05:14:32.529 INFO:teuthology.orchestra.run.smithi143.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-05-08T05:14:33.308 INFO:teuthology.orchestra.run.smithi059.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 2.9 MB/s | 4.7 MB 00:01 2024-05-08T05:14:33.829 INFO:teuthology.orchestra.run.smithi096.stdout:(10/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x8 2.9 MB/s | 17 MB 00:05 2024-05-08T05:14:34.304 INFO:teuthology.orchestra.run.smithi096.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 475 kB/s | 225 kB 00:00 2024-05-08T05:14:34.404 INFO:teuthology.orchestra.run.smithi096.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 271 kB/s | 27 kB 00:00 2024-05-08T05:14:34.505 INFO:teuthology.orchestra.run.smithi096.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 317 kB/s | 32 kB 00:00 2024-05-08T05:14:34.814 INFO:teuthology.orchestra.run.smithi143.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:14:34.814 INFO:teuthology.orchestra.run.smithi143.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-05-08T05:14:34.825 INFO:teuthology.orchestra.run.smithi143.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-05-08T05:14:34.826 INFO:teuthology.orchestra.run.smithi143.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-05-08T05:14:34.827 INFO:teuthology.orchestra.run.smithi143.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-05-08T05:14:34.828 INFO:teuthology.orchestra.run.smithi143.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-05-08T05:14:34.829 INFO:teuthology.orchestra.run.smithi143.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-05-08T05:14:34.832 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-05-08T05:14:34.832 INFO:teuthology.orchestra.run.smithi143.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-05-08T05:14:34.838 INFO:teuthology.orchestra.run.smithi096.stdout:(14/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 2.2 MB/s | 747 kB 00:00 2024-05-08T05:14:34.892 INFO:teuthology.orchestra.run.smithi143.stdout:Dependencies resolved. 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout: Package Arch Version Repository Size 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout:Installing: 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout: ceph x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 6.6 k 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 890 k 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 145 k 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 1.5 M 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 142 k 2024-05-08T05:14:34.903 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 3.5 M 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 7.4 M 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 49 k 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 11 M 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 49 M 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 273 k 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 763 k 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 225 k 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 27 k 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 18 k 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-05-08T05:14:34.904 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 32 k 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 747 k 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 127 k 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel x86_64 1:3.2.1-1.el9 appstream 4.4 M 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-05-08T05:14:34.905 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 161 k 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 321 k 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 299 k 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 100 k 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 85 k 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 171 k 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrading: 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager x86_64 1:1.47.5-1.el9 baseos 2.3 M 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm x86_64 1:1.47.5-1.el9 baseos 1.8 M 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team x86_64 1:1.47.5-1.el9 baseos 42 k 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui x86_64 1:1.47.5-1.el9 baseos 248 k 2024-05-08T05:14:34.906 INFO:teuthology.orchestra.run.smithi143.stdout: librados2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.4 M 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 3.1 M 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: openssl x86_64 1:3.2.1-1.el9 baseos 1.3 M 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs x86_64 1:3.2.1-1.el9 baseos 2.4 M 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout:Installing dependencies: 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 21 M 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 34 k 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 2.4 M 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 274 k 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 4.7 M 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 17 M 2024-05-08T05:14:34.907 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3538.gbee299f4.el9 ceph-noarch 16 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 25 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 163 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-05-08T05:14:34.908 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 500 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2 x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 5.4 M 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-05-08T05:14:34.909 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 45 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common x86_64 2:19.0.0-3538.gbee299f4.el9 ceph 135 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-05-08T05:14:34.910 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-05-08T05:14:34.911 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-05-08T05:14:34.932 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-05-08T05:14:34.933 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-05-08T05:14:34.933 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-05-08T05:14:34.933 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-05-08T05:14:34.933 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-05-08T05:14:34.934 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-05-08T05:14:34.934 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-05-08T05:14:34.934 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-05-08T05:14:34.934 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-05-08T05:14:34.934 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-05-08T05:14:34.934 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-05-08T05:14:34.934 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-05-08T05:14:34.935 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-05-08T05:14:34.936 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout:Installing weak dependencies: 2024-05-08T05:14:34.937 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-05-08T05:14:34.938 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:14:34.938 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction Summary 2024-05-08T05:14:34.938 INFO:teuthology.orchestra.run.smithi143.stdout:======================================================================================= 2024-05-08T05:14:34.938 INFO:teuthology.orchestra.run.smithi143.stdout:Install 148 Packages 2024-05-08T05:14:34.938 INFO:teuthology.orchestra.run.smithi143.stdout:Upgrade 8 Packages 2024-05-08T05:14:34.938 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:14:34.939 INFO:teuthology.orchestra.run.smithi143.stdout:Total download size: 226 M 2024-05-08T05:14:34.939 INFO:teuthology.orchestra.run.smithi143.stdout:Downloading Packages: 2024-05-08T05:14:34.947 INFO:teuthology.orchestra.run.smithi096.stdout:(15/156): libcephsqlite-19.0.0-3538.gbee299f4.e 1.5 MB/s | 163 kB 00:00 2024-05-08T05:14:35.039 INFO:teuthology.orchestra.run.smithi096.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.4 MB/s | 127 kB 00:00 2024-05-08T05:14:35.100 INFO:teuthology.orchestra.run.smithi059.stdout:(8/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x86 4.9 MB/s | 17 MB 00:03 2024-05-08T05:14:35.200 INFO:teuthology.orchestra.run.smithi059.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 252 kB/s | 25 kB 00:00 2024-05-08T05:14:35.206 INFO:teuthology.orchestra.run.smithi096.stdout:(17/156): libradosstriper1-19.0.0-3538.gbee299f 2.9 MB/s | 500 kB 00:00 2024-05-08T05:14:35.580 INFO:teuthology.orchestra.run.smithi143.stdout:(1/156): ceph-19.0.0-3538.gbee299f4.el9.x86_64. 19 kB/s | 6.6 kB 00:00 2024-05-08T05:14:36.492 INFO:teuthology.orchestra.run.smithi059.stdout:(10/156): ceph-radosgw-19.0.0-3538.gbee299f4.el 3.3 MB/s | 11 MB 00:03 2024-05-08T05:14:36.659 INFO:teuthology.orchestra.run.smithi059.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 1.3 MB/s | 225 kB 00:00 2024-05-08T05:14:36.706 INFO:teuthology.orchestra.run.smithi143.stdout:(2/156): ceph-fuse-19.0.0-3538.gbee299f4.el9.x8 791 kB/s | 890 kB 00:01 2024-05-08T05:14:36.751 INFO:teuthology.orchestra.run.smithi059.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 295 kB/s | 27 kB 00:00 2024-05-08T05:14:36.843 INFO:teuthology.orchestra.run.smithi059.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 346 kB/s | 32 kB 00:00 2024-05-08T05:14:36.923 INFO:teuthology.orchestra.run.smithi143.stdout:(3/156): ceph-immutable-object-cache-19.0.0-353 668 kB/s | 145 kB 00:00 2024-05-08T05:14:37.102 INFO:teuthology.orchestra.run.smithi059.stdout:(14/156): ceph-common-19.0.0-3538.gbee299f4.el9 2.8 MB/s | 21 MB 00:07 2024-05-08T05:14:37.272 INFO:teuthology.orchestra.run.smithi096.stdout:(18/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 2.6 MB/s | 5.4 MB 00:02 2024-05-08T05:14:37.344 INFO:teuthology.orchestra.run.smithi059.stdout:(15/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 1.5 MB/s | 747 kB 00:00 2024-05-08T05:14:37.372 INFO:teuthology.orchestra.run.smithi096.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gbe 452 kB/s | 45 kB 00:00 2024-05-08T05:14:37.427 INFO:teuthology.orchestra.run.smithi059.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.5 MB/s | 127 kB 00:00 2024-05-08T05:14:37.473 INFO:teuthology.orchestra.run.smithi143.stdout:(4/156): ceph-base-19.0.0-3538.gbee299f4.el9.x8 2.4 MB/s | 5.4 MB 00:02 2024-05-08T05:14:37.473 INFO:teuthology.orchestra.run.smithi096.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 1.3 MB/s | 135 kB 00:00 2024-05-08T05:14:37.581 INFO:teuthology.orchestra.run.smithi096.stdout:(21/156): python3-cephfs-19.0.0-3538.gbee299f4. 1.5 MB/s | 161 kB 00:00 2024-05-08T05:14:37.644 INFO:teuthology.orchestra.run.smithi059.stdout:(17/156): libcephsqlite-19.0.0-3538.gbee299f4.e 301 kB/s | 163 kB 00:00 2024-05-08T05:14:37.711 INFO:teuthology.orchestra.run.smithi059.stdout:(18/156): libradosstriper1-19.0.0-3538.gbee299f 1.7 MB/s | 500 kB 00:00 2024-05-08T05:14:37.790 INFO:teuthology.orchestra.run.smithi096.stdout:(22/156): python3-rados-19.0.0-3538.gbee299f4.e 1.5 MB/s | 321 kB 00:00 2024-05-08T05:14:37.811 INFO:teuthology.orchestra.run.smithi059.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gbe 451 kB/s | 45 kB 00:00 2024-05-08T05:14:37.931 INFO:teuthology.orchestra.run.smithi143.stdout:(5/156): ceph-mgr-19.0.0-3538.gbee299f4.el9.x86 3.3 MB/s | 1.5 MB 00:00 2024-05-08T05:14:37.978 INFO:teuthology.orchestra.run.smithi059.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 814 kB/s | 135 kB 00:00 2024-05-08T05:14:37.990 INFO:teuthology.orchestra.run.smithi096.stdout:(23/156): python3-rbd-19.0.0-3538.gbee299f4.el9 1.5 MB/s | 299 kB 00:00 2024-05-08T05:14:38.099 INFO:teuthology.orchestra.run.smithi096.stdout:(24/156): python3-rgw-19.0.0-3538.gbee299f4.el9 921 kB/s | 100 kB 00:00 2024-05-08T05:14:38.112 INFO:teuthology.orchestra.run.smithi059.stdout:(21/156): python3-cephfs-19.0.0-3538.gbee299f4. 1.2 MB/s | 161 kB 00:00 2024-05-08T05:14:38.233 INFO:teuthology.orchestra.run.smithi096.stdout:(25/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 640 kB/s | 85 kB 00:00 2024-05-08T05:14:38.337 INFO:teuthology.orchestra.run.smithi059.stdout:(22/156): python3-rados-19.0.0-3538.gbee299f4.e 1.4 MB/s | 321 kB 00:00 2024-05-08T05:14:38.579 INFO:teuthology.orchestra.run.smithi059.stdout:(23/156): python3-rbd-19.0.0-3538.gbee299f4.el9 1.2 MB/s | 299 kB 00:00 2024-05-08T05:14:38.679 INFO:teuthology.orchestra.run.smithi059.stdout:(24/156): python3-rgw-19.0.0-3538.gbee299f4.el9 996 kB/s | 100 kB 00:00 2024-05-08T05:14:38.771 INFO:teuthology.orchestra.run.smithi059.stdout:(25/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 931 kB/s | 85 kB 00:00 2024-05-08T05:14:39.240 INFO:teuthology.orchestra.run.smithi143.stdout:(6/156): ceph-mds-19.0.0-3538.gbee299f4.el9.x86 1.0 MB/s | 2.4 MB 00:02 2024-05-08T05:14:39.366 INFO:teuthology.orchestra.run.smithi096.stdout:(26/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 2.7 MB/s | 3.1 MB 00:01 2024-05-08T05:14:39.457 INFO:teuthology.orchestra.run.smithi143.stdout:(7/156): ceph-mon-19.0.0-3538.gbee299f4.el9.x86 3.1 MB/s | 4.7 MB 00:01 2024-05-08T05:14:39.475 INFO:teuthology.orchestra.run.smithi096.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 1.5 MB/s | 171 kB 00:00 2024-05-08T05:14:39.583 INFO:teuthology.orchestra.run.smithi096.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 315 kB/s | 34 kB 00:00 2024-05-08T05:14:39.859 INFO:teuthology.orchestra.run.smithi096.stdout:(29/156): ceph-common-19.0.0-3538.gbee299f4.el9 1.4 MB/s | 21 MB 00:14 2024-05-08T05:14:39.884 INFO:teuthology.orchestra.run.smithi096.stdout:(30/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 474 kB/s | 142 kB 00:00 2024-05-08T05:14:39.890 INFO:teuthology.orchestra.run.smithi143.stdout:(8/156): ceph-common-19.0.0-3538.gbee299f4.el9. 4.5 MB/s | 21 MB 00:04 2024-05-08T05:14:39.963 INFO:teuthology.orchestra.run.smithi059.stdout:(26/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 2.3 MB/s | 5.4 MB 00:02 2024-05-08T05:14:39.974 INFO:teuthology.orchestra.run.smithi143.stdout:(9/156): ceph-selinux-19.0.0-3538.gbee299f4.el9 303 kB/s | 25 kB 00:00 2024-05-08T05:14:40.105 INFO:teuthology.orchestra.run.smithi059.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 1.2 MB/s | 171 kB 00:00 2024-05-08T05:14:40.206 INFO:teuthology.orchestra.run.smithi059.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 343 kB/s | 34 kB 00:00 2024-05-08T05:14:40.306 INFO:teuthology.orchestra.run.smithi059.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 1.4 MB/s | 142 kB 00:00 2024-05-08T05:14:40.492 INFO:teuthology.orchestra.run.smithi059.stdout:(30/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 1.8 MB/s | 3.1 MB 00:01 2024-05-08T05:14:41.931 INFO:teuthology.orchestra.run.smithi059.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 2.2 MB/s | 3.5 MB 00:01 2024-05-08T05:14:41.941 INFO:teuthology.orchestra.run.smithi143.stdout:(10/156): ceph-radosgw-19.0.0-3538.gbee299f4.el 4.3 MB/s | 11 MB 00:02 2024-05-08T05:14:42.041 INFO:teuthology.orchestra.run.smithi143.stdout:(11/156): cephfs-mirror-19.0.0-3538.gbee299f4.e 2.2 MB/s | 225 kB 00:00 2024-05-08T05:14:42.065 INFO:teuthology.orchestra.run.smithi059.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gbe 2.0 MB/s | 274 kB 00:00 2024-05-08T05:14:42.125 INFO:teuthology.orchestra.run.smithi143.stdout:(12/156): cephfs-shell-19.0.0-3538.gbee299f4.el 323 kB/s | 27 kB 00:00 2024-05-08T05:14:42.149 INFO:teuthology.orchestra.run.smithi059.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 592 kB/s | 49 kB 00:00 2024-05-08T05:14:42.209 INFO:teuthology.orchestra.run.smithi143.stdout:(13/156): libcephfs-devel-19.0.0-3538.gbee299f4 380 kB/s | 32 kB 00:00 2024-05-08T05:14:42.224 INFO:teuthology.orchestra.run.smithi059.stdout:(34/156): ceph-prometheus-alerts-19.0.0-3538.gb 220 kB/s | 16 kB 00:00 2024-05-08T05:14:42.234 INFO:teuthology.orchestra.run.smithi096.stdout:(31/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 1.5 MB/s | 3.5 MB 00:02 2024-05-08T05:14:42.326 INFO:teuthology.orchestra.run.smithi096.stdout:(32/156): ceph-mgr-diskprediction-local-19.0.0- 3.0 MB/s | 7.4 MB 00:02 2024-05-08T05:14:42.366 INFO:teuthology.orchestra.run.smithi059.stdout:(35/156): ceph-volume-19.0.0-3538.gbee299f4.el9 1.9 MB/s | 273 kB 00:00 2024-05-08T05:14:42.384 INFO:teuthology.orchestra.run.smithi143.stdout:(14/156): libcephfs2-19.0.0-3538.gbee299f4.el9. 4.2 MB/s | 747 kB 00:00 2024-05-08T05:14:42.426 INFO:teuthology.orchestra.run.smithi096.stdout:(33/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 494 kB/s | 49 kB 00:00 2024-05-08T05:14:42.484 INFO:teuthology.orchestra.run.smithi143.stdout:(15/156): libcephsqlite-19.0.0-3538.gbee299f4.e 1.6 MB/s | 163 kB 00:00 2024-05-08T05:14:42.493 INFO:teuthology.orchestra.run.smithi096.stdout:(34/156): ceph-mgr-modules-core-19.0.0-3538.gbe 1.0 MB/s | 274 kB 00:00 2024-05-08T05:14:42.510 INFO:teuthology.orchestra.run.smithi096.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3538.gb 197 kB/s | 16 kB 00:00 2024-05-08T05:14:42.585 INFO:teuthology.orchestra.run.smithi143.stdout:(16/156): librados-devel-19.0.0-3538.gbee299f4. 1.2 MB/s | 127 kB 00:00 2024-05-08T05:14:42.649 INFO:teuthology.orchestra.run.smithi059.stdout:(36/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 2.6 MB/s | 763 kB 00:00 2024-05-08T05:14:42.733 INFO:teuthology.orchestra.run.smithi059.stdout:(37/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 219 kB/s | 18 kB 00:00 2024-05-08T05:14:42.752 INFO:teuthology.orchestra.run.smithi096.stdout:(36/156): ceph-volume-19.0.0-3538.gbee299f4.el9 1.0 MB/s | 273 kB 00:00 2024-05-08T05:14:42.760 INFO:teuthology.orchestra.run.smithi143.stdout:(17/156): libradosstriper1-19.0.0-3538.gbee299f 2.8 MB/s | 500 kB 00:00 2024-05-08T05:14:42.777 INFO:teuthology.orchestra.run.smithi096.stdout:(37/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 2.8 MB/s | 763 kB 00:00 2024-05-08T05:14:42.828 INFO:teuthology.orchestra.run.smithi096.stdout:(38/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 243 kB/s | 18 kB 00:00 2024-05-08T05:14:42.883 INFO:teuthology.orchestra.run.smithi059.stdout:(38/156): fuse-2.9.9-15.el9.x86_64.rpm 536 kB/s | 80 kB 00:00 2024-05-08T05:14:42.978 INFO:teuthology.orchestra.run.smithi096.stdout:(39/156): fuse-2.9.9-15.el9.x86_64.rpm 401 kB/s | 80 kB 00:00 2024-05-08T05:14:43.059 INFO:teuthology.orchestra.run.smithi059.stdout:(39/156): libgfortran-11.4.1-3.el9.x86_64.rpm 4.5 MB/s | 803 kB 00:00 2024-05-08T05:14:43.095 INFO:teuthology.orchestra.run.smithi096.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 1.6 MB/s | 195 kB 00:00 2024-05-08T05:14:43.117 INFO:teuthology.orchestra.run.smithi059.stdout:(40/156): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-05-08T05:14:43.128 INFO:teuthology.orchestra.run.smithi096.stdout:(41/156): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-05-08T05:14:43.154 INFO:teuthology.orchestra.run.smithi096.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.2 MB/s | 253 kB 00:00 2024-05-08T05:14:43.204 INFO:teuthology.orchestra.run.smithi096.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-08T05:14:43.271 INFO:teuthology.orchestra.run.smithi096.stdout:(44/156): python3-cryptography-36.0.1-4.el9.x86 8.6 MB/s | 1.2 MB 00:00 2024-05-08T05:14:43.296 INFO:teuthology.orchestra.run.smithi096.stdout:(45/156): python3-pycparser-2.20-6.el9.noarch.r 1.4 MB/s | 135 kB 00:00 2024-05-08T05:14:43.343 INFO:teuthology.orchestra.run.smithi059.stdout:(41/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-05-08T05:14:43.355 INFO:teuthology.orchestra.run.smithi096.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.5 MB/s | 125 kB 00:00 2024-05-08T05:14:43.380 INFO:teuthology.orchestra.run.smithi096.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 2.5 MB/s | 215 kB 00:00 2024-05-08T05:14:43.464 INFO:teuthology.orchestra.run.smithi096.stdout:(48/156): automake-1.16.2-8.el9.noarch.rpm 6.2 MB/s | 689 kB 00:00 2024-05-08T05:14:43.539 INFO:teuthology.orchestra.run.smithi096.stdout:(49/156): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-05-08T05:14:43.551 INFO:teuthology.orchestra.run.smithi059.stdout:(42/156): python3-cryptography-36.0.1-4.el9.x86 5.9 MB/s | 1.2 MB 00:00 2024-05-08T05:14:43.589 INFO:teuthology.orchestra.run.smithi096.stdout:(50/156): bison-3.7.4-5.el9.x86_64.rpm 4.4 MB/s | 943 kB 00:00 2024-05-08T05:14:43.602 INFO:teuthology.orchestra.run.smithi059.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-05-08T05:14:43.614 INFO:teuthology.orchestra.run.smithi096.stdout:(51/156): elfutils-libelf-devel-0.190-2.el9.x86 285 kB/s | 21 kB 00:00 2024-05-08T05:14:43.660 INFO:teuthology.orchestra.run.smithi059.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-05-08T05:14:43.665 INFO:teuthology.orchestra.run.smithi096.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-05-08T05:14:43.690 INFO:teuthology.orchestra.run.smithi096.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 434 kB/s | 33 kB 00:00 2024-05-08T05:14:43.711 INFO:teuthology.orchestra.run.smithi059.stdout:(45/156): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-05-08T05:14:43.757 INFO:teuthology.orchestra.run.smithi096.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 273 kB/s | 18 kB 00:00 2024-05-08T05:14:43.769 INFO:teuthology.orchestra.run.smithi059.stdout:(46/156): python3-urllib3-1.26.5-5.el9.noarch.r 3.6 MB/s | 215 kB 00:00 2024-05-08T05:14:43.802 INFO:teuthology.orchestra.run.smithi143.stdout:(18/156): librgw2-19.0.0-3538.gbee299f4.el9.x86 5.2 MB/s | 5.4 MB 00:01 2024-05-08T05:14:43.824 INFO:teuthology.orchestra.run.smithi096.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 168 kB/s | 11 kB 00:00 2024-05-08T05:14:43.877 INFO:teuthology.orchestra.run.smithi143.stdout:(19/156): python3-ceph-argparse-19.0.0-3538.gbe 603 kB/s | 45 kB 00:00 2024-05-08T05:14:43.891 INFO:teuthology.orchestra.run.smithi096.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.3 MB/s | 160 kB 00:00 2024-05-08T05:14:43.911 INFO:teuthology.orchestra.run.smithi059.stdout:(47/156): automake-1.16.2-8.el9.noarch.rpm 4.8 MB/s | 689 kB 00:00 2024-05-08T05:14:43.958 INFO:teuthology.orchestra.run.smithi096.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 680 kB/s | 45 kB 00:00 2024-05-08T05:14:43.960 INFO:teuthology.orchestra.run.smithi143.stdout:(20/156): python3-ceph-common-19.0.0-3538.gbee2 1.6 MB/s | 135 kB 00:00 2024-05-08T05:14:44.053 INFO:teuthology.orchestra.run.smithi143.stdout:(21/156): python3-cephfs-19.0.0-3538.gbee299f4. 1.7 MB/s | 161 kB 00:00 2024-05-08T05:14:44.078 INFO:teuthology.orchestra.run.smithi059.stdout:(48/156): ceph-mgr-diskprediction-local-19.0.0- 2.1 MB/s | 7.4 MB 00:03 2024-05-08T05:14:44.100 INFO:teuthology.orchestra.run.smithi096.stdout:(58/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 6.9 MB/s | 3.0 MB 00:00 2024-05-08T05:14:44.134 INFO:teuthology.orchestra.run.smithi096.stdout:(59/156): librdkafka-1.6.1-102.el9.x86_64.rpm 3.7 MB/s | 662 kB 00:00 2024-05-08T05:14:44.137 INFO:teuthology.orchestra.run.smithi059.stdout:(49/156): bison-3.7.4-5.el9.x86_64.rpm 4.1 MB/s | 943 kB 00:00 2024-05-08T05:14:44.186 INFO:teuthology.orchestra.run.smithi143.stdout:(22/156): python3-rados-19.0.0-3538.gbee299f4.e 2.4 MB/s | 321 kB 00:00 2024-05-08T05:14:44.187 INFO:teuthology.orchestra.run.smithi059.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 430 kB/s | 21 kB 00:00 2024-05-08T05:14:44.201 INFO:teuthology.orchestra.run.smithi096.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 382 kB/s | 25 kB 00:00 2024-05-08T05:14:44.226 INFO:teuthology.orchestra.run.smithi096.stdout:(61/156): libtool-2.4.6-45.el9.x86_64.rpm 4.5 MB/s | 582 kB 00:00 2024-05-08T05:14:44.246 INFO:teuthology.orchestra.run.smithi059.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 640 kB/s | 107 kB 00:00 2024-05-08T05:14:44.271 INFO:teuthology.orchestra.run.smithi059.stdout:(52/156): flex-2.6.4-9.el9.x86_64.rpm 3.7 MB/s | 313 kB 00:00 2024-05-08T05:14:44.276 INFO:teuthology.orchestra.run.smithi096.stdout:(62/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 629 kB/s | 47 kB 00:00 2024-05-08T05:14:44.296 INFO:teuthology.orchestra.run.smithi059.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 652 kB/s | 33 kB 00:00 2024-05-08T05:14:44.310 INFO:teuthology.orchestra.run.smithi096.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.4 MB/s | 292 kB 00:00 2024-05-08T05:14:44.320 INFO:teuthology.orchestra.run.smithi143.stdout:(23/156): python3-rbd-19.0.0-3538.gbee299f4.el9 2.2 MB/s | 299 kB 00:00 2024-05-08T05:14:44.346 INFO:teuthology.orchestra.run.smithi059.stdout:(54/156): flexiblas-openblas-openmp-3.0.4-8.el9 364 kB/s | 18 kB 00:00 2024-05-08T05:14:44.352 INFO:teuthology.orchestra.run.smithi096.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.5 MB/s | 188 kB 00:00 2024-05-08T05:14:44.377 INFO:teuthology.orchestra.run.smithi096.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 526 kB/s | 35 kB 00:00 2024-05-08T05:14:44.397 INFO:teuthology.orchestra.run.smithi059.stdout:(55/156): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-05-08T05:14:44.412 INFO:teuthology.orchestra.run.smithi143.stdout:(24/156): python3-rgw-19.0.0-3538.gbee299f4.el9 1.1 MB/s | 100 kB 00:00 2024-05-08T05:14:44.456 INFO:teuthology.orchestra.run.smithi059.stdout:(56/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.7 MB/s | 160 kB 00:00 2024-05-08T05:14:44.496 INFO:teuthology.orchestra.run.smithi143.stdout:(25/156): rbd-fuse-19.0.0-3538.gbee299f4.el9.x8 1.0 MB/s | 85 kB 00:00 2024-05-08T05:14:44.514 INFO:teuthology.orchestra.run.smithi059.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 779 kB/s | 45 kB 00:00 2024-05-08T05:14:44.648 INFO:teuthology.orchestra.run.smithi059.stdout:(58/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.8 MB/s | 662 kB 00:00 2024-05-08T05:14:44.719 INFO:teuthology.orchestra.run.smithi096.stdout:(66/156): openblas-openmp-0.3.21-2.el9.x86_64.r 13 MB/s | 4.7 MB 00:00 2024-05-08T05:14:44.790 INFO:teuthology.orchestra.run.smithi059.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 5.8 MB/s | 3.0 MB 00:00 2024-05-08T05:14:44.819 INFO:teuthology.orchestra.run.smithi096.stdout:(67/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 10 MB/s | 4.4 MB 00:00 2024-05-08T05:14:44.840 INFO:teuthology.orchestra.run.smithi059.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 512 kB/s | 25 kB 00:00 2024-05-08T05:14:44.899 INFO:teuthology.orchestra.run.smithi059.stdout:(61/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 814 kB/s | 47 kB 00:00 2024-05-08T05:14:44.911 INFO:teuthology.orchestra.run.smithi096.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 8.3 MB/s | 1.6 MB 00:00 2024-05-08T05:14:44.958 INFO:teuthology.orchestra.run.smithi059.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 1.8 MB/s | 582 kB 00:00 2024-05-08T05:14:45.003 INFO:teuthology.orchestra.run.smithi096.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 4.9 MB/s | 911 kB 00:00 2024-05-08T05:14:45.016 INFO:teuthology.orchestra.run.smithi059.stdout:(63/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-05-08T05:14:45.028 INFO:teuthology.orchestra.run.smithi096.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.1 MB/s | 133 kB 00:00 2024-05-08T05:14:45.041 INFO:teuthology.orchestra.run.smithi059.stdout:(64/156): lua-5.4.4-4.el9.x86_64.rpm 2.2 MB/s | 188 kB 00:00 2024-05-08T05:14:45.075 INFO:teuthology.orchestra.run.smithi059.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-05-08T05:14:45.129 INFO:teuthology.orchestra.run.smithi096.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-05-08T05:14:45.231 INFO:teuthology.orchestra.run.smithi143.stdout:(26/156): rbd-mirror-19.0.0-3538.gbee299f4.el9. 4.2 MB/s | 3.1 MB 00:00 2024-05-08T05:14:45.571 INFO:teuthology.orchestra.run.smithi096.stdout:(72/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 3.4 MB/s | 49 MB 00:14 2024-05-08T05:14:45.587 INFO:teuthology.orchestra.run.smithi096.stdout:(73/156): python3-attrs-20.3.0-7.el9.noarch.rpm 191 kB/s | 87 kB 00:00 2024-05-08T05:14:45.621 INFO:teuthology.orchestra.run.smithi143.stdout:(27/156): rbd-nbd-19.0.0-3538.gbee299f4.el9.x86 436 kB/s | 171 kB 00:00 2024-05-08T05:14:45.646 INFO:teuthology.orchestra.run.smithi096.stdout:(74/156): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-05-08T05:14:45.650 INFO:teuthology.orchestra.run.smithi059.stdout:(66/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 7.7 MB/s | 4.4 MB 00:00 2024-05-08T05:14:45.688 INFO:teuthology.orchestra.run.smithi143.stdout:(28/156): ceph-grafana-dashboards-19.0.0-3538.g 514 kB/s | 34 kB 00:00 2024-05-08T05:14:45.722 INFO:teuthology.orchestra.run.smithi096.stdout:(75/156): postgresql-server-13.14-1.el9.x86_64. 8.1 MB/s | 5.8 MB 00:00 2024-05-08T05:14:45.747 INFO:teuthology.orchestra.run.smithi096.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.4 MB/s | 248 kB 00:00 2024-05-08T05:14:45.763 INFO:teuthology.orchestra.run.smithi143.stdout:(29/156): ceph-mgr-cephadm-19.0.0-3538.gbee299f 1.9 MB/s | 142 kB 00:00 2024-05-08T05:14:45.780 INFO:teuthology.orchestra.run.smithi096.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 439 kB/s | 26 kB 00:00 2024-05-08T05:14:45.801 INFO:teuthology.orchestra.run.smithi059.stdout:(67/156): openblas-openmp-0.3.21-2.el9.x86_64.r 6.2 MB/s | 4.7 MB 00:00 2024-05-08T05:14:45.806 INFO:teuthology.orchestra.run.smithi096.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 327 kB/s | 19 kB 00:00 2024-05-08T05:14:45.831 INFO:teuthology.orchestra.run.smithi096.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-05-08T05:14:45.864 INFO:teuthology.orchestra.run.smithi096.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 594 kB/s | 35 kB 00:00 2024-05-08T05:14:45.909 INFO:teuthology.orchestra.run.smithi059.stdout:(68/156): postgresql-13.14-1.el9.x86_64.rpm 6.2 MB/s | 1.6 MB 00:00 2024-05-08T05:14:45.923 INFO:teuthology.orchestra.run.smithi096.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-08T05:14:45.959 INFO:teuthology.orchestra.run.smithi059.stdout:(69/156): postgresql-contrib-13.14-1.el9.x86_64 5.6 MB/s | 911 kB 00:00 2024-05-08T05:14:45.981 INFO:teuthology.orchestra.run.smithi096.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-08T05:14:45.985 INFO:teuthology.orchestra.run.smithi059.stdout:(70/156): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-05-08T05:14:46.040 INFO:teuthology.orchestra.run.smithi096.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-08T05:14:46.124 INFO:teuthology.orchestra.run.smithi096.stdout:(84/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-05-08T05:14:46.135 INFO:teuthology.orchestra.run.smithi059.stdout:(71/156): protobuf-3.14.0-13.el9.x86_64.rpm 6.7 MB/s | 1.0 MB 00:00 2024-05-08T05:14:46.149 INFO:teuthology.orchestra.run.smithi096.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 388 kB/s | 42 kB 00:00 2024-05-08T05:14:46.185 INFO:teuthology.orchestra.run.smithi059.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.7 MB/s | 87 kB 00:00 2024-05-08T05:14:46.191 INFO:teuthology.orchestra.run.smithi096.stdout:(86/156): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-05-08T05:14:46.217 INFO:teuthology.orchestra.run.smithi096.stdout:(87/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-05-08T05:14:46.250 INFO:teuthology.orchestra.run.smithi096.stdout:(88/156): python3-pyasn1-modules-0.4.8-6.el9.no 4.7 MB/s | 279 kB 00:00 2024-05-08T05:14:46.276 INFO:teuthology.orchestra.run.smithi096.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 870 kB/s | 51 kB 00:00 2024-05-08T05:14:46.309 INFO:teuthology.orchestra.run.smithi096.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 920 kB/s | 54 kB 00:00 2024-05-08T05:14:46.368 INFO:teuthology.orchestra.run.smithi096.stdout:(91/156): python3-toml-0.10.2-6.el9.noarch.rpm 714 kB/s | 42 kB 00:00 2024-05-08T05:14:46.427 INFO:teuthology.orchestra.run.smithi096.stdout:(92/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 759 kB/s | 44 kB 00:00 2024-05-08T05:14:46.489 INFO:teuthology.orchestra.run.smithi143.stdout:(30/156): ceph-mgr-dashboard-19.0.0-3538.gbee29 4.9 MB/s | 3.5 MB 00:00 2024-05-08T05:14:46.493 INFO:teuthology.orchestra.run.smithi096.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-05-08T05:14:46.552 INFO:teuthology.orchestra.run.smithi096.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-05-08T05:14:46.610 INFO:teuthology.orchestra.run.smithi096.stdout:(95/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 815 kB/s | 47 kB 00:00 2024-05-08T05:14:46.745 INFO:teuthology.orchestra.run.smithi096.stdout:(96/156): python3-babel-2.9.1-2.el9.noarch.rpm 5.1 MB/s | 6.0 MB 00:01 2024-05-08T05:14:46.760 INFO:teuthology.orchestra.run.smithi059.stdout:(73/156): postgresql-server-13.14-1.el9.x86_64. 7.2 MB/s | 5.8 MB 00:00 2024-05-08T05:14:46.778 INFO:teuthology.orchestra.run.smithi096.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 1.8 MB/s | 305 kB 00:00 2024-05-08T05:14:46.811 INFO:teuthology.orchestra.run.smithi096.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 869 kB/s | 58 kB 00:00 2024-05-08T05:14:46.836 INFO:teuthology.orchestra.run.smithi059.stdout:(74/156): python3-babel-2.9.1-2.el9.noarch.rpm 9.2 MB/s | 6.0 MB 00:00 2024-05-08T05:14:46.837 INFO:teuthology.orchestra.run.smithi096.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-05-08T05:14:46.861 INFO:teuthology.orchestra.run.smithi059.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 2.4 MB/s | 244 kB 00:00 2024-05-08T05:14:46.887 INFO:teuthology.orchestra.run.smithi096.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 844 kB/s | 64 kB 00:00 2024-05-08T05:14:46.894 INFO:teuthology.orchestra.run.smithi059.stdout:(76/156): python3-jinja2-2.11.3-5.el9.noarch.rp 4.2 MB/s | 248 kB 00:00 2024-05-08T05:14:46.911 INFO:teuthology.orchestra.run.smithi059.stdout:(77/156): python3-jsonpatch-1.21-16.el9.noarch. 510 kB/s | 26 kB 00:00 2024-05-08T05:14:46.937 INFO:teuthology.orchestra.run.smithi059.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 455 kB/s | 19 kB 00:00 2024-05-08T05:14:46.971 INFO:teuthology.orchestra.run.smithi059.stdout:(79/156): python3-mako-1.1.4-6.el9.noarch.rpm 2.9 MB/s | 172 kB 00:00 2024-05-08T05:14:46.996 INFO:teuthology.orchestra.run.smithi059.stdout:(80/156): python3-markupsafe-1.1.1-12.el9.x86_6 593 kB/s | 35 kB 00:00 2024-05-08T05:14:47.054 INFO:teuthology.orchestra.run.smithi096.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 206 kB/s | 45 kB 00:00 2024-05-08T05:14:47.055 INFO:teuthology.orchestra.run.smithi059.stdout:(81/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-05-08T05:14:47.105 INFO:teuthology.orchestra.run.smithi096.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 103 kB/s | 22 kB 00:00 2024-05-08T05:14:47.113 INFO:teuthology.orchestra.run.smithi059.stdout:(82/156): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-05-08T05:14:47.172 INFO:teuthology.orchestra.run.smithi059.stdout:(83/156): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-05-08T05:14:47.230 INFO:teuthology.orchestra.run.smithi096.stdout:(103/156): protobuf-compiler-3.14.0-13.el9.x86_ 4.8 MB/s | 863 kB 00:00 2024-05-08T05:14:47.247 INFO:teuthology.orchestra.run.smithi143.stdout:(31/156): ceph-osd-19.0.0-3538.gbee299f4.el9.x8 2.1 MB/s | 17 MB 00:08 2024-05-08T05:14:47.372 INFO:teuthology.orchestra.run.smithi143.stdout:(32/156): ceph-mgr-modules-core-19.0.0-3538.gbe 2.2 MB/s | 274 kB 00:00 2024-05-08T05:14:47.405 INFO:teuthology.orchestra.run.smithi096.stdout:(104/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 14 MB/s | 16 MB 00:01 2024-05-08T05:14:47.522 INFO:teuthology.orchestra.run.smithi096.stdout:(105/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-08T05:14:47.547 INFO:teuthology.orchestra.run.smithi096.stdout:(106/156): gperftools-libs-2.9.1-2.el9.x86_64.r 978 kB/s | 309 kB 00:00 2024-05-08T05:14:47.564 INFO:teuthology.orchestra.run.smithi096.stdout:(107/156): grpc-data-1.46.7-10.el9.noarch.rpm 123 kB/s | 19 kB 00:00 2024-05-08T05:14:47.606 INFO:teuthology.orchestra.run.smithi059.stdout:(84/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 4.0 MB/s | 49 MB 00:12 2024-05-08T05:14:47.630 INFO:teuthology.orchestra.run.smithi059.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 92 kB/s | 42 kB 00:00 2024-05-08T05:14:47.714 INFO:teuthology.orchestra.run.smithi059.stdout:(86/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.9 MB/s | 159 kB 00:00 2024-05-08T05:14:47.731 INFO:teuthology.orchestra.run.smithi096.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 156 kB/s | 26 kB 00:00 2024-05-08T05:14:47.748 INFO:teuthology.orchestra.run.smithi096.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 265 kB/s | 49 kB 00:00 2024-05-08T05:14:47.790 INFO:teuthology.orchestra.run.smithi143.stdout:(33/156): ceph-test-19.0.0-3538.gbee299f4.el9.x 6.3 MB/s | 49 MB 00:07 2024-05-08T05:14:47.798 INFO:teuthology.orchestra.run.smithi059.stdout:(87/156): python3-pyasn1-modules-0.4.8-6.el9.no 3.3 MB/s | 279 kB 00:00 2024-05-08T05:14:47.806 INFO:teuthology.orchestra.run.smithi096.stdout:(110/156): libarrow-9.0.0-11.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2024-05-08T05:14:47.823 INFO:teuthology.orchestra.run.smithi059.stdout:(88/156): python3-protobuf-3.14.0-13.el9.noarch 1.2 MB/s | 269 kB 00:00 2024-05-08T05:14:47.823 INFO:teuthology.orchestra.run.smithi143.stdout:(34/156): ceph-mgr-rook-19.0.0-3538.gbee299f4.e 109 kB/s | 49 kB 00:00 2024-05-08T05:14:47.832 INFO:teuthology.orchestra.run.smithi096.stdout:(111/156): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-05-08T05:14:47.848 INFO:teuthology.orchestra.run.smithi059.stdout:(89/156): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-05-08T05:14:47.849 INFO:teuthology.orchestra.run.smithi143.stdout:(35/156): ceph-prometheus-alerts-19.0.0-3538.gb 281 kB/s | 16 kB 00:00 2024-05-08T05:14:47.857 INFO:teuthology.orchestra.run.smithi096.stdout:(112/156): luarocks-3.9.2-1.el9.noarch.rpm 1.4 MB/s | 151 kB 00:00 2024-05-08T05:14:47.882 INFO:teuthology.orchestra.run.smithi059.stdout:(90/156): python3-requests-oauthlib-1.3.0-12.el 911 kB/s | 54 kB 00:00 2024-05-08T05:14:47.941 INFO:teuthology.orchestra.run.smithi143.stdout:(36/156): ceph-mgr-diskprediction-local-19.0.0- 5.1 MB/s | 7.4 MB 00:01 2024-05-08T05:14:47.958 INFO:teuthology.orchestra.run.smithi059.stdout:(91/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 5.2 MB/s | 5.1 MB 00:00 2024-05-08T05:14:47.966 INFO:teuthology.orchestra.run.smithi143.stdout:(37/156): ceph-volume-19.0.0-3538.gbee299f4.el9 1.9 MB/s | 273 kB 00:00 2024-05-08T05:14:47.983 INFO:teuthology.orchestra.run.smithi059.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 415 kB/s | 42 kB 00:00 2024-05-08T05:14:47.991 INFO:teuthology.orchestra.run.smithi096.stdout:(113/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.5 MB/s | 839 kB 00:00 2024-05-08T05:14:48.000 INFO:teuthology.orchestra.run.smithi143.stdout:(38/156): cephadm-19.0.0-3538.gbee299f4.el9.noa 5.0 MB/s | 763 kB 00:00 2024-05-08T05:14:48.025 INFO:teuthology.orchestra.run.smithi143.stdout:(39/156): cephfs-top-19.0.0-3538.gbee299f4.el9. 218 kB/s | 18 kB 00:00 2024-05-08T05:14:48.041 INFO:teuthology.orchestra.run.smithi059.stdout:(93/156): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-05-08T05:14:48.075 INFO:teuthology.orchestra.run.smithi096.stdout:(114/156): python3-asyncssh-2.13.2-5.el9.noarch 2.2 MB/s | 548 kB 00:00 2024-05-08T05:14:48.100 INFO:teuthology.orchestra.run.smithi096.stdout:(115/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 179 kB/s | 43 kB 00:00 2024-05-08T05:14:48.100 INFO:teuthology.orchestra.run.smithi059.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 615 kB/s | 36 kB 00:00 2024-05-08T05:14:48.125 INFO:teuthology.orchestra.run.smithi096.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 241 kB/s | 32 kB 00:00 2024-05-08T05:14:48.125 INFO:teuthology.orchestra.run.smithi059.stdout:(95/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 263 kB/s | 44 kB 00:00 2024-05-08T05:14:48.150 INFO:teuthology.orchestra.run.smithi096.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 199 kB/s | 15 kB 00:00 2024-05-08T05:14:48.159 INFO:teuthology.orchestra.run.smithi059.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 808 kB/s | 47 kB 00:00 2024-05-08T05:14:48.167 INFO:teuthology.orchestra.run.smithi143.stdout:(40/156): fuse-2.9.9-15.el9.x86_64.rpm 400 kB/s | 80 kB 00:00 2024-05-08T05:14:48.184 INFO:teuthology.orchestra.run.smithi096.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-05-08T05:14:48.209 INFO:teuthology.orchestra.run.smithi096.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-08T05:14:48.217 INFO:teuthology.orchestra.run.smithi059.stdout:(97/156): uuid-1.6.2-55.el9.x86_64.rpm 994 kB/s | 58 kB 00:00 2024-05-08T05:14:48.234 INFO:teuthology.orchestra.run.smithi096.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.6 MB/s | 303 kB 00:00 2024-05-08T05:14:48.243 INFO:teuthology.orchestra.run.smithi059.stdout:(98/156): socat-1.7.4.1-5.el9.x86_64.rpm 2.5 MB/s | 305 kB 00:00 2024-05-08T05:14:48.276 INFO:teuthology.orchestra.run.smithi143.stdout:(41/156): libquadmath-11.4.1-3.el9.x86_64.rpm 780 kB/s | 195 kB 00:00 2024-05-08T05:14:48.276 INFO:teuthology.orchestra.run.smithi059.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-05-08T05:14:48.277 INFO:teuthology.orchestra.run.smithi096.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 386 kB/s | 35 kB 00:00 2024-05-08T05:14:48.301 INFO:teuthology.orchestra.run.smithi096.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.2 MB/s | 206 kB 00:00 2024-05-08T05:14:48.302 INFO:teuthology.orchestra.run.smithi059.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-05-08T05:14:48.335 INFO:teuthology.orchestra.run.smithi059.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 764 kB/s | 45 kB 00:00 2024-05-08T05:14:48.367 INFO:teuthology.orchestra.run.smithi143.stdout:(42/156): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.2 MB/s | 253 kB 00:00 2024-05-08T05:14:48.418 INFO:teuthology.orchestra.run.smithi096.stdout:(123/156): python3-grpcio-1.46.7-10.el9.x86_64. 11 MB/s | 2.0 MB 00:00 2024-05-08T05:14:48.444 INFO:teuthology.orchestra.run.smithi096.stdout:(124/156): python3-grpcio-tools-1.46.7-10.el9.x 867 kB/s | 144 kB 00:00 2024-05-08T05:14:48.459 INFO:teuthology.orchestra.run.smithi143.stdout:(43/156): python3-ply-3.11-14.el9.noarch.rpm 1.1 MB/s | 106 kB 00:00 2024-05-08T05:14:48.469 INFO:teuthology.orchestra.run.smithi096.stdout:(125/156): python3-jaraco-8.2.1-3.el9.noarch.rp 64 kB/s | 11 kB 00:00 2024-05-08T05:14:48.494 INFO:teuthology.orchestra.run.smithi096.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 235 kB/s | 18 kB 00:00 2024-05-08T05:14:48.537 INFO:teuthology.orchestra.run.smithi096.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-05-08T05:14:48.560 INFO:teuthology.orchestra.run.smithi143.stdout:(44/156): python3-pycparser-2.20-6.el9.noarch.r 1.3 MB/s | 135 kB 00:00 2024-05-08T05:14:48.577 INFO:teuthology.orchestra.run.smithi059.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 81 kB/s | 22 kB 00:00 2024-05-08T05:14:48.593 INFO:teuthology.orchestra.run.smithi143.stdout:(45/156): libgfortran-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2024-05-08T05:14:48.611 INFO:teuthology.orchestra.run.smithi096.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 137 kB/s | 19 kB 00:00 2024-05-08T05:14:48.637 INFO:teuthology.orchestra.run.smithi096.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 138 kB/s | 20 kB 00:00 2024-05-08T05:14:48.652 INFO:teuthology.orchestra.run.smithi143.stdout:(46/156): python3-requests-2.25.1-8.el9.noarch. 1.3 MB/s | 125 kB 00:00 2024-05-08T05:14:48.719 INFO:teuthology.orchestra.run.smithi143.stdout:(47/156): python3-urllib3-1.26.5-5.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-05-08T05:14:48.744 INFO:teuthology.orchestra.run.smithi059.stdout:(103/156): abseil-cpp-20211102.0-3.el9.x86_64.r 3.2 MB/s | 548 kB 00:00 2024-05-08T05:14:48.752 INFO:teuthology.orchestra.run.smithi143.stdout:(48/156): python3-cryptography-36.0.1-4.el9.x86 2.6 MB/s | 1.2 MB 00:00 2024-05-08T05:14:48.786 INFO:teuthology.orchestra.run.smithi143.stdout:(49/156): automake-1.16.2-8.el9.noarch.rpm 5.0 MB/s | 689 kB 00:00 2024-05-08T05:14:48.811 INFO:teuthology.orchestra.run.smithi059.stdout:(104/156): gperftools-libs-2.9.1-2.el9.x86_64.r 4.5 MB/s | 309 kB 00:00 2024-05-08T05:14:48.837 INFO:teuthology.orchestra.run.smithi143.stdout:(50/156): elfutils-libelf-devel-0.190-2.el9.x86 431 kB/s | 21 kB 00:00 2024-05-08T05:14:48.845 INFO:teuthology.orchestra.run.smithi059.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 584 kB/s | 19 kB 00:00 2024-05-08T05:14:48.853 INFO:teuthology.orchestra.run.smithi096.stdout:(130/156): python3-kubernetes-26.1.0-2.el9.noar 3.2 MB/s | 1.0 MB 00:00 2024-05-08T05:14:48.862 INFO:teuthology.orchestra.run.smithi143.stdout:(51/156): boost-program-options-1.75.0-8.el9.x8 980 kB/s | 107 kB 00:00 2024-05-08T05:14:48.879 INFO:teuthology.orchestra.run.smithi096.stdout:(131/156): python3-logutils-0.3.5-21.el9.noarch 173 kB/s | 46 kB 00:00 2024-05-08T05:14:48.896 INFO:teuthology.orchestra.run.smithi143.stdout:(52/156): bison-3.7.4-5.el9.x86_64.rpm 5.2 MB/s | 943 kB 00:00 2024-05-08T05:14:48.904 INFO:teuthology.orchestra.run.smithi096.stdout:(132/156): python3-more-itertools-8.12.0-2.el9. 295 kB/s | 79 kB 00:00 2024-05-08T05:14:48.930 INFO:teuthology.orchestra.run.smithi096.stdout:(133/156): python3-natsort-7.1.1-5.el9.noarch.r 768 kB/s | 58 kB 00:00 2024-05-08T05:14:48.937 INFO:teuthology.orchestra.run.smithi143.stdout:(53/156): flexiblas-3.0.4-8.el9.x86_64.rpm 435 kB/s | 33 kB 00:00 2024-05-08T05:14:48.963 INFO:teuthology.orchestra.run.smithi143.stdout:(54/156): flex-2.6.4-9.el9.x86_64.rpm 2.4 MB/s | 313 kB 00:00 2024-05-08T05:14:49.005 INFO:teuthology.orchestra.run.smithi143.stdout:(55/156): flexiblas-openblas-openmp-3.0.4-8.el9 272 kB/s | 18 kB 00:00 2024-05-08T05:14:49.030 INFO:teuthology.orchestra.run.smithi143.stdout:(56/156): libaio-devel-0.3.111-13.el9.x86_64.rp 166 kB/s | 11 kB 00:00 2024-05-08T05:14:49.089 INFO:teuthology.orchestra.run.smithi143.stdout:(57/156): librabbitmq-0.11.0-7.el9.x86_64.rpm 778 kB/s | 45 kB 00:00 2024-05-08T05:14:49.104 INFO:teuthology.orchestra.run.smithi096.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 1.3 MB/s | 272 kB 00:00 2024-05-08T05:14:49.114 INFO:teuthology.orchestra.run.smithi143.stdout:(58/156): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-05-08T05:14:49.130 INFO:teuthology.orchestra.run.smithi096.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 73 kB/s | 16 kB 00:00 2024-05-08T05:14:49.145 INFO:teuthology.orchestra.run.smithi059.stdout:(106/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 13 MB/s | 16 MB 00:01 2024-05-08T05:14:49.155 INFO:teuthology.orchestra.run.smithi096.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 399 kB/s | 90 kB 00:00 2024-05-08T05:14:49.164 INFO:teuthology.orchestra.run.smithi143.stdout:(59/156): flexiblas-netlib-3.0.4-8.el9.x86_64.r 11 MB/s | 3.0 MB 00:00 2024-05-08T05:14:49.180 INFO:teuthology.orchestra.run.smithi096.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 374 kB/s | 28 kB 00:00 2024-05-08T05:14:49.204 INFO:teuthology.orchestra.run.smithi059.stdout:(107/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 447 kB/s | 26 kB 00:00 2024-05-08T05:14:49.206 INFO:teuthology.orchestra.run.smithi096.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-05-08T05:14:49.215 INFO:teuthology.orchestra.run.smithi143.stdout:(60/156): libuuid-devel-2.37.4-18.el9.x86_64.rp 505 kB/s | 25 kB 00:00 2024-05-08T05:14:49.231 INFO:teuthology.orchestra.run.smithi096.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.5 MB/s | 188 kB 00:00 2024-05-08T05:14:49.240 INFO:teuthology.orchestra.run.smithi143.stdout:(61/156): librdkafka-1.6.1-102.el9.x86_64.rpm 4.3 MB/s | 662 kB 00:00 2024-05-08T05:14:49.256 INFO:teuthology.orchestra.run.smithi096.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 791 kB/s | 59 kB 00:00 2024-05-08T05:14:49.262 INFO:teuthology.orchestra.run.smithi059.stdout:(108/156): liboath-2.6.7-2.el9.x86_64.rpm 833 kB/s | 49 kB 00:00 2024-05-08T05:14:49.274 INFO:teuthology.orchestra.run.smithi143.stdout:(62/156): libtool-2.4.6-45.el9.x86_64.rpm 3.6 MB/s | 582 kB 00:00 2024-05-08T05:14:49.281 INFO:teuthology.orchestra.run.smithi096.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 716 kB/s | 36 kB 00:00 2024-05-08T05:14:49.299 INFO:teuthology.orchestra.run.smithi143.stdout:(63/156): libzstd-devel-1.5.1-2.el9.x86_64.rpm 567 kB/s | 47 kB 00:00 2024-05-08T05:14:49.307 INFO:teuthology.orchestra.run.smithi096.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 679 kB/s | 51 kB 00:00 2024-05-08T05:14:49.321 INFO:teuthology.orchestra.run.smithi059.stdout:(109/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-05-08T05:14:49.332 INFO:teuthology.orchestra.run.smithi096.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-08T05:14:49.350 INFO:teuthology.orchestra.run.smithi143.stdout:(64/156): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.6 MB/s | 292 kB 00:00 2024-05-08T05:14:49.357 INFO:teuthology.orchestra.run.smithi096.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-05-08T05:14:49.375 INFO:teuthology.orchestra.run.smithi143.stdout:(65/156): openblas-0.3.21-2.el9.x86_64.rpm 467 kB/s | 35 kB 00:00 2024-05-08T05:14:49.380 INFO:teuthology.orchestra.run.smithi059.stdout:(110/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-08T05:14:49.382 INFO:teuthology.orchestra.run.smithi096.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 5.6 MB/s | 427 kB 00:00 2024-05-08T05:14:49.400 INFO:teuthology.orchestra.run.smithi143.stdout:(66/156): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-05-08T05:14:49.416 INFO:teuthology.orchestra.run.smithi096.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 345 kB/s | 20 kB 00:00 2024-05-08T05:14:49.475 INFO:teuthology.orchestra.run.smithi096.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 1.6 MB/s | 192 kB 00:00 2024-05-08T05:14:49.513 INFO:teuthology.orchestra.run.smithi059.stdout:(111/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 6.2 MB/s | 839 kB 00:00 2024-05-08T05:14:49.534 INFO:teuthology.orchestra.run.smithi143.stdout:(67/156): postgresql-13.14-1.el9.x86_64.rpm 12 MB/s | 1.6 MB 00:00 2024-05-08T05:14:49.558 INFO:teuthology.orchestra.run.smithi096.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 9.0 MB/s | 1.6 MB 00:00 2024-05-08T05:14:49.622 INFO:teuthology.orchestra.run.smithi059.stdout:(112/156): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-05-08T05:14:49.626 INFO:teuthology.orchestra.run.smithi143.stdout:(68/156): openblas-openmp-0.3.21-2.el9.x86_64.r 17 MB/s | 4.7 MB 00:00 2024-05-08T05:14:49.693 INFO:teuthology.orchestra.run.smithi143.stdout:(69/156): openssl-devel-3.2.1-1.el9.x86_64.rpm 14 MB/s | 4.4 MB 00:00 2024-05-08T05:14:49.752 INFO:teuthology.orchestra.run.smithi143.stdout:(70/156): postgresql-contrib-13.14-1.el9.x86_64 4.1 MB/s | 911 kB 00:00 2024-05-08T05:14:49.777 INFO:teuthology.orchestra.run.smithi143.stdout:(71/156): postgresql-private-libs-13.14-1.el9.x 886 kB/s | 133 kB 00:00 2024-05-08T05:14:49.806 INFO:teuthology.orchestra.run.smithi059.stdout:(113/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 236 kB/s | 43 kB 00:00 2024-05-08T05:14:49.844 INFO:teuthology.orchestra.run.smithi143.stdout:(72/156): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-05-08T05:14:49.873 INFO:teuthology.orchestra.run.smithi059.stdout:(114/156): libarrow-9.0.0-11.el9.x86_64.rpm 4.3 MB/s | 4.4 MB 00:01 2024-05-08T05:14:49.886 INFO:teuthology.orchestra.run.smithi143.stdout:(73/156): protobuf-3.14.0-13.el9.x86_64.rpm 7.5 MB/s | 1.0 MB 00:00 2024-05-08T05:14:49.892 INFO:teuthology.orchestra.run.smithi096.stdout:(149/156): NetworkManager-1.47.5-1.el9.x86_64.r 6.9 MB/s | 2.3 MB 00:00 2024-05-08T05:14:49.898 INFO:teuthology.orchestra.run.smithi059.stdout:(115/156): python3-cachetools-4.2.4-1.el9.noarc 351 kB/s | 32 kB 00:00 2024-05-08T05:14:49.923 INFO:teuthology.orchestra.run.smithi059.stdout:(116/156): python3-certifi-2021.10.8-2.el9.noar 300 kB/s | 15 kB 00:00 2024-05-08T05:14:49.965 INFO:teuthology.orchestra.run.smithi059.stdout:(117/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-05-08T05:14:50.008 INFO:teuthology.orchestra.run.smithi059.stdout:(118/156): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-05-08T05:14:50.017 INFO:teuthology.orchestra.run.smithi096.stdout:(150/156): NetworkManager-libnm-1.47.5-1.el9.x8 15 MB/s | 1.8 MB 00:00 2024-05-08T05:14:50.036 INFO:teuthology.orchestra.run.smithi143.stdout:(74/156): postgresql-server-13.14-1.el9.x86_64. 17 MB/s | 5.8 MB 00:00 2024-05-08T05:14:50.041 INFO:teuthology.orchestra.run.smithi059.stdout:(119/156): protobuf-compiler-3.14.0-13.el9.x86_ 506 kB/s | 863 kB 00:01 2024-05-08T05:14:50.061 INFO:teuthology.orchestra.run.smithi143.stdout:(75/156): python3-devel-3.9.18-3.el9.x86_64.rpm 1.4 MB/s | 244 kB 00:00 2024-05-08T05:14:50.066 INFO:teuthology.orchestra.run.smithi059.stdout:(120/156): python3-colorama-0.4.4-12.el9.noarch 604 kB/s | 35 kB 00:00 2024-05-08T05:14:50.067 INFO:teuthology.orchestra.run.smithi096.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 838 kB/s | 42 kB 00:00 2024-05-08T05:14:50.103 INFO:teuthology.orchestra.run.smithi143.stdout:(76/156): python3-jsonpatch-1.21-16.el9.noarch. 613 kB/s | 26 kB 00:00 2024-05-08T05:14:50.125 INFO:teuthology.orchestra.run.smithi059.stdout:(121/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 1.9 MB/s | 303 kB 00:00 2024-05-08T05:14:50.143 INFO:teuthology.orchestra.run.smithi096.stdout:(152/156): librados2-19.0.0-3538.gbee299f4.el9. 4.7 MB/s | 3.4 MB 00:00 2024-05-08T05:14:50.145 INFO:teuthology.orchestra.run.smithi143.stdout:(77/156): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-05-08T05:14:50.171 INFO:teuthology.orchestra.run.smithi143.stdout:(78/156): python3-jsonpointer-2.0-4.el9.noarch. 286 kB/s | 19 kB 00:00 2024-05-08T05:14:50.184 INFO:teuthology.orchestra.run.smithi059.stdout:(122/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-08T05:14:50.193 INFO:teuthology.orchestra.run.smithi096.stdout:(153/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 4.4 MB/s | 3.1 MB 00:00 2024-05-08T05:14:50.209 INFO:teuthology.orchestra.run.smithi059.stdout:(123/156): python3-google-auth-2.29.0-1.el9.noa 1.2 MB/s | 206 kB 00:00 2024-05-08T05:14:50.218 INFO:teuthology.orchestra.run.smithi096.stdout:(154/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.6 MB/s | 248 kB 00:00 2024-05-08T05:14:50.221 INFO:teuthology.orchestra.run.smithi143.stdout:(79/156): python3-markupsafe-1.1.1-12.el9.x86_6 694 kB/s | 35 kB 00:00 2024-05-08T05:14:50.234 INFO:teuthology.orchestra.run.smithi059.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-05-08T05:14:50.259 INFO:teuthology.orchestra.run.smithi059.stdout:(125/156): python3-jaraco-classes-3.2.1-5.el9.n 352 kB/s | 18 kB 00:00 2024-05-08T05:14:50.285 INFO:teuthology.orchestra.run.smithi059.stdout:(126/156): python3-jaraco-collections-3.0.0-8.e 462 kB/s | 23 kB 00:00 2024-05-08T05:14:50.296 INFO:teuthology.orchestra.run.smithi143.stdout:(80/156): python3-babel-2.9.1-2.el9.noarch.rpm 13 MB/s | 6.0 MB 00:00 2024-05-08T05:14:50.310 INFO:teuthology.orchestra.run.smithi059.stdout:(127/156): python3-jaraco-functools-3.5.0-2.el9 386 kB/s | 19 kB 00:00 2024-05-08T05:14:50.322 INFO:teuthology.orchestra.run.smithi143.stdout:(81/156): python3-mako-1.1.4-6.el9.noarch.rpm 978 kB/s | 172 kB 00:00 2024-05-08T05:14:50.336 INFO:teuthology.orchestra.run.smithi059.stdout:(128/156): python3-jaraco-text-3.2.0-6.el9.noar 392 kB/s | 20 kB 00:00 2024-05-08T05:14:50.377 INFO:teuthology.orchestra.run.smithi059.stdout:(129/156): python3-logutils-0.3.5-21.el9.noarch 1.1 MB/s | 46 kB 00:00 2024-05-08T05:14:50.388 INFO:teuthology.orchestra.run.smithi143.stdout:(82/156): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.7 MB/s | 250 kB 00:00 2024-05-08T05:14:50.414 INFO:teuthology.orchestra.run.smithi143.stdout:(83/156): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-05-08T05:14:50.418 INFO:teuthology.orchestra.run.smithi096.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 4.9 MB/s | 1.3 MB 00:00 2024-05-08T05:14:50.419 INFO:teuthology.orchestra.run.smithi059.stdout:(130/156): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2024-05-08T05:14:50.461 INFO:teuthology.orchestra.run.smithi059.stdout:(131/156): python3-grpcio-1.46.7-10.el9.x86_64. 5.2 MB/s | 2.0 MB 00:00 2024-05-08T05:14:50.464 INFO:teuthology.orchestra.run.smithi143.stdout:(84/156): python3-packaging-20.9-5.el9.noarch.r 1.0 MB/s | 77 kB 00:00 2024-05-08T05:14:50.486 INFO:teuthology.orchestra.run.smithi059.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 864 kB/s | 58 kB 00:00 2024-05-08T05:14:50.489 INFO:teuthology.orchestra.run.smithi143.stdout:(85/156): python3-prettytable-0.7.2-27.el9.noar 560 kB/s | 42 kB 00:00 2024-05-08T05:14:50.528 INFO:teuthology.orchestra.run.smithi059.stdout:(133/156): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-05-08T05:14:50.554 INFO:teuthology.orchestra.run.smithi059.stdout:(134/156): python3-portend-3.1.0-2.el9.noarch.r 245 kB/s | 16 kB 00:00 2024-05-08T05:14:50.557 INFO:teuthology.orchestra.run.smithi143.stdout:(86/156): python3-numpy-1.20.1-5.el9.x86_64.rpm 15 MB/s | 5.1 MB 00:00 2024-05-08T05:14:50.570 INFO:teuthology.orchestra.run.smithi059.stdout:(135/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.1 MB/s | 90 kB 00:00 2024-05-08T05:14:50.596 INFO:teuthology.orchestra.run.smithi059.stdout:(136/156): python3-pyperclip-1.8.2-2.el9.noarch 672 kB/s | 28 kB 00:00 2024-05-08T05:14:50.607 INFO:teuthology.orchestra.run.smithi143.stdout:(87/156): python3-protobuf-3.14.0-13.el9.noarch 1.8 MB/s | 269 kB 00:00 2024-05-08T05:14:50.621 INFO:teuthology.orchestra.run.smithi059.stdout:(137/156): python3-repoze-lru-0.7-16.el9.noarch 610 kB/s | 31 kB 00:00 2024-05-08T05:14:50.627 INFO:teuthology.orchestra.run.smithi096.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.5 MB/s | 2.4 MB 00:00 2024-05-08T05:14:50.631 INFO:teuthology.orchestra.run.smithi096.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:14:50.631 INFO:teuthology.orchestra.run.smithi096.stdout:Total 8.6 MB/s | 226 MB 00:26 2024-05-08T05:14:50.632 INFO:teuthology.orchestra.run.smithi143.stdout:(88/156): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-05-08T05:14:50.657 INFO:teuthology.orchestra.run.smithi143.stdout:(89/156): python3-pyasn1-modules-0.4.8-6.el9.no 2.7 MB/s | 279 kB 00:00 2024-05-08T05:14:50.663 INFO:teuthology.orchestra.run.smithi059.stdout:(138/156): python3-rsa-4.9-2.el9.noarch.rpm 1.4 MB/s | 59 kB 00:00 2024-05-08T05:14:50.683 INFO:teuthology.orchestra.run.smithi143.stdout:(90/156): python3-pytz-2021.1-5.el9.noarch.rpm 676 kB/s | 51 kB 00:00 2024-05-08T05:14:50.688 INFO:teuthology.orchestra.run.smithi059.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.0 MB/s | 188 kB 00:00 2024-05-08T05:14:50.708 INFO:teuthology.orchestra.run.smithi143.stdout:(91/156): python3-requests-oauthlib-1.3.0-12.el 711 kB/s | 54 kB 00:00 2024-05-08T05:14:50.714 INFO:teuthology.orchestra.run.smithi059.stdout:(140/156): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-05-08T05:14:50.739 INFO:teuthology.orchestra.run.smithi059.stdout:(141/156): python3-typing-extensions-4.4.0-2.el 1.0 MB/s | 51 kB 00:00 2024-05-08T05:14:50.758 INFO:teuthology.orchestra.run.smithi143.stdout:(92/156): python3-toml-0.10.2-6.el9.noarch.rpm 555 kB/s | 42 kB 00:00 2024-05-08T05:14:50.773 INFO:teuthology.orchestra.run.smithi059.stdout:(142/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.8 MB/s | 230 kB 00:00 2024-05-08T05:14:50.784 INFO:teuthology.orchestra.run.smithi143.stdout:(93/156): python3-wcwidth-0.2.5-8.el9.noarch.rp 587 kB/s | 44 kB 00:00 2024-05-08T05:14:50.798 INFO:teuthology.orchestra.run.smithi059.stdout:(143/156): python3-websocket-client-1.2.3-2.el9 1.5 MB/s | 90 kB 00:00 2024-05-08T05:14:50.834 INFO:teuthology.orchestra.run.smithi143.stdout:(94/156): qatlib-service-24.02.0-1.el9.x86_64.r 717 kB/s | 36 kB 00:00 2024-05-08T05:14:50.848 INFO:teuthology.orchestra.run.smithi059.stdout:(144/156): python3-zc-lockfile-2.0-10.el9.noarc 400 kB/s | 20 kB 00:00 2024-05-08T05:14:50.859 INFO:teuthology.orchestra.run.smithi143.stdout:(95/156): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-05-08T05:14:50.890 INFO:teuthology.orchestra.run.smithi059.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 3.6 MB/s | 427 kB 00:00 2024-05-08T05:14:50.910 INFO:teuthology.orchestra.run.smithi143.stdout:(96/156): qatzip-libs-1.2.0-1.el9.x86_64.rpm 628 kB/s | 47 kB 00:00 2024-05-08T05:14:50.915 INFO:teuthology.orchestra.run.smithi059.stdout:(146/156): re2-20211101-3.el9.x86_64.rpm 2.8 MB/s | 192 kB 00:00 2024-05-08T05:14:50.960 INFO:teuthology.orchestra.run.smithi143.stdout:(97/156): socat-1.7.4.1-5.el9.x86_64.rpm 3.0 MB/s | 305 kB 00:00 2024-05-08T05:14:50.986 INFO:teuthology.orchestra.run.smithi143.stdout:(98/156): uuid-1.6.2-55.el9.x86_64.rpm 767 kB/s | 58 kB 00:00 2024-05-08T05:14:51.044 INFO:teuthology.orchestra.run.smithi143.stdout:(99/156): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.1 MB/s | 94 kB 00:00 2024-05-08T05:14:51.069 INFO:teuthology.orchestra.run.smithi143.stdout:(100/156): xmlstarlet-1.6.1-20.el9.x86_64.rpm 762 kB/s | 64 kB 00:00 2024-05-08T05:14:51.128 INFO:teuthology.orchestra.run.smithi143.stdout:(101/156): zlib-devel-1.2.11-41.el9.x86_64.rpm 535 kB/s | 45 kB 00:00 2024-05-08T05:14:51.153 INFO:teuthology.orchestra.run.smithi143.stdout:(102/156): lua-devel-5.4.4-4.el9.x86_64.rpm 266 kB/s | 22 kB 00:00 2024-05-08T05:14:51.266 INFO:teuthology.orchestra.run.smithi059.stdout:(147/156): python3-kubernetes-26.1.0-2.el9.noar 1.1 MB/s | 1.0 MB 00:00 2024-05-08T05:14:51.346 INFO:teuthology.orchestra.run.smithi143.stdout:(103/156): python3-scipy-1.6.2-8.el9.x86_64.rpm 24 MB/s | 16 MB 00:00 2024-05-08T05:14:51.391 INFO:teuthology.orchestra.run.smithi059.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 3.2 MB/s | 1.6 MB 00:00 2024-05-08T05:14:51.421 INFO:teuthology.orchestra.run.smithi143.stdout:(104/156): protobuf-compiler-3.14.0-13.el9.x86_ 2.9 MB/s | 863 kB 00:00 2024-05-08T05:14:51.525 INFO:teuthology.orchestra.run.smithi059.stdout:(149/156): librados2-19.0.0-3538.gbee299f4.el9. 5.6 MB/s | 3.4 MB 00:00 2024-05-08T05:14:51.529 INFO:teuthology.orchestra.run.smithi143.stdout:(105/156): grpc-data-1.46.7-10.el9.noarch.rpm 180 kB/s | 19 kB 00:00 2024-05-08T05:14:51.571 INFO:teuthology.orchestra.run.smithi143.stdout:(106/156): abseil-cpp-20211102.0-3.el9.x86_64.r 1.3 MB/s | 548 kB 00:00 2024-05-08T05:14:51.597 INFO:teuthology.orchestra.run.smithi143.stdout:(107/156): gperftools-libs-2.9.1-2.el9.x86_64.r 1.2 MB/s | 309 kB 00:00 2024-05-08T05:14:51.614 INFO:teuthology.orchestra.run.smithi143.stdout:(108/156): libarrow-doc-9.0.0-11.el9.noarch.rpm 621 kB/s | 26 kB 00:00 2024-05-08T05:14:51.656 INFO:teuthology.orchestra.run.smithi143.stdout:(109/156): liboath-2.6.7-2.el9.x86_64.rpm 829 kB/s | 49 kB 00:00 2024-05-08T05:14:51.675 INFO:teuthology.orchestra.run.smithi059.stdout:(150/156): NetworkManager-1.47.5-1.el9.x86_64.r 8.2 MB/s | 2.3 MB 00:00 2024-05-08T05:14:51.681 INFO:teuthology.orchestra.run.smithi143.stdout:(110/156): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2024-05-08T05:14:51.714 INFO:teuthology.orchestra.run.smithi143.stdout:(111/156): luarocks-3.9.2-1.el9.noarch.rpm 2.5 MB/s | 151 kB 00:00 2024-05-08T05:14:51.725 INFO:teuthology.orchestra.run.smithi059.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 838 kB/s | 42 kB 00:00 2024-05-08T05:14:51.760 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction check 2024-05-08T05:14:51.782 INFO:teuthology.orchestra.run.smithi143.stdout:(112/156): parquet-libs-9.0.0-11.el9.x86_64.rpm 8.2 MB/s | 839 kB 00:00 2024-05-08T05:14:51.784 INFO:teuthology.orchestra.run.smithi059.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 4.2 MB/s | 248 kB 00:00 2024-05-08T05:14:51.841 INFO:teuthology.orchestra.run.smithi143.stdout:(113/156): python3-asyncssh-2.13.2-5.el9.noarch 4.3 MB/s | 548 kB 00:00 2024-05-08T05:14:51.866 INFO:teuthology.orchestra.run.smithi143.stdout:(114/156): python3-bcrypt-3.2.2-1.el9.x86_64.rp 520 kB/s | 43 kB 00:00 2024-05-08T05:14:51.912 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction check succeeded. 2024-05-08T05:14:51.912 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction test 2024-05-08T05:14:51.917 INFO:teuthology.orchestra.run.smithi059.stdout:(153/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 4.8 MB/s | 3.1 MB 00:00 2024-05-08T05:14:51.933 INFO:teuthology.orchestra.run.smithi143.stdout:(115/156): libarrow-9.0.0-11.el9.x86_64.rpm 11 MB/s | 4.4 MB 00:00 2024-05-08T05:14:51.958 INFO:teuthology.orchestra.run.smithi143.stdout:(116/156): python3-cachetools-4.2.4-1.el9.noarc 274 kB/s | 32 kB 00:00 2024-05-08T05:14:51.959 INFO:teuthology.orchestra.run.smithi059.stdout:(154/156): NetworkManager-libnm-1.47.5-1.el9.x8 4.2 MB/s | 1.8 MB 00:00 2024-05-08T05:14:51.983 INFO:teuthology.orchestra.run.smithi143.stdout:(117/156): python3-certifi-2021.10.8-2.el9.noar 128 kB/s | 15 kB 00:00 2024-05-08T05:14:51.993 INFO:teuthology.orchestra.run.smithi059.stdout:(155/156): openssl-3.2.1-1.el9.x86_64.rpm 6.5 MB/s | 1.3 MB 00:00 2024-05-08T05:14:52.017 INFO:teuthology.orchestra.run.smithi143.stdout:(118/156): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-05-08T05:14:52.050 INFO:teuthology.orchestra.run.smithi143.stdout:(119/156): python3-cherrypy-18.6.1-2.el9.noarch 5.2 MB/s | 358 kB 00:00 2024-05-08T05:14:52.101 INFO:teuthology.orchestra.run.smithi143.stdout:(120/156): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.5 MB/s | 303 kB 00:00 2024-05-08T05:14:52.126 INFO:teuthology.orchestra.run.smithi143.stdout:(121/156): python3-colorama-0.4.4-12.el9.noarch 323 kB/s | 35 kB 00:00 2024-05-08T05:14:52.152 INFO:teuthology.orchestra.run.smithi143.stdout:(122/156): python3-google-auth-2.29.0-1.el9.noa 2.0 MB/s | 206 kB 00:00 2024-05-08T05:14:52.185 INFO:teuthology.orchestra.run.smithi143.stdout:(123/156): python3-grpcio-tools-1.46.7-10.el9.x 2.4 MB/s | 144 kB 00:00 2024-05-08T05:14:52.210 INFO:teuthology.orchestra.run.smithi143.stdout:(124/156): python3-jaraco-8.2.1-3.el9.noarch.rp 182 kB/s | 11 kB 00:00 2024-05-08T05:14:52.261 INFO:teuthology.orchestra.run.smithi143.stdout:(125/156): python3-grpcio-1.46.7-10.el9.x86_64. 13 MB/s | 2.0 MB 00:00 2024-05-08T05:14:52.268 INFO:teuthology.orchestra.run.smithi059.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 6.8 MB/s | 2.4 MB 00:00 2024-05-08T05:14:52.271 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:14:52.271 INFO:teuthology.orchestra.run.smithi059.stdout:Total 9.6 MB/s | 226 MB 00:23 2024-05-08T05:14:52.286 INFO:teuthology.orchestra.run.smithi143.stdout:(126/156): python3-jaraco-classes-3.2.1-5.el9.n 175 kB/s | 18 kB 00:00 2024-05-08T05:14:52.311 INFO:teuthology.orchestra.run.smithi143.stdout:(127/156): python3-jaraco-collections-3.0.0-8.e 230 kB/s | 23 kB 00:00 2024-05-08T05:14:52.337 INFO:teuthology.orchestra.run.smithi143.stdout:(128/156): python3-jaraco-functools-3.5.0-2.el9 258 kB/s | 19 kB 00:00 2024-05-08T05:14:52.362 INFO:teuthology.orchestra.run.smithi143.stdout:(129/156): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-05-08T05:14:52.387 INFO:teuthology.orchestra.run.smithi143.stdout:(130/156): python3-logutils-0.3.5-21.el9.noarch 920 kB/s | 46 kB 00:00 2024-05-08T05:14:52.429 INFO:teuthology.orchestra.run.smithi143.stdout:(131/156): python3-kubernetes-26.1.0-2.el9.noar 8.7 MB/s | 1.0 MB 00:00 2024-05-08T05:14:52.455 INFO:teuthology.orchestra.run.smithi143.stdout:(132/156): python3-natsort-7.1.1-5.el9.noarch.r 863 kB/s | 58 kB 00:00 2024-05-08T05:14:52.480 INFO:teuthology.orchestra.run.smithi143.stdout:(133/156): python3-more-itertools-8.12.0-2.el9. 671 kB/s | 79 kB 00:00 2024-05-08T05:14:52.505 INFO:teuthology.orchestra.run.smithi143.stdout:(134/156): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-05-08T05:14:52.531 INFO:teuthology.orchestra.run.smithi143.stdout:(135/156): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-05-08T05:14:52.560 INFO:teuthology.orchestra.run.smithi143.stdout:(136/156): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.1 MB/s | 90 kB 00:00 2024-05-08T05:14:52.590 INFO:teuthology.orchestra.run.smithi143.stdout:(137/156): python3-pyperclip-1.8.2-2.el9.noarch 336 kB/s | 28 kB 00:00 2024-05-08T05:14:52.615 INFO:teuthology.orchestra.run.smithi143.stdout:(138/156): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-05-08T05:14:52.640 INFO:teuthology.orchestra.run.smithi143.stdout:(139/156): python3-routes-2.5.1-5.el9.noarch.rp 2.3 MB/s | 188 kB 00:00 2024-05-08T05:14:52.666 INFO:teuthology.orchestra.run.smithi143.stdout:(140/156): python3-rsa-4.9-2.el9.noarch.rpm 780 kB/s | 59 kB 00:00 2024-05-08T05:14:52.691 INFO:teuthology.orchestra.run.smithi143.stdout:(141/156): python3-tempora-5.0.0-2.el9.noarch.r 470 kB/s | 36 kB 00:00 2024-05-08T05:14:52.716 INFO:teuthology.orchestra.run.smithi143.stdout:(142/156): python3-typing-extensions-4.4.0-2.el 675 kB/s | 51 kB 00:00 2024-05-08T05:14:52.742 INFO:teuthology.orchestra.run.smithi143.stdout:(143/156): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-05-08T05:14:52.758 INFO:teuthology.orchestra.run.smithi143.stdout:(144/156): python3-websocket-client-1.2.3-2.el9 1.3 MB/s | 90 kB 00:00 2024-05-08T05:14:52.784 INFO:teuthology.orchestra.run.smithi143.stdout:(145/156): python3-werkzeug-2.0.3-3.el9.1.noarc 6.2 MB/s | 427 kB 00:00 2024-05-08T05:14:52.809 INFO:teuthology.orchestra.run.smithi143.stdout:(146/156): python3-zc-lockfile-2.0-10.el9.noarc 297 kB/s | 20 kB 00:00 2024-05-08T05:14:52.834 INFO:teuthology.orchestra.run.smithi143.stdout:(147/156): re2-20211101-3.el9.x86_64.rpm 2.5 MB/s | 192 kB 00:00 2024-05-08T05:14:52.876 INFO:teuthology.orchestra.run.smithi143.stdout:(148/156): thrift-0.15.0-2.el9.x86_64.rpm 17 MB/s | 1.6 MB 00:00 2024-05-08T05:14:53.335 INFO:teuthology.orchestra.run.smithi143.stdout:(149/156): librados2-19.0.0-3538.gbee299f4.el9. 6.5 MB/s | 3.4 MB 00:00 2024-05-08T05:14:53.385 INFO:teuthology.orchestra.run.smithi143.stdout:(150/156): librbd1-19.0.0-3538.gbee299f4.el9.x8 5.7 MB/s | 3.1 MB 00:00 2024-05-08T05:14:53.495 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction check 2024-05-08T05:14:53.502 INFO:teuthology.orchestra.run.smithi143.stdout:(151/156): NetworkManager-team-1.47.5-1.el9.x86 358 kB/s | 42 kB 00:00 2024-05-08T05:14:53.628 INFO:teuthology.orchestra.run.smithi143.stdout:(152/156): NetworkManager-tui-1.47.5-1.el9.x86_ 1.9 MB/s | 248 kB 00:00 2024-05-08T05:14:53.663 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction check succeeded. 2024-05-08T05:14:53.663 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction test 2024-05-08T05:14:53.719 INFO:teuthology.orchestra.run.smithi096.stdout:Transaction test succeeded. 2024-05-08T05:14:53.719 INFO:teuthology.orchestra.run.smithi096.stdout:Running transaction 2024-05-08T05:14:53.778 INFO:teuthology.orchestra.run.smithi143.stdout:(153/156): NetworkManager-1.47.5-1.el9.x86_64.r 2.6 MB/s | 2.3 MB 00:00 2024-05-08T05:14:54.037 INFO:teuthology.orchestra.run.smithi143.stdout:(154/156): openssl-3.2.1-1.el9.x86_64.rpm 3.3 MB/s | 1.3 MB 00:00 2024-05-08T05:14:54.079 INFO:teuthology.orchestra.run.smithi143.stdout:(155/156): NetworkManager-libnm-1.47.5-1.el9.x8 2.5 MB/s | 1.8 MB 00:00 2024-05-08T05:14:54.254 INFO:teuthology.orchestra.run.smithi143.stdout:(156/156): openssl-libs-3.2.1-1.el9.x86_64.rpm 5.0 MB/s | 2.4 MB 00:00 2024-05-08T05:14:54.257 INFO:teuthology.orchestra.run.smithi143.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:14:54.257 INFO:teuthology.orchestra.run.smithi143.stdout:Total 12 MB/s | 226 MB 00:19 2024-05-08T05:14:55.416 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction check 2024-05-08T05:14:55.566 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction check succeeded. 2024-05-08T05:14:55.566 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction test 2024-05-08T05:14:55.905 INFO:teuthology.orchestra.run.smithi059.stdout:Transaction test succeeded. 2024-05-08T05:14:55.906 INFO:teuthology.orchestra.run.smithi059.stdout:Running transaction 2024-05-08T05:14:57.359 INFO:teuthology.orchestra.run.smithi143.stdout:Transaction test succeeded. 2024-05-08T05:14:57.359 INFO:teuthology.orchestra.run.smithi143.stdout:Running transaction 2024-05-08T05:14:59.804 INFO:teuthology.orchestra.run.smithi096.stdout: Preparing : 1/1 2024-05-08T05:14:59.905 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:14:59.943 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:14:59.997 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:15:00.031 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:15:00.057 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:15:00.108 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:15:00.428 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:15:00.533 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:15:00.654 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:15:00.684 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:15:00.776 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:15:00.805 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:15:00.953 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:15:01.010 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:15:01.038 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:15:01.125 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:15:01.148 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:15:01.235 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:15:01.257 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:15:01.280 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:15:01.465 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:15:01.642 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:15:01.779 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:15:01.831 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:15:01.959 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:15:02.041 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:15:02.258 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:15:02.297 INFO:teuthology.orchestra.run.smithi059.stdout: Preparing : 1/1 2024-05-08T05:15:02.373 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:15:02.375 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:15:02.404 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:15:02.412 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:15:02.438 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:15:02.488 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:15:02.512 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:15:02.524 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:15:02.546 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:15:02.548 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:15:02.590 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:15:02.596 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:15:02.622 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:15:02.650 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:15:02.719 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:15:02.781 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:15:02.811 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:15:02.847 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:15:02.924 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:15:02.940 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:15:02.958 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:15:02.992 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:15:03.021 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:15:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:15:03.090 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:15:03.136 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:15:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:15:03.210 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:15:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:15:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:15:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:15:03.342 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:15:03.397 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:15:03.439 INFO:teuthology.orchestra.run.smithi143.stdout: Preparing : 1/1 2024-05-08T05:15:03.517 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-libs-1:3.2.1-1.el9.x86_64 1/164 2024-05-08T05:15:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:15:03.555 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/164 2024-05-08T05:15:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:15:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:15:03.608 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/164 2024-05-08T05:15:03.643 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/164 2024-05-08T05:15:03.668 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/164 2024-05-08T05:15:03.712 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/164 2024-05-08T05:15:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:15:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:15:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:15:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:15:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:15:03.913 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:15:03.991 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:15:04.021 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:15:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:15:04.041 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/164 2024-05-08T05:15:04.064 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:15:04.111 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:15:04.152 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:15:04.170 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:15:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:15:04.294 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:15:04.305 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:15:04.321 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:15:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:15:04.363 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:15:04.414 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:15:04.414 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:15:04.434 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:15:04.455 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:15:04.519 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:15:04.526 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:15:04.587 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 11/164 2024-05-08T05:15:04.599 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:15:04.607 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:15:04.631 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:15:04.649 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-13.14-1.el9.x86_64 12/164 2024-05-08T05:15:04.668 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:15:04.676 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:15:04.714 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:15:04.750 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:15:04.781 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:15:04.793 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:15:04.810 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:15:04.819 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:15:04.835 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:15:04.873 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 14/164 2024-05-08T05:15:04.895 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:15:04.918 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 15/164 2024-05-08T05:15:04.938 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:15:04.970 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:15:04.971 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:15:05.005 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:15:05.015 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:15:05.046 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:15:05.070 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:15:05.081 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:15:05.087 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:15:05.129 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:15:05.173 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:15:05.181 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:15:05.200 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 16/164 2024-05-08T05:15:05.205 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:15:05.212 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:15:05.241 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:15:05.253 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:15:05.278 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:15:05.303 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:15:05.314 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:15:05.365 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:15:05.376 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:15:05.394 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:15:05.396 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:15:05.438 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:15:05.453 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 17/164 2024-05-08T05:15:05.503 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:15:05.518 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:15:05.535 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:15:05.549 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:15:05.552 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:15:05.563 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:15:05.592 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:15:05.597 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:15:05.621 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:15:05.636 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:15:05.652 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:15:05.668 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:15:05.695 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:15:05.714 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:15:05.741 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:15:05.752 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:15:05.774 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:15:05.815 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:15:05.831 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:15:05.838 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.47.5-1.el9.x86_64 18/164 2024-05-08T05:15:05.850 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:15:05.937 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:15:05.944 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : re2-1:20211101-3.el9.x86_64 19/164 2024-05-08T05:15:05.976 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 20/164 2024-05-08T05:15:05.978 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:15:05.980 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:15:06.011 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : liboath-2.6.7-2.el9.x86_64 21/164 2024-05-08T05:15:06.011 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:15:06.031 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:15:06.058 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:15:06.085 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 22/164 2024-05-08T05:15:06.119 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 23/164 2024-05-08T05:15:06.157 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:15:06.163 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 24/164 2024-05-08T05:15:06.195 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 25/164 2024-05-08T05:15:06.213 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:15:06.224 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-5.4.4-4.el9.x86_64 26/164 2024-05-08T05:15:06.293 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 27/164 2024-05-08T05:15:06.346 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 28/164 2024-05-08T05:15:06.377 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 29/164 2024-05-08T05:15:06.412 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 30/164 2024-05-08T05:15:06.489 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 31/164 2024-05-08T05:15:06.523 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 32/164 2024-05-08T05:15:06.558 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 33/164 2024-05-08T05:15:06.578 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 34/164 2024-05-08T05:15:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:15:06.649 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 35/164 2024-05-08T05:15:06.693 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : luarocks-3.9.2-1.el9.noarch 36/164 2024-05-08T05:15:06.695 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:15:06.729 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:15:06.768 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 37/164 2024-05-08T05:15:06.774 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:15:06.810 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:15:06.854 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:15:06.902 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 38/164 2024-05-08T05:15:06.927 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:15:06.954 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 39/164 2024-05-08T05:15:06.976 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:15:07.007 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:15:07.010 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:15:07.071 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:15:07.108 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:15:07.168 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:15:07.193 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:15:07.229 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:15:07.279 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:15:07.304 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:15:07.312 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:15:07.339 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:15:07.376 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:15:07.424 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:15:07.459 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:15:07.487 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rsa-4.9-2.el9.noarch 40/164 2024-05-08T05:15:07.490 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:15:07.503 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:15:07.518 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:15:07.545 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:15:07.568 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 41/164 2024-05-08T05:15:07.596 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 42/164 2024-05-08T05:15:07.596 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:15:07.639 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 43/164 2024-05-08T05:15:07.677 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 44/164 2024-05-08T05:15:07.689 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:15:07.719 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 45/164 2024-05-08T05:15:07.733 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:15:07.763 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:15:07.805 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:15:07.883 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 46/164 2024-05-08T05:15:07.904 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:15:07.938 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:15:07.939 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : openssl-1:3.2.1-1.el9.x86_64 47/164 2024-05-08T05:15:07.979 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:15:08.011 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:15:08.030 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 48/164 2024-05-08T05:15:08.036 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:15:08.095 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 49/164 2024-05-08T05:15:08.115 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:15:08.168 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 50/164 2024-05-08T05:15:08.199 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 51/164 2024-05-08T05:15:08.235 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 52/164 2024-05-08T05:15:08.236 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:15:08.244 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:15:08.277 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:15:08.282 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 53/164 2024-05-08T05:15:08.305 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:15:08.318 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 54/164 2024-05-08T05:15:08.340 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:15:08.341 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:15:08.361 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 55/164 2024-05-08T05:15:08.379 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:15:08.404 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 56/164 2024-05-08T05:15:08.418 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:15:08.448 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:15:08.483 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:15:08.516 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:15:08.558 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:15:08.564 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 57/164 2024-05-08T05:15:08.592 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:15:08.675 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:15:08.724 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:15:08.754 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:15:08.793 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 58/164 2024-05-08T05:15:08.802 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:15:08.823 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 59/164 2024-05-08T05:15:08.865 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 60/164 2024-05-08T05:15:08.913 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:15:08.960 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 61/164 2024-05-08T05:15:08.981 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:15:08.998 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 62/164 2024-05-08T05:15:09.038 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 63/164 2024-05-08T05:15:09.064 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 64/164 2024-05-08T05:15:09.088 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 65/164 2024-05-08T05:15:09.164 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 66/164 2024-05-08T05:15:09.305 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 67/164 2024-05-08T05:15:09.338 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 68/164 2024-05-08T05:15:09.366 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 69/164 2024-05-08T05:15:09.400 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 70/164 2024-05-08T05:15:09.440 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : uuid-1.6.2-55.el9.x86_64 71/164 2024-05-08T05:15:09.480 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 72/164 2024-05-08T05:15:09.510 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 73/164 2024-05-08T05:15:09.544 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 74/164 2024-05-08T05:15:09.561 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:15:09.569 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 75/164 2024-05-08T05:15:09.611 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 76/164 2024-05-08T05:15:09.640 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:15:09.646 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 77/164 2024-05-08T05:15:09.668 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:15:09.701 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:15:09.733 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 78/164 2024-05-08T05:15:09.765 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:15:09.776 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 79/164 2024-05-08T05:15:09.808 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 80/164 2024-05-08T05:15:09.810 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:15:09.844 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:15:09.855 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 81/164 2024-05-08T05:15:09.950 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:15:09.953 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-packaging-20.9-5.el9.noarch 82/164 2024-05-08T05:15:10.003 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:15:10.009 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/164 2024-05-08T05:15:10.016 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:15:10.064 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:15:10.095 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:15:10.138 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:15:10.154 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:15:10.185 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:15:10.222 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:15:10.330 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:15:10.360 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:15:10.441 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:15:10.701 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:15:10.734 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:15:10.738 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/164 2024-05-08T05:15:10.782 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/164 2024-05-08T05:15:10.805 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:15:10.816 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/164 2024-05-08T05:15:10.827 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:15:10.915 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/164 2024-05-08T05:15:10.974 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 88/164 2024-05-08T05:15:11.062 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 89/164 2024-05-08T05:15:11.101 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 90/164 2024-05-08T05:15:11.148 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:15:11.280 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-0.3.21-2.el9.x86_64 91/164 2024-05-08T05:15:11.307 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 92/164 2024-05-08T05:15:11.385 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 93/164 2024-05-08T05:15:11.441 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:15:12.029 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 94/164 2024-05-08T05:15:12.157 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 95/164 2024-05-08T05:15:13.062 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:15:13.120 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:15:13.139 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:15:13.167 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:15:13.173 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:15:13.200 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:15:13.388 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 96/164 2024-05-08T05:15:13.489 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 97/164 2024-05-08T05:15:13.517 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 98/164 2024-05-08T05:15:13.529 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:15:13.549 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 99/164 2024-05-08T05:15:13.562 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:15:13.637 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:15:13.668 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:15:13.705 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:15:13.709 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:15:13.760 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:15:13.844 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:15:13.845 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:15:13.845 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:15:13.845 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:13.860 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 100/164 2024-05-08T05:15:13.877 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:15:13.905 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:15:14.003 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 101/164 2024-05-08T05:15:14.034 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 102/164 2024-05-08T05:15:14.079 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 103/164 2024-05-08T05:15:14.210 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:15:14.240 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:15:14.312 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:15:14.351 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:15:14.561 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 104/164 2024-05-08T05:15:14.593 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:15:14.665 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 105/164 2024-05-08T05:15:14.695 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 106/164 2024-05-08T05:15:16.725 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:15:16.755 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:15:16.873 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:15:16.916 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:15:17.341 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:15:17.384 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:15:17.455 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 107/164 2024-05-08T05:15:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:15:17.476 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:15:17.476 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:15:17.476 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:17.494 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:15:17.518 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:15:17.586 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 108/164 2024-05-08T05:15:17.586 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-05-08T05:15:17.587 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-05-08T05:15:17.587 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:17.620 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:15:25.310 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:15:25.310 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /sys 2024-05-08T05:15:25.310 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /proc 2024-05-08T05:15:25.310 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /mnt 2024-05-08T05:15:25.310 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /var/tmp 2024-05-08T05:15:25.310 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /home 2024-05-08T05:15:25.311 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /root 2024-05-08T05:15:25.311 INFO:teuthology.orchestra.run.smithi096.stdout:skipping the directory /tmp 2024-05-08T05:15:25.311 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:25.473 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:15:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:15:25.487 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:25.487 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:15:25.487 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:26.984 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:15:27.010 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:15:27.010 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:27.010 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:15:27.010 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:15:27.010 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:15:27.010 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:27.235 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:15:27.258 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:15:27.258 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:27.258 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:15:27.258 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:15:27.258 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:15:27.258 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:27.703 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:15:27.725 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:15:27.725 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:27.725 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:15:27.725 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:15:27.725 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:15:27.725 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:27.825 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:15:27.881 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:15:27.937 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:15:28.122 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:15:28.171 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:15:28.248 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:15:28.335 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:15:28.476 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:15:28.520 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:15:28.552 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:15:28.580 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:15:29.007 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:15:29.037 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:15:29.069 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:15:29.130 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:15:29.160 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:15:29.251 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:15:29.252 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /sys 2024-05-08T05:15:29.252 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /proc 2024-05-08T05:15:29.252 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /mnt 2024-05-08T05:15:29.252 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /var/tmp 2024-05-08T05:15:29.252 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /home 2024-05-08T05:15:29.252 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /root 2024-05-08T05:15:29.252 INFO:teuthology.orchestra.run.smithi143.stdout:skipping the directory /tmp 2024-05-08T05:15:29.252 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:29.341 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 109/164 2024-05-08T05:15:29.341 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /sys 2024-05-08T05:15:29.341 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /proc 2024-05-08T05:15:29.341 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /mnt 2024-05-08T05:15:29.342 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /var/tmp 2024-05-08T05:15:29.342 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /home 2024-05-08T05:15:29.342 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /root 2024-05-08T05:15:29.342 INFO:teuthology.orchestra.run.smithi059.stdout:skipping the directory /tmp 2024-05-08T05:15:29.342 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:29.423 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:15:29.438 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:15:29.438 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:29.438 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:15:29.438 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:29.523 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:15:29.540 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 110/164 2024-05-08T05:15:29.540 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:29.540 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-05-08T05:15:29.540 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:30.188 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:15:30.220 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:15:30.926 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:15:30.952 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:15:30.952 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:30.952 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:15:30.952 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:15:30.952 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:15:30.952 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:31.102 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:15:31.130 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 111/164 2024-05-08T05:15:31.131 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:31.131 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-05-08T05:15:31.131 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:15:31.131 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-05-08T05:15:31.131 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:31.177 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:15:31.183 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:15:31.199 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:15:31.199 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:31.199 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:15:31.200 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:15:31.200 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:15:31.200 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:31.203 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:15:31.317 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:15:31.370 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:15:31.395 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 112/164 2024-05-08T05:15:31.395 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:31.395 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-05-08T05:15:31.395 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:15:31.395 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-05-08T05:15:31.395 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:31.483 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:15:31.514 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:15:31.536 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:15:31.537 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:31.537 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:15:31.537 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:15:31.537 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:15:31.537 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:31.581 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:15:31.592 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:15:31.636 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:15:31.658 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:15:31.658 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:31.658 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:15:31.658 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:15:31.659 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:15:31.659 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:31.690 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:15:31.746 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:15:31.766 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:15:31.823 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:15:31.862 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:15:31.878 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:15:31.887 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 113/164 2024-05-08T05:15:31.887 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:31.887 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-05-08T05:15:31.887 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:15:31.887 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-05-08T05:15:31.888 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:31.993 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : automake-1.16.2-8.el9.noarch 114/164 2024-05-08T05:15:32.044 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-ply-3.11-14.el9.noarch 115/164 2024-05-08T05:15:32.065 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:15:32.106 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 116/164 2024-05-08T05:15:32.111 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:15:32.190 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:15:32.295 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:15:32.304 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 117/164 2024-05-08T05:15:32.363 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 118/164 2024-05-08T05:15:32.435 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:15:32.445 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 119/164 2024-05-08T05:15:32.486 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:15:32.526 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:15:32.544 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 120/164 2024-05-08T05:15:32.553 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:15:32.668 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:15:32.687 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 121/164 2024-05-08T05:15:32.700 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:15:32.722 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 122/164 2024-05-08T05:15:32.723 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:15:32.723 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:32.723 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:15:32.723 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:15:32.723 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:15:32.723 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:32.763 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 123/164 2024-05-08T05:15:32.768 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:15:32.789 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : fuse-2.9.9-15.el9.x86_64 124/164 2024-05-08T05:15:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:15:32.790 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:32.790 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:15:32.790 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:32.835 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:15:32.856 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:15:32.857 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:32.857 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:15:32.857 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:15:32.857 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:15:32.857 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:32.990 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:15:33.019 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:15:33.051 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:15:33.113 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:15:33.150 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:15:33.160 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:15:33.177 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:15:33.182 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:33.204 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 125/164 2024-05-08T05:15:33.237 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 126/164 2024-05-08T05:15:33.301 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 127/164 2024-05-08T05:15:33.335 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:15:34.196 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:15:34.227 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:15:34.434 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 128/164 2024-05-08T05:15:34.471 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:15:35.190 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:15:35.219 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:15:35.329 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:15:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:15:35.489 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 129/164 2024-05-08T05:15:35.512 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:15:35.522 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:15:35.535 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:15:35.535 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:35.535 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:15:35.535 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:15:35.535 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:15:35.535 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:35.580 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:15:35.591 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:15:35.639 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 130/164 2024-05-08T05:15:35.688 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:15:35.745 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:15:35.815 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 131/164 2024-05-08T05:15:35.839 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:15:35.865 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 132/164 2024-05-08T05:15:35.865 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:35.865 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-05-08T05:15:35.865 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:15:35.865 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-05-08T05:15:35.865 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:35.915 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:15:35.927 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 133/164 2024-05-08T05:15:36.028 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 134/164 2024-05-08T05:15:36.098 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 135/164 2024-05-08T05:15:36.665 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:15:36.699 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:15:36.724 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:15:36.724 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:36.724 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:15:36.724 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:15:36.724 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:15:36.724 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:36.766 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:15:36.788 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:15:36.788 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:36.788 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:15:36.788 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:36.842 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:15:36.864 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:15:36.865 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:36.865 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:15:36.865 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:15:36.865 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:15:36.865 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:37.073 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libtool-2.4.6-45.el9.x86_64 136/164 2024-05-08T05:15:37.101 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:15:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 137/164 2024-05-08T05:15:37.127 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:37.127 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-05-08T05:15:37.127 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:15:37.127 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-05-08T05:15:37.127 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:37.167 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:15:37.168 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:15:37.189 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:15:37.189 INFO:teuthology.orchestra.run.smithi143.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:37.189 INFO:teuthology.orchestra.run.smithi143.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:15:37.190 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:15:37.190 INFO:teuthology.orchestra.run.smithi143.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:15:37.190 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:37.192 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3538.gbee29 138/164 2024-05-08T05:15:37.193 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:37.193 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-05-08T05:15:37.193 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:37.252 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:15:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 139/164 2024-05-08T05:15:37.277 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:37.277 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-05-08T05:15:37.277 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:15:37.277 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-05-08T05:15:37.277 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:37.595 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:15:37.619 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 140/164 2024-05-08T05:15:37.619 INFO:teuthology.orchestra.run.smithi059.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-05-08T05:15:37.620 INFO:teuthology.orchestra.run.smithi059.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-05-08T05:15:37.620 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:15:37.620 INFO:teuthology.orchestra.run.smithi059.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-05-08T05:15:37.620 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:37.781 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:15:37.822 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:15:37.862 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:15:37.894 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:15:37.922 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:15:38.103 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:15:38.151 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:15:38.181 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:15:38.211 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:15:38.249 INFO:teuthology.orchestra.run.smithi096.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:15:38.871 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:15:38.913 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:15:38.933 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:15:39.756 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:15:39.851 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:15:39.894 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:15:39.972 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:15:40.012 INFO:teuthology.orchestra.run.smithi096.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:15:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:15:40.038 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:15:40.114 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:15:40.139 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:15:40.139 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:15:40.223 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:15:40.248 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:15:40.253 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:15:40.253 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:15:40.434 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:15:40.434 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:15:40.501 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:15:40.563 INFO:teuthology.orchestra.run.smithi096.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:15:41.803 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:15:41.845 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:15:41.886 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:15:41.918 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:15:41.945 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:15:42.132 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:15:42.174 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:15:42.204 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:15:42.234 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:15:42.280 INFO:teuthology.orchestra.run.smithi143.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:15:42.515 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 141/164 2024-05-08T05:15:42.567 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 142/164 2024-05-08T05:15:42.607 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 143/164 2024-05-08T05:15:42.638 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 144/164 2024-05-08T05:15:42.665 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 145/164 2024-05-08T05:15:42.853 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:15:42.873 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 146/164 2024-05-08T05:15:42.894 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:15:42.914 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:15:42.919 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 147/164 2024-05-08T05:15:42.941 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 148/164 2024-05-08T05:15:42.971 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-team-1:1.47.5-1.el9.x86_64 149/164 2024-05-08T05:15:43.017 INFO:teuthology.orchestra.run.smithi059.stdout: Upgrading : NetworkManager-tui-1:1.47.5-1.el9.x86_64 150/164 2024-05-08T05:15:43.586 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:15:43.623 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:15:43.645 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 151/164 2024-05-08T05:15:43.749 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:15:43.840 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:15:43.881 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:15:43.982 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:15:44.009 INFO:teuthology.orchestra.run.smithi143.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:15:44.035 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:15:44.035 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:15:44.103 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:15:44.144 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:15:44.144 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:15:44.237 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:15:44.262 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:15:44.267 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:15:44.267 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:15:44.455 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:15:44.455 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:15:44.539 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:15:44.562 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 152/164 2024-05-08T05:15:44.612 INFO:teuthology.orchestra.run.smithi143.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:15:44.652 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : openssl-devel-1:3.2.1-1.el9.x86_64 153/164 2024-05-08T05:15:44.711 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 154/164 2024-05-08T05:15:44.797 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : flex-2.6.4-9.el9.x86_64 155/164 2024-05-08T05:15:44.831 INFO:teuthology.orchestra.run.smithi059.stdout: Installing : bison-3.7.4-5.el9.x86_64 156/164 2024-05-08T05:15:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-tui-1:1.47.4-1.el9.x86_64 157/164 2024-05-08T05:15:44.865 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:15:44.949 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 158/164 2024-05-08T05:15:44.982 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-1:3.0.7-27.el9.x86_64 159/164 2024-05-08T05:15:44.982 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:15:45.067 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/164 2024-05-08T05:15:45.091 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-team-1:1.47.4-1.el9.x86_64 161/164 2024-05-08T05:15:45.097 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:15:45.097 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:15:45.294 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-1:1.47.4-1.el9.x86_64 162/164 2024-05-08T05:15:45.294 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:15:45.361 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: NetworkManager-libnm-1:1.47.4-1.el9.x86_64 163/164 2024-05-08T05:15:45.429 INFO:teuthology.orchestra.run.smithi059.stdout: Cleanup : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:15:46.338 INFO:teuthology.orchestra.run.smithi096.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:15:46.338 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:15:46.339 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:15:46.340 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:15:46.341 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:15:46.342 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:15:46.343 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:15:46.344 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:15:46.345 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:15:46.346 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:15:46.347 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:15:46.348 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:15:46.349 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:15:46.349 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:15:46.349 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:15:46.349 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:15:46.349 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:15:46.349 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:15:46.350 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:15:46.351 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:15:46.351 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:15:46.351 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:15:46.351 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:15:46.351 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:15:49.036 INFO:teuthology.orchestra.run.smithi096.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:15:49.036 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout:Upgraded: 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout:Installed: 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:15:49.037 INFO:teuthology.orchestra.run.smithi096.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.038 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:15:49.039 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:15:49.040 INFO:teuthology.orchestra.run.smithi096.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:15:49.041 INFO:teuthology.orchestra.run.smithi096.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.042 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:15:49.043 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:15:49.044 INFO:teuthology.orchestra.run.smithi096.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.045 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:15:49.046 INFO:teuthology.orchestra.run.smithi096.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:15:49.047 INFO:teuthology.orchestra.run.smithi096.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:15:49.048 INFO:teuthology.orchestra.run.smithi096.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:15:49.048 INFO:teuthology.orchestra.run.smithi096.stdout: 2024-05-08T05:15:49.048 INFO:teuthology.orchestra.run.smithi096.stdout:Complete! 2024-05-08T05:15:49.448 DEBUG:teuthology.parallel:result is None 2024-05-08T05:15:50.183 INFO:teuthology.orchestra.run.smithi143.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:15:50.183 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:15:50.183 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:15:50.183 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:15:50.184 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:15:50.185 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:15:50.186 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:15:50.186 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:15:50.186 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:15:50.186 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:15:50.186 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:15:50.186 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:15:50.186 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:15:50.205 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:15:50.205 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:15:50.205 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:15:50.205 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:15:50.205 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:15:50.205 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:15:50.205 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:15:50.205 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:15:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:15:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:15:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:15:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:15:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:15:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:15:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:15:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:15:50.206 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:15:50.207 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:15:50.207 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:15:50.207 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:15:50.207 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:15:50.207 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:15:50.207 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:15:50.207 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:15:50.207 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:15:50.207 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:15:50.208 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:15:50.208 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:15:50.208 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:15:50.208 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:15:50.208 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:15:50.208 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:15:50.208 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:15:50.208 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:15:50.208 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:15:50.209 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:15:50.209 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:15:50.209 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:15:50.209 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:15:50.209 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:15:50.209 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:15:50.209 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:15:50.209 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:15:50.209 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:15:50.210 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:15:50.210 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:15:50.210 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:15:50.210 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:15:50.210 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:15:50.210 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:15:50.210 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:15:50.210 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:15:50.211 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:15:50.212 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:15:50.212 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:15:50.212 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:15:50.212 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:15:50.212 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:15:50.212 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:15:50.212 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:15:50.212 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:15:50.212 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:15:50.213 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:15:50.213 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:15:50.213 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:15:50.213 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:15:50.213 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:15:50.213 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:15:50.213 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:15:50.213 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:15:50.213 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:15:50.214 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:15:50.214 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:15:50.214 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:15:50.214 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:15:50.214 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:15:50.214 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:15:50.214 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:15:50.214 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:15:50.214 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:15:50.215 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:15:50.215 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:15:50.215 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:15:50.215 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:15:50.215 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:15:50.215 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:15:50.215 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:15:50.215 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:15:50.215 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:15:50.216 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:15:50.216 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:15:50.216 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:15:50.216 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:15:50.216 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:15:50.216 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:15:50.216 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:15:50.216 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:15:50.216 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:15:50.217 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:15:50.217 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:15:50.217 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:15:50.217 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:15:50.217 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:15:50.217 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:15:50.217 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:15:50.217 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:15:50.217 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:15:50.218 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:15:50.218 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:15:50.218 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:15:50.218 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:15:50.218 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:15:50.218 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:15:50.218 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:15:50.218 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:15:50.218 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:15:50.219 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:15:50.219 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:15:50.219 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:15:50.219 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:15:50.219 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:15:50.219 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:15:50.219 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:15:50.219 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Running scriptlet: openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 1/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 3/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 4/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3538.gbee29 5/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 6/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 7/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 8/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 9/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 10/164 2024-05-08T05:15:50.811 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 11/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 12/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 13/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 14/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_ 15/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 16/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 17/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_6 18/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86 19/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 20/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el 21/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ceph-common-2:19.0.0-3538.gbee299f4.el9. 22/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_6 23/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 24/164 2024-05-08T05:15:50.812 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 25/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 26/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 27/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 28/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 29/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4. 30/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noa 31/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.n 32/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee 33/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el 34/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 35/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.e 36/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 37/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 38/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 39/164 2024-05-08T05:15:50.813 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/164 2024-05-08T05:15:50.814 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/164 2024-05-08T05:15:50.815 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-devel-1:3.2.1-1.el9.x86_64 68/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/164 2024-05-08T05:15:50.816 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/164 2024-05-08T05:15:50.817 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/164 2024-05-08T05:15:50.818 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 108/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 109/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 113/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/164 2024-05-08T05:15:50.819 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 122/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/164 2024-05-08T05:15:50.820 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/164 2024-05-08T05:15:50.821 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 149/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 151/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.47.5-1.el9.x86_64 153/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-1:1.47.4-1.el9.x86_64 154/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.47.5-1.el9.x86_64 155/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-libnm-1:1.47.4-1.el9.x86_64 156/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.47.5-1.el9.x86_64 157/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-team-1:1.47.4-1.el9.x86_64 158/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.47.5-1.el9.x86_64 159/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : NetworkManager-tui-1:1.47.4-1.el9.x86_64 160/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.2.1-1.el9.x86_64 161/164 2024-05-08T05:15:50.822 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-1:3.0.7-27.el9.x86_64 162/164 2024-05-08T05:15:50.823 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.2.1-1.el9.x86_64 163/164 2024-05-08T05:15:53.011 INFO:teuthology.orchestra.run.smithi143.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:15:53.012 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:53.012 INFO:teuthology.orchestra.run.smithi143.stdout:Upgraded: 2024-05-08T05:15:53.012 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:53.012 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:53.012 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:53.012 INFO:teuthology.orchestra.run.smithi143.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:53.012 INFO:teuthology.orchestra.run.smithi143.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.013 INFO:teuthology.orchestra.run.smithi143.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.013 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:15:53.013 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:15:53.013 INFO:teuthology.orchestra.run.smithi143.stdout:Installed: 2024-05-08T05:15:53.013 INFO:teuthology.orchestra.run.smithi143.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:15:53.013 INFO:teuthology.orchestra.run.smithi143.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:15:53.013 INFO:teuthology.orchestra.run.smithi143.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:15:53.013 INFO:teuthology.orchestra.run.smithi143.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:15:53.014 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.014 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.014 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.014 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.014 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.014 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.015 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.015 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.015 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.015 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.015 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.015 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.015 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.016 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.016 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.016 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.016 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.016 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.016 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.017 INFO:teuthology.orchestra.run.smithi143.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.017 INFO:teuthology.orchestra.run.smithi143.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.017 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.017 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.017 INFO:teuthology.orchestra.run.smithi143.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.017 INFO:teuthology.orchestra.run.smithi143.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:15:53.018 INFO:teuthology.orchestra.run.smithi143.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:15:53.018 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:15:53.018 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:15:53.018 INFO:teuthology.orchestra.run.smithi143.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:15:53.018 INFO:teuthology.orchestra.run.smithi143.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:15:53.018 INFO:teuthology.orchestra.run.smithi143.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.019 INFO:teuthology.orchestra.run.smithi143.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:15:53.020 INFO:teuthology.orchestra.run.smithi143.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.021 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:15:53.022 INFO:teuthology.orchestra.run.smithi143.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:15:53.023 INFO:teuthology.orchestra.run.smithi143.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:15:53.024 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:15:53.025 INFO:teuthology.orchestra.run.smithi143.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.026 INFO:teuthology.orchestra.run.smithi143.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:15:53.027 INFO:teuthology.orchestra.run.smithi143.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:15:53.027 INFO:teuthology.orchestra.run.smithi143.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:15:53.027 INFO:teuthology.orchestra.run.smithi143.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:15:53.027 INFO:teuthology.orchestra.run.smithi143.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:15:53.027 INFO:teuthology.orchestra.run.smithi143.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:15:53.027 INFO:teuthology.orchestra.run.smithi143.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:15:53.027 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:15:53.027 INFO:teuthology.orchestra.run.smithi143.stdout:Complete! 2024-05-08T05:15:53.292 INFO:teuthology.orchestra.run.smithi059.stdout: Verifying : openssl-libs-1:3.0.7-27.el9.x86_64 164/164 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout:Upgraded: 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-libnm-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-team-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: NetworkManager-tui-1:1.47.5-1.el9.x86_64 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: librados2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: librbd1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-1:3.2.1-1.el9.x86_64 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-libs-1:3.2.1-1.el9.x86_64 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout:Installed: 2024-05-08T05:15:53.293 INFO:teuthology.orchestra.run.smithi059.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: automake-1.16.2-8.el9.noarch 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: bison-3.7.4-5.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-base-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-grafana-dashboards-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-immutable-object-cache-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mds-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.294 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-dashboard-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-modules-core-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mgr-rook-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-mon-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-osd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-prometheus-alerts-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-radosgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-selinux-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-test-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: ceph-volume-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: cephadm-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-shell-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.295 INFO:teuthology.orchestra.run.smithi059.stdout: cephfs-top-2:19.0.0-3538.gbee299f4.el9.noarch 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: flex-2.6.4-9.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: fuse-2.9.9-15.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: grpc-data-1.46.7-10.el9.noarch 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-9.0.0-11.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: libcephfs2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.296 INFO:teuthology.orchestra.run.smithi059.stdout: libcephsqlite-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: liboath-2.6.7-2.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: librados-devel-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: libradosstriper1-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: librgw2-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: libtool-2.4.6-45.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: libunwind-1.6.2-1.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-05-08T05:15:53.297 INFO:teuthology.orchestra.run.smithi059.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: lua-5.4.4-4.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: luarocks-3.9.2-1.el9.noarch 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-0.3.21-2.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: openssl-devel-1:3.2.1-1.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-13.14-1.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: postgresql-server-13.14-1.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-3.14.0-13.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-05-08T05:15:53.298 INFO:teuthology.orchestra.run.smithi059.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-babel-2.9.1-2.el9.noarch 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-argparse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ceph-common-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cephfs-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-05-08T05:15:53.299 INFO:teuthology.orchestra.run.smithi059.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-05-08T05:15:53.300 INFO:teuthology.orchestra.run.smithi059.stdout: python3-mako-1.1.4-6.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-packaging-20.9-5.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-ply-3.11-14.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-portend-3.1.0-2.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-05-08T05:15:53.301 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pycparser-2.20-6.el9.noarch 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-pytz-2021.1-5.el9.noarch 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rados-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-2.25.1-8.el9.noarch 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-05-08T05:15:53.302 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rgw-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-routes-2.5.1-5.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-rsa-4.9-2.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-toml-0.10.2-6.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-webob-1.8.7-6.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-24.02.0-1.el9.x86_64 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-05-08T05:15:53.303 INFO:teuthology.orchestra.run.smithi059.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-05-08T05:15:53.304 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-fuse-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.304 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-mirror-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.304 INFO:teuthology.orchestra.run.smithi059.stdout: rbd-nbd-2:19.0.0-3538.gbee299f4.el9.x86_64 2024-05-08T05:15:53.304 INFO:teuthology.orchestra.run.smithi059.stdout: re2-1:20211101-3.el9.x86_64 2024-05-08T05:15:53.304 INFO:teuthology.orchestra.run.smithi059.stdout: socat-1.7.4.1-5.el9.x86_64 2024-05-08T05:15:53.304 INFO:teuthology.orchestra.run.smithi059.stdout: thrift-0.15.0-2.el9.x86_64 2024-05-08T05:15:53.304 INFO:teuthology.orchestra.run.smithi059.stdout: uuid-1.6.2-55.el9.x86_64 2024-05-08T05:15:53.304 INFO:teuthology.orchestra.run.smithi059.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-05-08T05:15:53.304 INFO:teuthology.orchestra.run.smithi059.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-05-08T05:15:53.305 INFO:teuthology.orchestra.run.smithi059.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-05-08T05:15:53.305 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:15:53.305 INFO:teuthology.orchestra.run.smithi059.stdout:Complete! 2024-05-08T05:15:53.429 DEBUG:teuthology.parallel:result is None 2024-05-08T05:15:53.675 DEBUG:teuthology.parallel:result is None 2024-05-08T05:15:53.675 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:15:53.825 DEBUG:teuthology.orchestra.run.smithi059:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:15:53.847 INFO:teuthology.orchestra.run.smithi059.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:15:53.848 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:15:53.848 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:15:53.850 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:15:53.994 DEBUG:teuthology.orchestra.run.smithi096:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:15:54.020 INFO:teuthology.orchestra.run.smithi096.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:15:54.020 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:15:54.020 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:15:54.023 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:15:54.243 DEBUG:teuthology.orchestra.run.smithi143:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-05-08T05:15:54.269 INFO:teuthology.orchestra.run.smithi143.stdout:19.0.0-3538.gbee299f4.el9 2024-05-08T05:15:54.270 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3538.gbee299f4.el9 2024-05-08T05:15:54.270 INFO:teuthology.task.install:The correct ceph version 19.0.0-3538.gbee299f4 is installed. 2024-05-08T05:15:54.272 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-05-08T05:15:54.272 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:15:54.272 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:15:54.302 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:15:54.302 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:15:54.335 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:15:54.335 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-05-08T05:15:54.365 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-05-08T05:15:54.365 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:15:54.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:15:54.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:15:54.461 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:15:54.461 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:15:54.488 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:15:54.555 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:15:54.555 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/daemon-helper 2024-05-08T05:15:54.584 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-05-08T05:15:54.652 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-05-08T05:15:54.652 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:15:54.653 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:15:54.680 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:15:54.748 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:15:54.748 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:15:54.776 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:15:54.842 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:15:54.842 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/adjust-ulimits 2024-05-08T05:15:54.870 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-05-08T05:15:54.938 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-05-08T05:15:54.938 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:15:54.938 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:15:54.966 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:15:55.033 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:15:55.034 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:15:55.061 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:15:55.128 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:15:55.129 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/usr/bin/stdin-killer 2024-05-08T05:15:55.157 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-05-08T05:15:55.224 INFO:teuthology.run_tasks:Running task cephadm... 2024-05-08T05:15:55.340 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_export_ephemeral_random_max': 0.1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore compression mode': 'aggressive', 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability'], 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-08T05:15:55.341 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:15:55.341 INFO:tasks.cephadm:Cluster fsid is 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:15:55.341 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-05-08T05:15:55.342 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.59', 'mon.b': '172.21.15.96', 'mon.c': '172.21.15.143'} 2024-05-08T05:15:55.342 INFO:tasks.cephadm:First mon is mon.a on smithi059 2024-05-08T05:15:55.342 INFO:tasks.cephadm:First mgr is x 2024-05-08T05:15:55.342 INFO:tasks.cephadm:Normalizing hostnames... 2024-05-08T05:15:55.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo hostname $(hostname -s) 2024-05-08T05:15:55.371 DEBUG:teuthology.orchestra.run.smithi096:> sudo hostname $(hostname -s) 2024-05-08T05:15:55.400 DEBUG:teuthology.orchestra.run.smithi143:> sudo hostname $(hostname -s) 2024-05-08T05:15:55.428 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-05-08T05:15:55.428 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:15:55.570 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/79166/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3538-gbee299f4', 'node_name': '172.21.5.39+adami09', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3538.gbee299f4'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-05-07 17:50:48.409840', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-57676-3', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-05-08T05:15:55.717 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-57676-3, sha1 bee299f4ac344e4c45e550d7a80abcadd7bc3d5a from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:15:55.718 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm 2024-05-08T05:15:55.718 DEBUG:teuthology.orchestra.run.smithi059:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:15:57.043 INFO:teuthology.orchestra.run.smithi059.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:15 /home/ubuntu/cephtest/cephadm 2024-05-08T05:15:57.044 DEBUG:teuthology.orchestra.run.smithi096:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:15:58.751 INFO:teuthology.orchestra.run.smithi096.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:15 /home/ubuntu/cephtest/cephadm 2024-05-08T05:15:58.752 DEBUG:teuthology.orchestra.run.smithi143:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-57676-3/bee299f4ac344e4c45e550d7a80abcadd7bc3d5a/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-05-08T05:16:00.055 INFO:teuthology.orchestra.run.smithi143.stdout:-rw-r--r--. 1 ubuntu ubuntu 782795 May 8 05:16 /home/ubuntu/cephtest/cephadm 2024-05-08T05:16:00.055 DEBUG:teuthology.orchestra.run.smithi059:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:16:00.076 DEBUG:teuthology.orchestra.run.smithi096:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:16:00.095 DEBUG:teuthology.orchestra.run.smithi143:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-05-08T05:16:00.131 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a on all hosts... 2024-05-08T05:16:00.131 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:16:00.134 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:16:00.138 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a pull 2024-05-08T05:16:01.742 INFO:teuthology.orchestra.run.smithi059.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:16:01.784 INFO:teuthology.orchestra.run.smithi096.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:16:01.806 INFO:teuthology.orchestra.run.smithi143.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:16:36.963 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-08T05:16:36.963 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:16:36.964 INFO:teuthology.orchestra.run.smithi143.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:16:36.964 INFO:teuthology.orchestra.run.smithi143.stdout: "repo_digests": [ 2024-05-08T05:16:36.964 INFO:teuthology.orchestra.run.smithi143.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:16:36.964 INFO:teuthology.orchestra.run.smithi143.stdout: ] 2024-05-08T05:16:36.964 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-08T05:16:48.359 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-08T05:16:48.359 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:16:48.359 INFO:teuthology.orchestra.run.smithi096.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:16:48.360 INFO:teuthology.orchestra.run.smithi096.stdout: "repo_digests": [ 2024-05-08T05:16:48.360 INFO:teuthology.orchestra.run.smithi096.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:16:48.360 INFO:teuthology.orchestra.run.smithi096.stdout: ] 2024-05-08T05:16:48.360 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-08T05:16:51.338 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:16:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:16:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "image_id": "352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce", 2024-05-08T05:16:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "repo_digests": [ 2024-05-08T05:16:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c" 2024-05-08T05:16:51.338 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:16:51.338 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:16:51.367 DEBUG:teuthology.orchestra.run.smithi059:> sudo mkdir -p /etc/ceph 2024-05-08T05:16:51.419 DEBUG:teuthology.orchestra.run.smithi096:> sudo mkdir -p /etc/ceph 2024-05-08T05:16:51.451 DEBUG:teuthology.orchestra.run.smithi143:> sudo mkdir -p /etc/ceph 2024-05-08T05:16:51.484 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 777 /etc/ceph 2024-05-08T05:16:51.512 DEBUG:teuthology.orchestra.run.smithi096:> sudo chmod 777 /etc/ceph 2024-05-08T05:16:51.539 DEBUG:teuthology.orchestra.run.smithi143:> sudo chmod 777 /etc/ceph 2024-05-08T05:16:51.567 INFO:tasks.cephadm:Writing seed config... 2024-05-08T05:16:51.568 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-05-08T05:16:51.568 INFO:tasks.cephadm: override: [client] debug client = 20 2024-05-08T05:16:51.568 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-05-08T05:16:51.568 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-05-08T05:16:51.568 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] mds_export_ephemeral_random_max = 0.1 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-05-08T05:16:51.569 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [osd] bluestore compression mode = aggressive 2024-05-08T05:16:51.570 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-05-08T05:16:51.571 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-05-08T05:16:51.572 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:16:51.572 DEBUG:teuthology.orchestra.run.smithi059:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-05-08T05:16:51.590 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = 0bf01a10-0cfa-11ef-bc97-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore compression mode = aggressive bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_export_ephemeral_random_max = 0.1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-05-08T05:16:51.590 DEBUG:teuthology.orchestra.run.smithi059:mon.a> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.a.service 2024-05-08T05:16:51.633 DEBUG:teuthology.orchestra.run.smithi059:mgr.x> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.x.service 2024-05-08T05:16:51.676 INFO:tasks.cephadm:Bootstrapping... 2024-05-08T05:16:51.677 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a -v bootstrap --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.59 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:16:51.878 INFO:teuthology.orchestra.run.smithi059.stdout:-------------------------------------------------------------------------------- 2024-05-08T05:16:51.878 INFO:teuthology.orchestra.run.smithi059.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a', '-v', 'bootstrap', '--fsid', '0bf01a10-0cfa-11ef-bc97-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.59', '--skip-admin-label'] 2024-05-08T05:16:51.878 INFO:teuthology.orchestra.run.smithi059.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-05-08T05:16:51.878 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying podman|docker is present... 2024-05-08T05:16:51.909 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T05:16:51.909 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying lvm2 is present... 2024-05-08T05:16:51.909 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying time synchronization is in place... 2024-05-08T05:16:51.917 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T05:16:51.917 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T05:16:51.923 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T05:16:51.924 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-05-08T05:16:51.931 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout enabled 2024-05-08T05:16:51.937 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout active 2024-05-08T05:16:51.937 INFO:teuthology.orchestra.run.smithi059.stdout:Unit chronyd.service is enabled and running 2024-05-08T05:16:51.937 INFO:teuthology.orchestra.run.smithi059.stdout:Repeating the final host check... 2024-05-08T05:16:51.962 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 4.9.4-dev 2024-05-08T05:16:51.962 INFO:teuthology.orchestra.run.smithi059.stdout:podman (/bin/podman) version 4.9.4 is present 2024-05-08T05:16:51.962 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl is present 2024-05-08T05:16:51.962 INFO:teuthology.orchestra.run.smithi059.stdout:lvcreate is present 2024-05-08T05:16:51.969 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-05-08T05:16:51.969 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-05-08T05:16:51.975 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-05-08T05:16:51.975 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-05-08T05:16:51.982 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout enabled 2024-05-08T05:16:51.989 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout active 2024-05-08T05:16:51.989 INFO:teuthology.orchestra.run.smithi059.stdout:Unit chronyd.service is enabled and running 2024-05-08T05:16:51.989 INFO:teuthology.orchestra.run.smithi059.stdout:Host looks OK 2024-05-08T05:16:51.989 INFO:teuthology.orchestra.run.smithi059.stdout:Cluster fsid: 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:16:51.989 INFO:teuthology.orchestra.run.smithi059.stdout:Acquiring lock 140227492519360 on /run/cephadm/0bf01a10-0cfa-11ef-bc97-c7b262605968.lock 2024-05-08T05:16:51.989 INFO:teuthology.orchestra.run.smithi059.stdout:Lock 140227492519360 acquired on /run/cephadm/0bf01a10-0cfa-11ef-bc97-c7b262605968.lock 2024-05-08T05:16:51.990 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying IP 172.21.15.59 port 3300 ... 2024-05-08T05:16:51.990 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying IP 172.21.15.59 port 6789 ... 2024-05-08T05:16:51.990 INFO:teuthology.orchestra.run.smithi059.stdout:Base mon IP(s) is [172.21.15.59:3300, 172.21.15.59:6789], mon addrv is [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2024-05-08T05:16:51.993 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.59 metric 100 2024-05-08T05:16:51.994 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.59 metric 100 2024-05-08T05:16:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-05-08T05:16:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 1024 pref medium 2024-05-08T05:16:51.996 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 100 pref medium 2024-05-08T05:16:51.999 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-05-08T05:16:51.999 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-05-08T05:16:51.999 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T05:16:51.999 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-05-08T05:16:51.999 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-05-08T05:16:51.999 INFO:teuthology.orchestra.run.smithi059.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-05-08T05:16:52.000 INFO:teuthology.orchestra.run.smithi059.stdout:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2024-05-08T05:16:52.000 INFO:teuthology.orchestra.run.smithi059.stdout:Mon IP `172.21.15.59` is in CIDR network `172.21.0.0/20` 2024-05-08T05:16:52.000 INFO:teuthology.orchestra.run.smithi059.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-05-08T05:16:52.000 INFO:teuthology.orchestra.run.smithi059.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-05-08T05:16:52.001 INFO:teuthology.orchestra.run.smithi059.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:16:52.481 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stdout 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce 2024-05-08T05:16:52.481 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a... 2024-05-08T05:16:52.481 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Getting image source signatures 2024-05-08T05:16:52.481 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying blob sha256:bf5b95bd68251da39c6cd2eb96c6ba4138bb37a62bdc39287908b25e92047d7e 2024-05-08T05:16:52.481 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying blob sha256:f08a3efca46759c0efc03dda6bb4c9adf7daf77004822005f2f0da3e8d1877bb 2024-05-08T05:16:52.481 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Copying config sha256:352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce 2024-05-08T05:16:52.481 INFO:teuthology.orchestra.run.smithi059.stdout:/bin/podman: stderr Writing manifest to image destination 2024-05-08T05:16:53.421 INFO:teuthology.orchestra.run.smithi059.stdout:ceph: stdout ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:16:53.421 INFO:teuthology.orchestra.run.smithi059.stdout:Ceph version: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev) 2024-05-08T05:16:53.421 INFO:teuthology.orchestra.run.smithi059.stdout:Extracting ceph user uid/gid from container image... 2024-05-08T05:16:54.294 INFO:teuthology.orchestra.run.smithi059.stdout:stat: stdout 167 167 2024-05-08T05:16:54.294 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial keys... 2024-05-08T05:16:55.327 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQDGCjtm+leHJhAAckOYXxr6h7l9bs7Exb9fFQ== 2024-05-08T05:16:56.348 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQDHCjtmWDIIJxAAbf2IEKBhvrtwO5sJ6/jMUg== 2024-05-08T05:16:57.212 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph-authtool: stdout AQDICjtme7zPJxAAX3u3Ol6NBmt3gTMKlyIEwA== 2024-05-08T05:16:57.213 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial monmap... 2024-05-08T05:16:58.126 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T05:16:58.127 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-05-08T05:16:58.127 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:16:58.127 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T05:16:58.127 INFO:teuthology.orchestra.run.smithi059.stdout:monmaptool for a [v2:172.21.15.59:3300,v1:172.21.15.59:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-05-08T05:16:58.127 INFO:teuthology.orchestra.run.smithi059.stdout:setting min_mon_release = quincy 2024-05-08T05:16:58.127 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: set fsid to 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:16:58.127 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-05-08T05:16:58.127 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:16:58.127 INFO:teuthology.orchestra.run.smithi059.stdout:Creating mon... 2024-05-08T05:16:59.057 INFO:teuthology.orchestra.run.smithi059.stdout:create mon.a on 2024-05-08T05:16:59.284 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-05-08T05:16:59.428 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-05-08T05:16:59.609 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968.target. 2024-05-08T05:16:59.610 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968.target → /etc/systemd/system/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968.target. 2024-05-08T05:16:59.850 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.a 2024-05-08T05:16:59.850 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.a.service: Unit ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.a.service not loaded. 2024-05-08T05:17:00.017 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968.target.wants/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.a.service → /etc/systemd/system/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@.service. 2024-05-08T05:17:00.616 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:17:00.616 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-05-08T05:17:00.623 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:17:00.624 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-05-08T05:17:00.624 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-05-08T05:17:00.624 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T05:17:00.624 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mon to start... 2024-05-08T05:17:00.624 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mon... 2024-05-08T05:17:01.724 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:01 smithi059 ceph-mon[29921]: from='client.? 172.21.15.59:0/326047287' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-05-08T05:17:01.726 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout cluster: 2024-05-08T05:17:01.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout id: 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:17:01.732 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout services: 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.616581s) 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout data: 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout pgs: 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:01.733 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.054+0000 7fb6bd5e2700 1 Processor -- start 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.054+0000 7fb6bd5e2700 1 -- start start 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.055+0000 7fb6bd5e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b810bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.055+0000 7fb6bd5e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b807ce50 con 0x7fb6b8109900 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.055+0000 7fb6b6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b810bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.055+0000 7fb6b6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b810bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60792/0 (socket says 172.21.15.59:60792) 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.055+0000 7fb6b6ffd700 1 -- 172.21.15.59:0/2488089708 learned_addr learned my addr 172.21.15.59:0/2488089708 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.055+0000 7fb6b6ffd700 1 -- 172.21.15.59:0/2488089708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b810c230 con 0x7fb6b8109900 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.055+0000 7fb6b6ffd700 1 --2- 172.21.15.59:0/2488089708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b810bcf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098a0 tx=0x7fb6a8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4024a949517a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6b5ffb700 1 -- 172.21.15.59:0/2488089708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8004160 con 0x7fb6b8109900 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6b5ffb700 1 -- 172.21.15.59:0/2488089708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a8004300 con 0x7fb6b8109900 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6b5ffb700 1 -- 172.21.15.59:0/2488089708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a80036a0 con 0x7fb6b8109900 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/2488089708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 msgr2=0x7fb6b810bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:01.734 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6bd5e2700 1 --2- 172.21.15.59:0/2488089708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b810bcf0 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fb6a80098a0 tx=0x7fb6a8009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/2488089708 shutdown_connections 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6bd5e2700 1 --2- 172.21.15.59:0/2488089708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b810bcf0 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/2488089708 >> 172.21.15.59:0/2488089708 conn(0x7fb6b81032b0 msgr2=0x7fb6b8105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/2488089708 shutdown_connections 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.056+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/2488089708 wait complete. 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.057+0000 7fb6bd5e2700 1 Processor -- start 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.057+0000 7fb6bd5e2700 1 -- start start 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.057+0000 7fb6bd5e2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b819c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.057+0000 7fb6bd5e2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b810c6a0 con 0x7fb6b8109900 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.057+0000 7fb6b6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b819c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.057+0000 7fb6b6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b819c5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60806/0 (socket says 172.21.15.59:60806) 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.057+0000 7fb6b6ffd700 1 -- 172.21.15.59:0/326047287 learned_addr learned my addr 172.21.15.59:0/326047287 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:01.735 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.058+0000 7fb6b6ffd700 1 -- 172.21.15.59:0/326047287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b819caf0 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.058+0000 7fb6b6ffd700 1 --2- 172.21.15.59:0/326047287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b819c5b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8000c00 tx=0x7fb6a8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.058+0000 7fb69ffff700 1 -- 172.21.15.59:0/326047287 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8003db0 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.058+0000 7fb69ffff700 1 -- 172.21.15.59:0/326047287 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb6a8011630 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.058+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/326047287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b819cd80 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.058+0000 7fb69ffff700 1 -- 172.21.15.59:0/326047287 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb6a8011950 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.058+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/326047287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b819d8a0 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.058+0000 7fb69ffff700 1 -- 172.21.15.59:0/326047287 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fb6a801a460 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.059+0000 7fb69ffff700 1 -- 172.21.15.59:0/326047287 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb6a801b780 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.059+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/326047287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b8064b00 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.062+0000 7fb69ffff700 1 -- 172.21.15.59:0/326047287 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fb6a801b980 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.135+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/326047287 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fb6b819e470 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.135+0000 7fb69ffff700 1 -- 172.21.15.59:0/326047287 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fb6a801bb60 con 0x7fb6b8109900 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.136+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/326047287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 msgr2=0x7fb6b819c5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:01.736 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.136+0000 7fb6bd5e2700 1 --2- 172.21.15.59:0/326047287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b819c5b0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fb6a8000c00 tx=0x7fb6a8004750 comp rx=0 tx=0).stop 2024-05-08T05:17:01.737 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.136+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/326047287 shutdown_connections 2024-05-08T05:17:01.737 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.136+0000 7fb6bd5e2700 1 --2- 172.21.15.59:0/326047287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb6b8109900 0x7fb6b819c5b0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:01.737 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.136+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/326047287 >> 172.21.15.59:0/326047287 conn(0x7fb6b81032b0 msgr2=0x7fb6b8104150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:01.737 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.136+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/326047287 shutdown_connections 2024-05-08T05:17:01.737 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:01.136+0000 7fb6bd5e2700 1 -- 172.21.15.59:0/326047287 wait complete. 2024-05-08T05:17:01.737 INFO:teuthology.orchestra.run.smithi059.stdout:mon is available 2024-05-08T05:17:01.737 INFO:teuthology.orchestra.run.smithi059.stdout:Assimilating anything we can from ceph.conf... 2024-05-08T05:17:02.772 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:02.772 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout fsid = 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.59:3300,v1:172.21.15.59:6789] 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:02.773 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.162+0000 7fc7f11af700 1 Processor -- start 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.162+0000 7fc7f11af700 1 -- start start 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.162+0000 7fc7f11af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec10bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.162+0000 7fc7f11af700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec07ce90 con 0x7fc7ec1098c0 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.163+0000 7fc7ead9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec10bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.163+0000 7fc7ead9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec10bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60822/0 (socket says 172.21.15.59:60822) 2024-05-08T05:17:02.774 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.163+0000 7fc7ead9d700 1 -- 172.21.15.59:0/1337838695 learned_addr learned my addr 172.21.15.59:0/1337838695 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.163+0000 7fc7ead9d700 1 -- 172.21.15.59:0/1337838695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec10c1f0 con 0x7fc7ec1098c0 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.163+0000 7fc7ead9d700 1 --2- 172.21.15.59:0/1337838695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec10bcb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0098a0 tx=0x7fc7dc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a6938bd3a752516 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.163+0000 7fc7ea59c700 1 -- 172.21.15.59:0/1337838695 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc004160 con 0x7fc7ec1098c0 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.163+0000 7fc7ea59c700 1 -- 172.21.15.59:0/1337838695 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7dc004300 con 0x7fc7ec1098c0 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.163+0000 7fc7ea59c700 1 -- 172.21.15.59:0/1337838695 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc0036a0 con 0x7fc7ec1098c0 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.164+0000 7fc7f11af700 1 -- 172.21.15.59:0/1337838695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 msgr2=0x7fc7ec10bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.164+0000 7fc7f11af700 1 --2- 172.21.15.59:0/1337838695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec10bcb0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc0098a0 tx=0x7fc7dc009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.164+0000 7fc7f11af700 1 -- 172.21.15.59:0/1337838695 shutdown_connections 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.164+0000 7fc7f11af700 1 --2- 172.21.15.59:0/1337838695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec10bcb0 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.164+0000 7fc7f11af700 1 -- 172.21.15.59:0/1337838695 >> 172.21.15.59:0/1337838695 conn(0x7fc7ec1032b0 msgr2=0x7fc7ec105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.164+0000 7fc7f11af700 1 -- 172.21.15.59:0/1337838695 shutdown_connections 2024-05-08T05:17:02.775 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.164+0000 7fc7f11af700 1 -- 172.21.15.59:0/1337838695 wait complete. 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.165+0000 7fc7f11af700 1 Processor -- start 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.165+0000 7fc7f11af700 1 -- start start 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.165+0000 7fc7f11af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec109320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.165+0000 7fc7f11af700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc7ec10c6c0 con 0x7fc7ec1098c0 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.165+0000 7fc7ead9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec109320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.165+0000 7fc7ead9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec109320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60834/0 (socket says 172.21.15.59:60834) 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.165+0000 7fc7ead9d700 1 -- 172.21.15.59:0/1817582829 learned_addr learned my addr 172.21.15.59:0/1817582829 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.166+0000 7fc7ead9d700 1 -- 172.21.15.59:0/1817582829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc7ec107870 con 0x7fc7ec1098c0 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.166+0000 7fc7ead9d700 1 --2- 172.21.15.59:0/1817582829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec109320 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc000c00 tx=0x7fc7dc004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.166+0000 7fc7e8d99700 1 -- 172.21.15.59:0/1817582829 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc003db0 con 0x7fc7ec1098c0 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.166+0000 7fc7e8d99700 1 -- 172.21.15.59:0/1817582829 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc7dc011630 con 0x7fc7ec1098c0 2024-05-08T05:17:02.776 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.166+0000 7fc7f11af700 1 -- 172.21.15.59:0/1817582829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7ec107b00 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.166+0000 7fc7e8d99700 1 -- 172.21.15.59:0/1817582829 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc7dc011950 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.166+0000 7fc7f11af700 1 -- 172.21.15.59:0/1817582829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7ec107fe0 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.166+0000 7fc7e8d99700 1 -- 172.21.15.59:0/1817582829 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fc7dc01a460 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.166+0000 7fc7e8d99700 1 -- 172.21.15.59:0/1817582829 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc7dc01bb40 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.167+0000 7fc7f11af700 1 -- 172.21.15.59:0/1817582829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc7ec064b00 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.169+0000 7fc7e8d99700 1 -- 172.21.15.59:0/1817582829 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7fc7dc004260 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.240+0000 7fc7f11af700 1 -- 172.21.15.59:0/1817582829 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fc7ec06bcb0 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.243+0000 7fc7e8d99700 1 -- 172.21.15.59:0/1817582829 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fc7dc01bd40 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.244+0000 7fc7e8d99700 1 -- 172.21.15.59:0/1817582829 <== mon.0 v2:172.21.15.59:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc7dc01ad70 con 0x7fc7ec1098c0 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.245+0000 7fc7f11af700 1 -- 172.21.15.59:0/1817582829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 msgr2=0x7fc7ec109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.245+0000 7fc7f11af700 1 --2- 172.21.15.59:0/1817582829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec109320 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fc7dc000c00 tx=0x7fc7dc004750 comp rx=0 tx=0).stop 2024-05-08T05:17:02.777 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.245+0000 7fc7f11af700 1 -- 172.21.15.59:0/1817582829 shutdown_connections 2024-05-08T05:17:02.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.245+0000 7fc7f11af700 1 --2- 172.21.15.59:0/1817582829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7ec1098c0 0x7fc7ec109320 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:02.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.245+0000 7fc7f11af700 1 -- 172.21.15.59:0/1817582829 >> 172.21.15.59:0/1817582829 conn(0x7fc7ec1032b0 msgr2=0x7fc7ec104160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:02.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.245+0000 7fc7f11af700 1 -- 172.21.15.59:0/1817582829 shutdown_connections 2024-05-08T05:17:02.778 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:02.245+0000 7fc7f11af700 1 -- 172.21.15.59:0/1817582829 wait complete. 2024-05-08T05:17:02.778 INFO:teuthology.orchestra.run.smithi059.stdout:Generating new minimal ceph.conf... 2024-05-08T05:17:02.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:02 smithi059 ceph-mon[29921]: from='client.? 172.21.15.59:0/1817582829' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T05:17:02.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:02 smithi059 ceph-mon[29921]: from='client.? 172.21.15.59:0/1817582829' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-05-08T05:17:03.810 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:03 smithi059 ceph-mon[29921]: from='client.? 172.21.15.59:0/3113471163' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:17:03.819 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.188+0000 7f680070f700 1 Processor -- start 2024-05-08T05:17:03.819 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.188+0000 7f680070f700 1 -- start start 2024-05-08T05:17:03.819 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.189+0000 7f680070f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f8109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:03.819 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.189+0000 7f680070f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67f8109790 con 0x7f67f8108e30 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.189+0000 7f67fe4ab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f8109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.189+0000 7f67fe4ab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f8109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60848/0 (socket says 172.21.15.59:60848) 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.189+0000 7f67fe4ab700 1 -- 172.21.15.59:0/1748768210 learned_addr learned my addr 172.21.15.59:0/1748768210 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.189+0000 7f67fe4ab700 1 -- 172.21.15.59:0/1748768210 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f8109f90 con 0x7f67f8108e30 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.189+0000 7f67fe4ab700 1 --2- 172.21.15.59:0/1748768210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f8109250 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67f40098a0 tx=0x7f67f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19af8ec0c2cc62ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f67fd4a9700 1 -- 172.21.15.59:0/1748768210 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f4004160 con 0x7f67f8108e30 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f67fd4a9700 1 -- 172.21.15.59:0/1748768210 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f67f4004300 con 0x7f67f8108e30 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f67fd4a9700 1 -- 172.21.15.59:0/1748768210 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f4011af0 con 0x7f67f8108e30 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f680070f700 1 -- 172.21.15.59:0/1748768210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 msgr2=0x7f67f8109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f680070f700 1 --2- 172.21.15.59:0/1748768210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f8109250 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f67f40098a0 tx=0x7f67f4009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f680070f700 1 -- 172.21.15.59:0/1748768210 shutdown_connections 2024-05-08T05:17:03.820 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f680070f700 1 --2- 172.21.15.59:0/1748768210 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f8109250 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f680070f700 1 -- 172.21.15.59:0/1748768210 >> 172.21.15.59:0/1748768210 conn(0x7f67f8104410 msgr2=0x7f67f8106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f680070f700 1 -- 172.21.15.59:0/1748768210 shutdown_connections 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.190+0000 7f680070f700 1 -- 172.21.15.59:0/1748768210 wait complete. 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.191+0000 7f680070f700 1 Processor -- start 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.191+0000 7f680070f700 1 -- start start 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.191+0000 7f680070f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f819ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.191+0000 7f680070f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67f810d3d0 con 0x7f67f8108e30 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.191+0000 7f67fe4ab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f819ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.191+0000 7f67fe4ab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f819ce20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60856/0 (socket says 172.21.15.59:60856) 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.191+0000 7f67fe4ab700 1 -- 172.21.15.59:0/3113471163 learned_addr learned my addr 172.21.15.59:0/3113471163 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.192+0000 7f67fe4ab700 1 -- 172.21.15.59:0/3113471163 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67f819d360 con 0x7f67f8108e30 2024-05-08T05:17:03.821 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.192+0000 7f67fe4ab700 1 --2- 172.21.15.59:0/3113471163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f819ce20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67f4003f40 tx=0x7f67f4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.192+0000 7f67eb7fe700 1 -- 172.21.15.59:0/3113471163 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f40038b0 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.192+0000 7f67eb7fe700 1 -- 172.21.15.59:0/3113471163 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f67f4003a50 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.192+0000 7f680070f700 1 -- 172.21.15.59:0/3113471163 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67f819d5f0 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.192+0000 7f67eb7fe700 1 -- 172.21.15.59:0/3113471163 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67f401a5c0 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.192+0000 7f680070f700 1 -- 172.21.15.59:0/3113471163 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67f819dfd0 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.192+0000 7f67eb7fe700 1 -- 172.21.15.59:0/3113471163 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f67f401ac50 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.192+0000 7f67eb7fe700 1 -- 172.21.15.59:0/3113471163 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f67f4019700 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.193+0000 7f680070f700 1 -- 172.21.15.59:0/3113471163 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67f8064b00 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.195+0000 7f67eb7fe700 1 -- 172.21.15.59:0/3113471163 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f67f4019900 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.266+0000 7f680070f700 1 -- 172.21.15.59:0/3113471163 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f67f819e2c0 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.267+0000 7f67eb7fe700 1 -- 172.21.15.59:0/3113471163 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f67f4019ae0 con 0x7f67f8108e30 2024-05-08T05:17:03.822 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.267+0000 7f680070f700 1 -- 172.21.15.59:0/3113471163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 msgr2=0x7f67f819ce20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:03.823 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.267+0000 7f680070f700 1 --2- 172.21.15.59:0/3113471163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f819ce20 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f67f4003f40 tx=0x7f67f4011890 comp rx=0 tx=0).stop 2024-05-08T05:17:03.823 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.267+0000 7f680070f700 1 -- 172.21.15.59:0/3113471163 shutdown_connections 2024-05-08T05:17:03.823 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.267+0000 7f680070f700 1 --2- 172.21.15.59:0/3113471163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f67f8108e30 0x7f67f819ce20 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:03.823 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.267+0000 7f680070f700 1 -- 172.21.15.59:0/3113471163 >> 172.21.15.59:0/3113471163 conn(0x7f67f8104410 msgr2=0x7f67f8104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:03.823 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.268+0000 7f680070f700 1 -- 172.21.15.59:0/3113471163 shutdown_connections 2024-05-08T05:17:03.823 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:03.268+0000 7f680070f700 1 -- 172.21.15.59:0/3113471163 wait complete. 2024-05-08T05:17:03.823 INFO:teuthology.orchestra.run.smithi059.stdout:Restarting the monitor... 2024-05-08T05:17:04.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:03 smithi059 systemd[1]: Stopping Ceph mon.a for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:17:04.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a[29913]: 2024-05-08T05:17:04.087+0000 7fbbc018a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:17:04.497 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a[29913]: 2024-05-08T05:17:04.087+0000 7fbbc018a700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-05-08T05:17:04.497 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:04 smithi059 podman[30174]: 2024-05-08 05:17:04.231964555 +0000 UTC m=+0.239986749 container died 5a69a2226fa8e85cb1ed127fd1847fd413baa2cb9dcb2f8f1f38bfac229dae72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, ceph=True, io.buildah.version=1.33.5) 2024-05-08T05:17:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:04 smithi059 podman[30174]: 2024-05-08 05:17:04.596443455 +0000 UTC m=+0.604465652 container cleanup 5a69a2226fa8e85cb1ed127fd1847fd413baa2cb9dcb2f8f1f38bfac229dae72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5) 2024-05-08T05:17:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:04 smithi059 bash[30174]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a 2024-05-08T05:17:05.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:04 smithi059 podman[30189]: 2024-05-08 05:17:04.838686036 +0000 UTC m=+0.598975062 container remove 5a69a2226fa8e85cb1ed127fd1847fd413baa2cb9dcb2f8f1f38bfac229dae72 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD) 2024-05-08T05:17:05.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:04 smithi059 systemd[1]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.a.service: Deactivated successfully. 2024-05-08T05:17:05.189 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:04 smithi059 systemd[1]: Stopped Ceph mon.a for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:17:05.190 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:04 smithi059 systemd[1]: Starting Ceph mon.a for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 podman[30285]: 2024-05-08 05:17:05.189581158 +0000 UTC m=+0.105857993 container create cc0f25ef8e855159af26d490b10ca789e8fb78095b2286346a16f8fa17343dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, ceph=True) 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 podman[30285]: 2024-05-08 05:17:05.103096591 +0000 UTC m=+0.019373400 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 podman[30285]: 2024-05-08 05:17:05.375028484 +0000 UTC m=+0.291305277 container init cc0f25ef8e855159af26d490b10ca789e8fb78095b2286346a16f8fa17343dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 podman[30285]: 2024-05-08 05:17:05.378179342 +0000 UTC m=+0.294456137 container start cc0f25ef8e855159af26d490b10ca789e8fb78095b2286346a16f8fa17343dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: set uid:gid to 167:167 (ceph:ceph) 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev), process ceph-mon, pid 2 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: pidfile_write: ignore empty --pid-file 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: load: jerasure load: lrc 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: RocksDB version: 7.9.2 2024-05-08T05:17:05.459 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Git sha 0 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Compile date 2024-05-07 11:18:56 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: DB SUMMARY 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: DB Session ID: 8ZZDQJX788USSPX78C9E 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: CURRENT file: CURRENT 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: IDENTITY file: IDENTITY 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 105098 ; 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.error_if_exists: 0 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.create_if_missing: 0 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.paranoid_checks: 1 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.flush_verify_memtable_count: 1 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-05-08T05:17:05.460 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.env: 0x55864122dcc0 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.fs: PosixFileSystem 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.info_log: 0x558642b0fda0 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_file_opening_threads: 16 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.statistics: (nil) 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.use_fsync: 0 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_log_file_size: 0 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.log_file_time_to_roll: 0 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.keep_log_file_num: 1000 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.recycle_log_file_num: 0 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.allow_fallocate: 1 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.allow_mmap_reads: 0 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.allow_mmap_writes: 0 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.use_direct_reads: 0 2024-05-08T05:17:05.461 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.create_missing_column_families: 0 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.db_log_dir: 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.wal_dir: 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.table_cache_numshardbits: 6 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.WAL_ttl_seconds: 0 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.WAL_size_limit_MB: 0 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.is_fd_close_on_exec: 1 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.advise_random_on_open: 1 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.db_write_buffer_size: 0 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.write_buffer_manager: 0x558642ae10e0 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-05-08T05:17:05.462 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.use_adaptive_mutex: 0 2024-05-08T05:17:05.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.rate_limiter: (nil) 2024-05-08T05:17:05.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-05-08T05:17:05.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.wal_recovery_mode: 2 2024-05-08T05:17:05.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.enable_thread_tracking: 0 2024-05-08T05:17:05.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.enable_pipelined_write: 0 2024-05-08T05:17:05.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.unordered_write: 0 2024-05-08T05:17:05.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-05-08T05:17:05.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-05-08T05:17:05.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-05-08T05:17:05.464 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.row_cache: None 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.wal_filter: None 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.allow_ingest_behind: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.two_write_queues: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.manual_wal_flush: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.wal_compression: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.atomic_flush: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.persist_stats_to_disk: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.write_dbid_to_manifest: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.log_readahead_size: 0 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-05-08T05:17:05.465 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.best_efforts_recovery: 0 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.allow_data_in_errors: 0 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.db_host_id: __hostname__ 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.enforce_single_del_contracts: true 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_background_jobs: 2 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_background_compactions: -1 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_subcompactions: 1 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.delayed_write_rate : 16777216 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_total_wal_size: 0 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-05-08T05:17:05.466 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.stats_dump_period_sec: 600 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.stats_persist_period_sec: 600 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_open_files: -1 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bytes_per_sync: 0 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.wal_bytes_per_sync: 0 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.strict_bytes_per_sync: 0 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_readahead_size: 0 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_background_flushes: -1 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Compression algorithms supported: 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: kZSTD supported: 0 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: kXpressCompression supported: 0 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: kLZ4HCCompression supported: 1 2024-05-08T05:17:05.467 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: kZlibCompression supported: 1 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: kSnappyCompression supported: 1 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: kLZ4Compression supported: 1 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: kBZip2Compression supported: 0 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Fast CRC32 supported: Supported on x86 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: DMutex implementation: pthread_mutex_t 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.merge_operator: 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_filter: None 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_filter_factory: None 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.sst_partitioner_factory: None 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.memtable_factory: SkipListFactory 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.table_factory: BlockBasedTable 2024-05-08T05:17:05.468 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558642b0ff60) 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: cache_index_and_filter_blocks: 1 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: pin_top_level_index_and_filter: 1 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_type: 0 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: data_block_index_type: 0 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_shortening: 1 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: data_block_hash_table_util_ratio: 0.750000 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: checksum: 4 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: no_block_cache: 0 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache: 0x558642b131f0 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_name: BinnedLRUCache 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_options: 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: capacity : 536870912 2024-05-08T05:17:05.469 INFO:journalctl@ceph.mon.a.smithi059.stdout: num_shard_bits : 4 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: strict_capacity_limit : 0 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: high_pri_pool_ratio: 0.000 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_cache_compressed: (nil) 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: persistent_cache: (nil) 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_size: 4096 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_size_deviation: 10 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_restart_interval: 16 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: index_block_restart_interval: 1 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: metadata_block_size: 4096 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: partition_filters: 0 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: use_delta_encoding: 1 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: filter_policy: bloomfilter 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: whole_key_filtering: 1 2024-05-08T05:17:05.470 INFO:journalctl@ceph.mon.a.smithi059.stdout: verify_compression: 0 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout: read_amp_bytes_per_bit: 0 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout: format_version: 5 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout: enable_index_compression: 1 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout: block_align: 0 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout: max_auto_readahead_size: 262144 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout: prepopulate_block_cache: 0 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout: initial_auto_readahead_size: 8192 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout: num_file_reads_for_auto_readahead: 2 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.write_buffer_size: 33554432 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_write_buffer_number: 2 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression: NoCompression 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression: Disabled 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.prefix_extractor: nullptr 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-05-08T05:17:05.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.num_levels: 7 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression_opts.window_bits: -14 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression_opts.level: 32767 2024-05-08T05:17:05.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression_opts.strategy: 0 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression_opts.enabled: false 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.target_file_size_base: 67108864 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.target_file_size_multiplier: 1 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-05-08T05:17:05.473 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.arena_block_size: 1048576 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.disable_auto_compactions: 0 2024-05-08T05:17:05.474 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.inplace_update_support: 0 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.inplace_update_num_locks: 10000 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-05-08T05:17:05.475 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.memtable_huge_page_size: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.bloom_locality: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.max_successive_merges: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.optimize_filters_for_hits: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.paranoid_file_checks: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.force_consistency_checks: 1 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.report_bg_io_stats: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.ttl: 2592000 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.periodic_compaction_seconds: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.enable_blob_files: false 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.min_blob_size: 0 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.blob_file_size: 268435456 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.blob_compression_type: NoCompression 2024-05-08T05:17:05.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.enable_blob_garbage_collection: false 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.blob_file_starting_level: 0 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 83fcea9d-27f2-4f9e-9b8a-219aa0a54812 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715145425408491, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715145425409658, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 100265, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 326, "table_properties": {"data_size": 98150, "index_size": 307, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14169, "raw_average_key_size": 49, "raw_value_size": 90434, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 285, "num_filter_entries": 285, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1715145425, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "83fcea9d-27f2-4f9e-9b8a-219aa0a54812", "db_session_id": "8ZZDQJX788USSPX78C9E", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: EVENT_LOG_v1 {"time_micros": 1715145425409751, "job": 1, "event": "recovery_finished"} 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-05-08T05:17:05.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558642c30000 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: DB pointer 0x558642c1c000 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** DB Stats ** 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-05-08T05:17:05.478 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** Compaction Stats [default] ** 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: L0 2/0 99.78 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: Sum 2/0 99.78 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 129.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** Compaction Stats [default] ** 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 129.4 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-05-08T05:17:05.479 INFO:journalctl@ceph.mon.a.smithi059.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: AddFile(Keys): cumulative 0, interval 0 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: Cumulative compaction: 0.00 GB write, 29.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: Interval compaction: 0.00 GB write, 29.59 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: Block cache BinnedLRUCache@0x558642b131f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout: ** File Read Latency Histogram By Level [default] ** 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: starting mon.a rank 0 at public addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] at bind addrs [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mon.a@-1(???) e1 preinit fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mon.a@-1(???).mds e1 new map 2024-05-08T05:17:05.480 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mon.a@-1(???).mds e1 print_map 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout: e1 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout: legacy client fscid: -1 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout: 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout: No filesystems configured 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 bash[30285]: cc0f25ef8e855159af26d490b10ca789e8fb78095b2286346a16f8fa17343dec 2024-05-08T05:17:05.481 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 systemd[1]: Started Ceph mon.a for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:17:05.481 INFO:teuthology.orchestra.run.smithi059.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-05-08T05:17:05.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mon.a is new leader, mons a in quorum (ranks 0) 2024-05-08T05:17:05.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: monmap epoch 1 2024-05-08T05:17:05.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:17:05.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: last_changed 2024-05-08T05:16:57.521541+0000 2024-05-08T05:17:05.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: created 2024-05-08T05:16:57.521541+0000 2024-05-08T05:17:05.778 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: min_mon_release 19 (squid) 2024-05-08T05:17:05.779 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: election_strategy: 1 2024-05-08T05:17:05.779 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-05-08T05:17:05.779 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: fsmap 2024-05-08T05:17:05.779 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: osdmap e1: 0 total, 0 up, 0 in 2024-05-08T05:17:05.779 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:05 smithi059 ceph-mon[30304]: mgrmap e1: no daemons active 2024-05-08T05:17:06.572 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.885+0000 7f6486d7f700 1 Processor -- start 2024-05-08T05:17:06.572 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.885+0000 7f6486d7f700 1 -- start start 2024-05-08T05:17:06.572 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.885+0000 7f6486d7f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f6480109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:06.572 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.885+0000 7f6486d7f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6480109790 con 0x7f6480108e30 2024-05-08T05:17:06.573 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.885+0000 7f6484b1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f6480109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:06.573 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.886+0000 7f6484b1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f6480109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60870/0 (socket says 172.21.15.59:60870) 2024-05-08T05:17:06.573 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.886+0000 7f6484b1b700 1 -- 172.21.15.59:0/4030805885 learned_addr learned my addr 172.21.15.59:0/4030805885 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:06.573 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.886+0000 7f6484b1b700 1 -- 172.21.15.59:0/4030805885 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6480109f90 con 0x7f6480108e30 2024-05-08T05:17:06.573 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.886+0000 7f6484b1b700 1 --2- 172.21.15.59:0/4030805885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f6480109250 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64740098a0 tx=0x7f6474009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b16b2d1121152e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:06.573 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.886+0000 7f647f7fe700 1 -- 172.21.15.59:0/4030805885 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6474004160 con 0x7f6480108e30 2024-05-08T05:17:06.573 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.886+0000 7f647f7fe700 1 -- 172.21.15.59:0/4030805885 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6474004300 con 0x7f6480108e30 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.886+0000 7f647f7fe700 1 -- 172.21.15.59:0/4030805885 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6474011af0 con 0x7f6480108e30 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.887+0000 7f6486d7f700 1 -- 172.21.15.59:0/4030805885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 msgr2=0x7f6480109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.887+0000 7f6486d7f700 1 --2- 172.21.15.59:0/4030805885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f6480109250 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f64740098a0 tx=0x7f6474009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.887+0000 7f6486d7f700 1 -- 172.21.15.59:0/4030805885 shutdown_connections 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.887+0000 7f6486d7f700 1 --2- 172.21.15.59:0/4030805885 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f6480109250 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.887+0000 7f6486d7f700 1 -- 172.21.15.59:0/4030805885 >> 172.21.15.59:0/4030805885 conn(0x7f6480104410 msgr2=0x7f6480106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.887+0000 7f6486d7f700 1 -- 172.21.15.59:0/4030805885 shutdown_connections 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.887+0000 7f6486d7f700 1 -- 172.21.15.59:0/4030805885 wait complete. 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.887+0000 7f6486d7f700 1 Processor -- start 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f6486d7f700 1 -- start start 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f6486d7f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f648007f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f6486d7f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f648010d3d0 con 0x7f6480108e30 2024-05-08T05:17:06.574 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f6484b1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f648007f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f6484b1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f648007f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60872/0 (socket says 172.21.15.59:60872) 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f6484b1b700 1 -- 172.21.15.59:0/2902057638 learned_addr learned my addr 172.21.15.59:0/2902057638 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f6484b1b700 1 -- 172.21.15.59:0/2902057638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648007db10 con 0x7f6480108e30 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f6484b1b700 1 --2- 172.21.15.59:0/2902057638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f648007f5c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6474003f40 tx=0x7f6474011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f647dffb700 1 -- 172.21.15.59:0/2902057638 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f64740038b0 con 0x7f6480108e30 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f647dffb700 1 -- 172.21.15.59:0/2902057638 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6474003a50 con 0x7f6480108e30 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.888+0000 7f647dffb700 1 -- 172.21.15.59:0/2902057638 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f647401a5c0 con 0x7f6480108e30 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.889+0000 7f6486d7f700 1 -- 172.21.15.59:0/2902057638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648007dda0 con 0x7f6480108e30 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.889+0000 7f6486d7f700 1 -- 172.21.15.59:0/2902057638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f648007e280 con 0x7f6480108e30 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.889+0000 7f647dffb700 1 -- 172.21.15.59:0/2902057638 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f647401ac00 con 0x7f6480108e30 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.889+0000 7f647dffb700 1 -- 172.21.15.59:0/2902057638 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6474019700 con 0x7f6480108e30 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.890+0000 7f6486d7f700 1 -- 172.21.15.59:0/2902057638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6480064b00 con 0x7f6480108e30 2024-05-08T05:17:06.575 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.892+0000 7f647dffb700 1 -- 172.21.15.59:0/2902057638 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6474019900 con 0x7f6480108e30 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.962+0000 7f6486d7f700 1 -- 172.21.15.59:0/2902057638 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f648006bcb0 con 0x7f6480108e30 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.964+0000 7f647dffb700 1 -- 172.21.15.59:0/2902057638 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f6474019ae0 con 0x7f6480108e30 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.965+0000 7f6486d7f700 1 -- 172.21.15.59:0/2902057638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 msgr2=0x7f648007f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.965+0000 7f6486d7f700 1 --2- 172.21.15.59:0/2902057638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f648007f5c0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6474003f40 tx=0x7f6474011890 comp rx=0 tx=0).stop 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.965+0000 7f6486d7f700 1 -- 172.21.15.59:0/2902057638 shutdown_connections 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.965+0000 7f6486d7f700 1 --2- 172.21.15.59:0/2902057638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6480108e30 0x7f648007f5c0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.965+0000 7f6486d7f700 1 -- 172.21.15.59:0/2902057638 >> 172.21.15.59:0/2902057638 conn(0x7f6480104410 msgr2=0x7f6480104e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.965+0000 7f6486d7f700 1 -- 172.21.15.59:0/2902057638 shutdown_connections 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:05.965+0000 7f6486d7f700 1 -- 172.21.15.59:0/2902057638 wait complete. 2024-05-08T05:17:06.576 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote config to /etc/ceph/ceph.conf 2024-05-08T05:17:06.589 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:17:06.589 INFO:teuthology.orchestra.run.smithi059.stdout:Creating mgr... 2024-05-08T05:17:06.590 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying port 0.0.0.0:9283 ... 2024-05-08T05:17:06.590 INFO:teuthology.orchestra.run.smithi059.stdout:Verifying port 0.0.0.0:8765 ... 2024-05-08T05:17:06.848 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.x 2024-05-08T05:17:06.849 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Failed to reset failed state of unit ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.x.service: Unit ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.x.service not loaded. 2024-05-08T05:17:07.009 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968.target.wants/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@.service. 2024-05-08T05:17:07.250 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:06 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2902057638' entity='client.admin' 2024-05-08T05:17:07.250 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 systemd[1]: Starting Ceph mgr.x for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:17:07.510 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 podman[30524]: 2024-05-08 05:17:07.250029915 +0000 UTC m=+0.108486169 container create f894f8c5035ab47b868d6442cea142b266a8665234bcb0d87f234798ba8fdd03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:17:07.510 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 podman[30524]: 2024-05-08 05:17:07.160238271 +0000 UTC m=+0.018694496 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:17:07.510 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 podman[30524]: 2024-05-08 05:17:07.443715436 +0000 UTC m=+0.302171652 container init f894f8c5035ab47b868d6442cea142b266a8665234bcb0d87f234798ba8fdd03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-08T05:17:07.510 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 podman[30524]: 2024-05-08 05:17:07.446435065 +0000 UTC m=+0.304891289 container start f894f8c5035ab47b868d6442cea142b266a8665234bcb0d87f234798ba8fdd03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:17:07.511 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 bash[30524]: f894f8c5035ab47b868d6442cea142b266a8665234bcb0d87f234798ba8fdd03 2024-05-08T05:17:07.527 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:17:07.527 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-05-08T05:17:07.536 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:17:07.536 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-05-08T05:17:07.536 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-05-08T05:17:07.536 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to enable service . firewalld.service is not available 2024-05-08T05:17:07.547 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:17:07.547 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-05-08T05:17:07.555 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:17:07.555 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-05-08T05:17:07.555 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-05-08T05:17:07.555 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-05-08T05:17:07.555 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr to start... 2024-05-08T05:17:07.555 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr... 2024-05-08T05:17:07.822 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 systemd[1]: Started Ceph mgr.x for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:17:07.822 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:07.623+0000 7fb45e24a200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:17:08.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:07.822+0000 7fb45e24a200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:17:08.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:07.931+0000 7fb45e24a200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:17:08.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:08 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3860435686' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:17:08.458 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:08.267+0000 7fb45e24a200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:17:08.459 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:08.358+0000 7fb45e24a200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:17:08.730 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:17:08.731 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:17:08.739 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:17:08.739 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:17:08.739 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:17:08.739 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:08.739 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:17:08.740 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:17:00.522027+0000", 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:17:08.741 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.042+0000 7f9e7d036700 1 Processor -- start 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.042+0000 7f9e7d036700 1 -- start start 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.042+0000 7f9e7d036700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7807db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.042+0000 7f9e7d036700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7807e160 con 0x7f9e7807f730 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.043+0000 7f9e76d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7807db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.043+0000 7f9e76d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7807db90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60904/0 (socket says 172.21.15.59:60904) 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.043+0000 7f9e76d9d700 1 -- 172.21.15.59:0/3210493796 learned_addr learned my addr 172.21.15.59:0/3210493796 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.043+0000 7f9e76d9d700 1 -- 172.21.15.59:0/3210493796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7807e2e0 con 0x7f9e7807f730 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.043+0000 7f9e76d9d700 1 --2- 172.21.15.59:0/3210493796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7807db90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e680098a0 tx=0x7f9e68009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba84eb0feedacf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.043+0000 7f9e75d9b700 1 -- 172.21.15.59:0/3210493796 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68004160 con 0x7f9e7807f730 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.043+0000 7f9e75d9b700 1 -- 172.21.15.59:0/3210493796 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9e68004300 con 0x7f9e7807f730 2024-05-08T05:17:08.742 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.043+0000 7f9e75d9b700 1 -- 172.21.15.59:0/3210493796 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68011af0 con 0x7f9e7807f730 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.044+0000 7f9e7d036700 1 -- 172.21.15.59:0/3210493796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 msgr2=0x7f9e7807db90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.044+0000 7f9e7d036700 1 --2- 172.21.15.59:0/3210493796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7807db90 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9e680098a0 tx=0x7f9e68009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.044+0000 7f9e7d036700 1 -- 172.21.15.59:0/3210493796 shutdown_connections 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.044+0000 7f9e7d036700 1 --2- 172.21.15.59:0/3210493796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7807db90 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.044+0000 7f9e7d036700 1 -- 172.21.15.59:0/3210493796 >> 172.21.15.59:0/3210493796 conn(0x7f9e78104410 msgr2=0x7f9e78106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.044+0000 7f9e7d036700 1 -- 172.21.15.59:0/3210493796 shutdown_connections 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.044+0000 7f9e7d036700 1 -- 172.21.15.59:0/3210493796 wait complete. 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.045+0000 7f9e7d036700 1 Processor -- start 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.045+0000 7f9e7d036700 1 -- start start 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.045+0000 7f9e7d036700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7819ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.045+0000 7f9e7d036700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9e7810f3c0 con 0x7f9e7807f730 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.045+0000 7f9e76d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7819ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.045+0000 7f9e76d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7819ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60916/0 (socket says 172.21.15.59:60916) 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.045+0000 7f9e76d9d700 1 -- 172.21.15.59:0/3860435686 learned_addr learned my addr 172.21.15.59:0/3860435686 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.046+0000 7f9e76d9d700 1 -- 172.21.15.59:0/3860435686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e7819f4a0 con 0x7f9e7807f730 2024-05-08T05:17:08.743 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.046+0000 7f9e76d9d700 1 --2- 172.21.15.59:0/3860435686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7819ef60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e68000c00 tx=0x7f9e68011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.046+0000 7f9e6ffff700 1 -- 172.21.15.59:0/3860435686 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68003b20 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.046+0000 7f9e7d036700 1 -- 172.21.15.59:0/3860435686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e7819f730 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.046+0000 7f9e6ffff700 1 -- 172.21.15.59:0/3860435686 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9e6801a8c0 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.046+0000 7f9e6ffff700 1 -- 172.21.15.59:0/3860435686 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9e68024940 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.046+0000 7f9e7d036700 1 -- 172.21.15.59:0/3860435686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e781a0250 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.046+0000 7f9e6ffff700 1 -- 172.21.15.59:0/3860435686 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9e68022020 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.046+0000 7f9e6ffff700 1 -- 172.21.15.59:0/3860435686 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9e6802dde0 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.047+0000 7f9e7d036700 1 -- 172.21.15.59:0/3860435686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9e78064b00 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.050+0000 7f9e6ffff700 1 -- 172.21.15.59:0/3860435686 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f9e680042a0 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.125+0000 7f9e7d036700 1 -- 172.21.15.59:0/3860435686 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9e781a0660 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.125+0000 7f9e6ffff700 1 -- 172.21.15.59:0/3860435686 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9e6801aa70 con 0x7f9e7807f730 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.126+0000 7f9e7d036700 1 -- 172.21.15.59:0/3860435686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 msgr2=0x7f9e7819ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.126+0000 7f9e7d036700 1 --2- 172.21.15.59:0/3860435686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7819ef60 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9e68000c00 tx=0x7f9e68011b70 comp rx=0 tx=0).stop 2024-05-08T05:17:08.744 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.126+0000 7f9e7d036700 1 -- 172.21.15.59:0/3860435686 shutdown_connections 2024-05-08T05:17:08.745 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.126+0000 7f9e7d036700 1 --2- 172.21.15.59:0/3860435686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9e7807f730 0x7f9e7819ef60 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:08.745 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.126+0000 7f9e7d036700 1 -- 172.21.15.59:0/3860435686 >> 172.21.15.59:0/3860435686 conn(0x7f9e78104410 msgr2=0x7f9e78104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:08.745 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.126+0000 7f9e7d036700 1 -- 172.21.15.59:0/3860435686 shutdown_connections 2024-05-08T05:17:08.745 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:08.126+0000 7f9e7d036700 1 -- 172.21.15.59:0/3860435686 wait complete. 2024-05-08T05:17:08.745 INFO:teuthology.orchestra.run.smithi059.stdout:mgr not available, waiting (1/15)... 2024-05-08T05:17:09.987 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:09.632+0000 7fb45e24a200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:17:09.987 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:09.809+0000 7fb45e24a200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:17:09.987 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:09.986+0000 7fb45e24a200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:17:10.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:10.287+0000 7fb45e24a200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:17:10.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:10.383+0000 7fb45e24a200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:17:10.859 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:10.595+0000 7fb45e24a200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:17:10.859 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:10.802+0000 7fb45e24a200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:17:11.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:11 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2177548610' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:17:11.822 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:11 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:11.456+0000 7fb45e24a200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:17:11.822 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:11 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:11.692+0000 7fb45e24a200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:17:11.827 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:11.827 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-05-08T05:17:11.828 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:17:11.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:11.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:17:11.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:17:11.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:11.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:17:11.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:11.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:17:11.829 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:17:11.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:17:11.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:17:11.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:17:11.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:17:11.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:11.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:17:00.522027+0000", 2024-05-08T05:17:11.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.180+0000 7f677b10e700 1 Processor -- start 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.180+0000 7f677b10e700 1 -- start start 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.181+0000 7f677b10e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677410bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.181+0000 7f677b10e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f677407d690 con 0x7f6774109940 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.181+0000 7f6778eaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677410bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.181+0000 7f6778eaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677410bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40604/0 (socket says 172.21.15.59:40604) 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.181+0000 7f6778eaa700 1 -- 172.21.15.59:0/3281384983 learned_addr learned my addr 172.21.15.59:0/3281384983 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.181+0000 7f6778eaa700 1 -- 172.21.15.59:0/3281384983 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677410c270 con 0x7f6774109940 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.181+0000 7f6778eaa700 1 --2- 172.21.15.59:0/3281384983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677410bd30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67680098a0 tx=0x7f6768009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ddd346fb3c893ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.181+0000 7f67737fe700 1 -- 172.21.15.59:0/3281384983 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6768004160 con 0x7f6774109940 2024-05-08T05:17:11.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.182+0000 7f67737fe700 1 -- 172.21.15.59:0/3281384983 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6768004300 con 0x7f6774109940 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.182+0000 7f67737fe700 1 -- 172.21.15.59:0/3281384983 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6768011af0 con 0x7f6774109940 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.182+0000 7f677b10e700 1 -- 172.21.15.59:0/3281384983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 msgr2=0x7f677410bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.182+0000 7f677b10e700 1 --2- 172.21.15.59:0/3281384983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677410bd30 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f67680098a0 tx=0x7f6768009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.182+0000 7f677b10e700 1 -- 172.21.15.59:0/3281384983 shutdown_connections 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.182+0000 7f677b10e700 1 --2- 172.21.15.59:0/3281384983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677410bd30 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.182+0000 7f677b10e700 1 -- 172.21.15.59:0/3281384983 >> 172.21.15.59:0/3281384983 conn(0x7f6774103350 msgr2=0x7f6774105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.182+0000 7f677b10e700 1 -- 172.21.15.59:0/3281384983 shutdown_connections 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.182+0000 7f677b10e700 1 -- 172.21.15.59:0/3281384983 wait complete. 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.183+0000 7f677b10e700 1 Processor -- start 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.183+0000 7f677b10e700 1 -- start start 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f677b10e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677419ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f677b10e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f677410d1d0 con 0x7f6774109940 2024-05-08T05:17:11.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f6778eaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677419ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f6778eaa700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677419ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40618/0 (socket says 172.21.15.59:40618) 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f6778eaa700 1 -- 172.21.15.59:0/2177548610 learned_addr learned my addr 172.21.15.59:0/2177548610 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f6778eaa700 1 -- 172.21.15.59:0/2177548610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f677419d390 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f6778eaa700 1 --2- 172.21.15.59:0/2177548610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677419ce50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6768009fb0 tx=0x7f6768011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f6771ffb700 1 -- 172.21.15.59:0/2177548610 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f67680036a0 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f6771ffb700 1 -- 172.21.15.59:0/2177548610 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f676801a430 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f6771ffb700 1 -- 172.21.15.59:0/2177548610 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6768019550 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.184+0000 7f677b10e700 1 -- 172.21.15.59:0/2177548610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f677419d620 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.185+0000 7f677b10e700 1 -- 172.21.15.59:0/2177548610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f677419e090 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.185+0000 7f6771ffb700 1 -- 172.21.15.59:0/2177548610 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6768019b90 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.185+0000 7f6771ffb700 1 -- 172.21.15.59:0/2177548610 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6768024730 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.185+0000 7f677b10e700 1 -- 172.21.15.59:0/2177548610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6774064b00 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.188+0000 7f6771ffb700 1 -- 172.21.15.59:0/2177548610 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f6768043ec0 con 0x7f6774109940 2024-05-08T05:17:11.842 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.263+0000 7f677b10e700 1 -- 172.21.15.59:0/2177548610 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f677419e590 con 0x7f6774109940 2024-05-08T05:17:11.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.264+0000 7f6771ffb700 1 -- 172.21.15.59:0/2177548610 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f6768032030 con 0x7f6774109940 2024-05-08T05:17:11.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.265+0000 7f677b10e700 1 -- 172.21.15.59:0/2177548610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 msgr2=0x7f677419ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:11.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.265+0000 7f677b10e700 1 --2- 172.21.15.59:0/2177548610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677419ce50 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f6768009fb0 tx=0x7f6768011940 comp rx=0 tx=0).stop 2024-05-08T05:17:11.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.265+0000 7f677b10e700 1 -- 172.21.15.59:0/2177548610 shutdown_connections 2024-05-08T05:17:11.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.265+0000 7f677b10e700 1 --2- 172.21.15.59:0/2177548610 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6774109940 0x7f677419ce50 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:11.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.265+0000 7f677b10e700 1 -- 172.21.15.59:0/2177548610 >> 172.21.15.59:0/2177548610 conn(0x7f6774103350 msgr2=0x7f6774103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:11.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.265+0000 7f677b10e700 1 -- 172.21.15.59:0/2177548610 shutdown_connections 2024-05-08T05:17:11.843 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:11.265+0000 7f677b10e700 1 -- 172.21.15.59:0/2177548610 wait complete. 2024-05-08T05:17:11.843 INFO:teuthology.orchestra.run.smithi059.stdout:mgr not available, waiting (2/15)... 2024-05-08T05:17:12.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:12.138+0000 7fb45e24a200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:17:12.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:12.239+0000 7fb45e24a200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:17:13.026 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:12.649+0000 7fb45e24a200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:17:13.026 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:13.025+0000 7fb45e24a200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:17:13.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:13.123+0000 7fb45e24a200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:17:13.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:13.384+0000 7fb45e24a200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:17:13.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:13.525+0000 7fb45e24a200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:17:13.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:13.627+0000 7fb45e24a200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:17:14.271 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:13.977+0000 7fb45e24a200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:17:14.271 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:14 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:14.075+0000 7fb45e24a200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:17:14.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:14 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/297785285' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:17:14.949 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:14 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:14.732+0000 7fb45e24a200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:17:14.955 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 8, 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:17:14.956 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:17:14.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:17:14.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:17:14.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:14.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:17:14.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:17:14.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:17:14.957 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:17:14.963 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": false, 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:17:14.964 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:17:00.522027+0000", 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.297+0000 7f5ef48e7700 1 Processor -- start 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.297+0000 7f5ef48e7700 1 -- start start 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.297+0000 7f5ef48e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef0108170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:14.965 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.297+0000 7f5ef48e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef0108740 con 0x7f5ef0107d50 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.297+0000 7f5eee59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef0108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.297+0000 7f5eee59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef0108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40634/0 (socket says 172.21.15.59:40634) 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.297+0000 7f5eee59c700 1 -- 172.21.15.59:0/954753866 learned_addr learned my addr 172.21.15.59:0/954753866 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.298+0000 7f5eee59c700 1 -- 172.21.15.59:0/954753866 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef0108f50 con 0x7f5ef0107d50 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.298+0000 7f5eee59c700 1 --2- 172.21.15.59:0/954753866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef0108170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00098a0 tx=0x7f5ee0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=625298b1b004a4e8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.298+0000 7f5eed59a700 1 -- 172.21.15.59:0/954753866 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ee0004160 con 0x7f5ef0107d50 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.298+0000 7f5eed59a700 1 -- 172.21.15.59:0/954753866 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5ee0004300 con 0x7f5ef0107d50 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.298+0000 7f5eed59a700 1 -- 172.21.15.59:0/954753866 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ee0011af0 con 0x7f5ef0107d50 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.299+0000 7f5ef48e7700 1 -- 172.21.15.59:0/954753866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 msgr2=0x7f5ef0108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.299+0000 7f5ef48e7700 1 --2- 172.21.15.59:0/954753866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef0108170 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00098a0 tx=0x7f5ee0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.299+0000 7f5ef48e7700 1 -- 172.21.15.59:0/954753866 shutdown_connections 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.299+0000 7f5ef48e7700 1 --2- 172.21.15.59:0/954753866 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef0108170 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.299+0000 7f5ef48e7700 1 -- 172.21.15.59:0/954753866 >> 172.21.15.59:0/954753866 conn(0x7f5ef01032b0 msgr2=0x7f5ef0105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.299+0000 7f5ef48e7700 1 -- 172.21.15.59:0/954753866 shutdown_connections 2024-05-08T05:17:14.966 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.299+0000 7f5ef48e7700 1 -- 172.21.15.59:0/954753866 wait complete. 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.300+0000 7f5ef48e7700 1 Processor -- start 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.300+0000 7f5ef48e7700 1 -- start start 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.300+0000 7f5ef48e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef019ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.300+0000 7f5ef48e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5ef007bfb0 con 0x7f5ef0107d50 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.300+0000 7f5eee59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef019ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.300+0000 7f5eee59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef019ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40640/0 (socket says 172.21.15.59:40640) 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.300+0000 7f5eee59c700 1 -- 172.21.15.59:0/297785285 learned_addr learned my addr 172.21.15.59:0/297785285 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.301+0000 7f5eee59c700 1 -- 172.21.15.59:0/297785285 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ef019b190 con 0x7f5ef0107d50 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.301+0000 7f5eee59c700 1 --2- 172.21.15.59:0/297785285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef019ac50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00117b0 tx=0x7f5ee0011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.301+0000 7f5edf7fe700 1 -- 172.21.15.59:0/297785285 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ee0003af0 con 0x7f5ef0107d50 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.301+0000 7f5ef48e7700 1 -- 172.21.15.59:0/297785285 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ef019b420 con 0x7f5ef0107d50 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.301+0000 7f5edf7fe700 1 -- 172.21.15.59:0/297785285 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f5ee0020070 con 0x7f5ef0107d50 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.301+0000 7f5edf7fe700 1 -- 172.21.15.59:0/297785285 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5ee0019630 con 0x7f5ef0107d50 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.301+0000 7f5ef48e7700 1 -- 172.21.15.59:0/297785285 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ef019bf40 con 0x7f5ef0107d50 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.301+0000 7f5edf7fe700 1 -- 172.21.15.59:0/297785285 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f5ee0019c40 con 0x7f5ef0107d50 2024-05-08T05:17:14.967 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.301+0000 7f5edf7fe700 1 -- 172.21.15.59:0/297785285 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f5ee000b720 con 0x7f5ef0107d50 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.302+0000 7f5ef48e7700 1 -- 172.21.15.59:0/297785285 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ef0064b00 con 0x7f5ef0107d50 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.304+0000 7f5edf7fe700 1 -- 172.21.15.59:0/297785285 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73389 (secure 0 0 0) 0x7f5ee00042a0 con 0x7f5ef0107d50 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.380+0000 7f5ef48e7700 1 -- 172.21.15.59:0/297785285 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f5ef019cb20 con 0x7f5ef0107d50 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.381+0000 7f5edf7fe700 1 -- 172.21.15.59:0/297785285 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f5ee001a9c0 con 0x7f5ef0107d50 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.381+0000 7f5ef48e7700 1 -- 172.21.15.59:0/297785285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 msgr2=0x7f5ef019ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.381+0000 7f5ef48e7700 1 --2- 172.21.15.59:0/297785285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef019ac50 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5ee00117b0 tx=0x7f5ee0011b70 comp rx=0 tx=0).stop 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.381+0000 7f5ef48e7700 1 -- 172.21.15.59:0/297785285 shutdown_connections 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.381+0000 7f5ef48e7700 1 --2- 172.21.15.59:0/297785285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5ef0107d50 0x7f5ef019ac50 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.381+0000 7f5ef48e7700 1 -- 172.21.15.59:0/297785285 >> 172.21.15.59:0/297785285 conn(0x7f5ef01032b0 msgr2=0x7f5ef0191190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.382+0000 7f5ef48e7700 1 -- 172.21.15.59:0/297785285 shutdown_connections 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:14.382+0000 7f5ef48e7700 1 -- 172.21.15.59:0/297785285 wait complete. 2024-05-08T05:17:14.968 INFO:teuthology.orchestra.run.smithi059.stdout:mgr not available, waiting (3/15)... 2024-05-08T05:17:15.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:14 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:14.961+0000 7fb45e24a200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:17:15.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: Activating manager daemon x 2024-05-08T05:17:15.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: mgrmap e2: x(active, starting, since 0.00366282s) 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: Manager daemon x is now available 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' 2024-05-08T05:17:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:15 smithi059 ceph-mon[30304]: from='mgr.14100 172.21.15.59:0/988828320' entity='mgr.x' 2024-05-08T05:17:17.268 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:16 smithi059 ceph-mon[30304]: mgrmap e3: x(active, since 1.00779s) 2024-05-08T05:17:18.340 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:17 smithi059 ceph-mon[30304]: mgrmap e4: x(active, since 2s) 2024-05-08T05:17:18.340 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:17 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2970251568' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-05-08T05:17:19.368 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "health": { 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 0 2024-05-08T05:17:19.369 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "a" 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "monmap": { 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-05-08T05:17:19.370 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-05-08T05:17:19.371 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-05-08T05:17:19.372 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modules": [ 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "iostat", 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "nfs", 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "restful" 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ], 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-05-08T05:17:19.373 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "modified": "2024-05-08T05:17:00.522027+0000", 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "services": {} 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout }, 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.377+0000 7f45b8fb7700 1 Processor -- start 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.377+0000 7f45b8fb7700 1 -- start start 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.377+0000 7f45b8fb7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b4107b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.377+0000 7f45b8fb7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45b407ce30 con 0x7f45b4107710 2024-05-08T05:17:19.374 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.378+0000 7f45b259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b4107b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.378+0000 7f45b259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b4107b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40708/0 (socket says 172.21.15.59:40708) 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.378+0000 7f45b259c700 1 -- 172.21.15.59:0/303833285 learned_addr learned my addr 172.21.15.59:0/303833285 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.378+0000 7f45b259c700 1 -- 172.21.15.59:0/303833285 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b4108070 con 0x7f45b4107710 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.378+0000 7f45b259c700 1 --2- 172.21.15.59:0/303833285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b4107b30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009e80 tx=0x7f45a400b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e5f0f4dcc50ad89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.378+0000 7f45b1d9b700 1 -- 172.21.15.59:0/303833285 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a4017070 con 0x7f45b4107710 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.378+0000 7f45b1d9b700 1 -- 172.21.15.59:0/303833285 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f45a400b8c0 con 0x7f45b4107710 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.378+0000 7f45b1d9b700 1 -- 172.21.15.59:0/303833285 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a4004490 con 0x7f45b4107710 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.379+0000 7f45b8fb7700 1 -- 172.21.15.59:0/303833285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 msgr2=0x7f45b4107b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.379+0000 7f45b8fb7700 1 --2- 172.21.15.59:0/303833285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b4107b30 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f45a4009e80 tx=0x7f45a400b330 comp rx=0 tx=0).stop 2024-05-08T05:17:19.375 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.379+0000 7f45b8fb7700 1 -- 172.21.15.59:0/303833285 shutdown_connections 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.379+0000 7f45b8fb7700 1 --2- 172.21.15.59:0/303833285 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b4107b30 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.379+0000 7f45b8fb7700 1 -- 172.21.15.59:0/303833285 >> 172.21.15.59:0/303833285 conn(0x7f45b41032b0 msgr2=0x7f45b4105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.379+0000 7f45b8fb7700 1 -- 172.21.15.59:0/303833285 shutdown_connections 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.379+0000 7f45b8fb7700 1 -- 172.21.15.59:0/303833285 wait complete. 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.380+0000 7f45b8fb7700 1 Processor -- start 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.380+0000 7f45b8fb7700 1 -- start start 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.380+0000 7f45b8fb7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b419f000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.380+0000 7f45b8fb7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f45b41090a0 con 0x7f45b4107710 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.380+0000 7f45b259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b419f000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.380+0000 7f45b259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b419f000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40720/0 (socket says 172.21.15.59:40720) 2024-05-08T05:17:19.376 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.380+0000 7f45b259c700 1 -- 172.21.15.59:0/2970251568 learned_addr learned my addr 172.21.15.59:0/2970251568 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.381+0000 7f45b259c700 1 -- 172.21.15.59:0/2970251568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b419f540 con 0x7f45b4107710 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.381+0000 7f45b259c700 1 --2- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b419f000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45a400b7b0 tx=0x7f45a400be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.381+0000 7f45abfff700 1 -- 172.21.15.59:0/2970251568 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a4020020 con 0x7f45b4107710 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.381+0000 7f45abfff700 1 -- 172.21.15.59:0/2970251568 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f45a4012430 con 0x7f45b4107710 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.381+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b419f7d0 con 0x7f45b4107710 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.381+0000 7f45abfff700 1 -- 172.21.15.59:0/2970251568 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a4017040 con 0x7f45b4107710 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.381+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b41a02f0 con 0x7f45b4107710 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.382+0000 7f45abfff700 1 -- 172.21.15.59:0/2970251568 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f45a4023020 con 0x7f45b4107710 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.382+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b4064b00 con 0x7f45b4107710 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.382+0000 7f45abfff700 1 --2- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f45a003e490 0x7f45a0040950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:19.377 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.382+0000 7f45abfff700 1 -- 172.21.15.59:0/2970251568 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f45a400f050 con 0x7f45b4107710 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.383+0000 7f45a9d9b700 1 --2- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f45a003e490 0x7f45a0040950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.383+0000 7f45a9d9b700 1 --2- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f45a003e490 0x7f45a0040950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f459c006fd0 tx=0x7f459c006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.388+0000 7f45abfff700 1 -- 172.21.15.59:0/2970251568 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f45a402a020 con 0x7f45b4107710 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.703+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f45b41a1050 con 0x7f45b4107710 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.703+0000 7f45abfff700 1 -- 172.21.15.59:0/2970251568 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f45a401b9d0 con 0x7f45b4107710 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.705+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f45a003e490 msgr2=0x7f45a0040950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.705+0000 7f45b8fb7700 1 --2- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f45a003e490 0x7f45a0040950 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f459c006fd0 tx=0x7f459c006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.705+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 msgr2=0x7f45b419f000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.705+0000 7f45b8fb7700 1 --2- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b419f000 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f45a400b7b0 tx=0x7f45a400be20 comp rx=0 tx=0).stop 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.705+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 shutdown_connections 2024-05-08T05:17:19.378 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.705+0000 7f45b8fb7700 1 --2- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f45a003e490 0x7f45a0040950 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:19.379 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.705+0000 7f45b8fb7700 1 --2- 172.21.15.59:0/2970251568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f45b4107710 0x7f45b419f000 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:19.379 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.705+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 >> 172.21.15.59:0/2970251568 conn(0x7f45b41032b0 msgr2=0x7f45b4195b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:19.379 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.706+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 shutdown_connections 2024-05-08T05:17:19.379 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:17.706+0000 7f45b8fb7700 1 -- 172.21.15.59:0/2970251568 wait complete. 2024-05-08T05:17:19.379 INFO:teuthology.orchestra.run.smithi059.stdout:mgr is available 2024-05-08T05:17:22.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:22 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2865578698' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-05-08T05:17:27.268 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:27.268 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [global] 2024-05-08T05:17:27.268 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout fsid = 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [mgr] 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout [osd] 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-05-08T05:17:27.269 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.266+0000 7fee2ad82700 1 Processor -- start 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.266+0000 7fee2ad82700 1 -- start start 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.267+0000 7fee2ad82700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee24109300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.267+0000 7fee2ad82700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee24109840 con 0x7fee24108ee0 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.267+0000 7fee28b1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee24109300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.267+0000 7fee28b1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee24109300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38790/0 (socket says 172.21.15.59:38790) 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.267+0000 7fee28b1e700 1 -- 172.21.15.59:0/3649752967 learned_addr learned my addr 172.21.15.59:0/3649752967 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.267+0000 7fee28b1e700 1 -- 172.21.15.59:0/3649752967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee24109fa0 con 0x7fee24108ee0 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.267+0000 7fee28b1e700 1 --2- 172.21.15.59:0/3649752967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee24109300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fee140098a0 tx=0x7fee14009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20da292ea22357f0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.267+0000 7fee237fe700 1 -- 172.21.15.59:0/3649752967 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee14004160 con 0x7fee24108ee0 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.268+0000 7fee237fe700 1 -- 172.21.15.59:0/3649752967 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fee14004300 con 0x7fee24108ee0 2024-05-08T05:17:27.270 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.268+0000 7fee237fe700 1 -- 172.21.15.59:0/3649752967 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee14011af0 con 0x7fee24108ee0 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.268+0000 7fee2ad82700 1 -- 172.21.15.59:0/3649752967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 msgr2=0x7fee24109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.268+0000 7fee2ad82700 1 --2- 172.21.15.59:0/3649752967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee24109300 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fee140098a0 tx=0x7fee14009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.268+0000 7fee2ad82700 1 -- 172.21.15.59:0/3649752967 shutdown_connections 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.268+0000 7fee2ad82700 1 --2- 172.21.15.59:0/3649752967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee24109300 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.268+0000 7fee2ad82700 1 -- 172.21.15.59:0/3649752967 >> 172.21.15.59:0/3649752967 conn(0x7fee241044a0 msgr2=0x7fee241068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.268+0000 7fee2ad82700 1 -- 172.21.15.59:0/3649752967 shutdown_connections 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.268+0000 7fee2ad82700 1 -- 172.21.15.59:0/3649752967 wait complete. 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.269+0000 7fee2ad82700 1 Processor -- start 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.269+0000 7fee2ad82700 1 -- start start 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.269+0000 7fee2ad82700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee2407ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.269+0000 7fee2ad82700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fee2410d3e0 con 0x7fee24108ee0 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.269+0000 7fee28b1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee2407ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.269+0000 7fee28b1e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee2407ed30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38798/0 (socket says 172.21.15.59:38798) 2024-05-08T05:17:27.271 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.269+0000 7fee28b1e700 1 -- 172.21.15.59:0/2865578698 learned_addr learned my addr 172.21.15.59:0/2865578698 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.269+0000 7fee28b1e700 1 -- 172.21.15.59:0/2865578698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee2407d280 con 0x7fee24108ee0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.269+0000 7fee28b1e700 1 --2- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee2407ed30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee14009fb0 tx=0x7fee14011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.270+0000 7fee21ffb700 1 -- 172.21.15.59:0/2865578698 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee140036a0 con 0x7fee24108ee0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.270+0000 7fee21ffb700 1 -- 172.21.15.59:0/2865578698 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fee1401a430 con 0x7fee24108ee0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.270+0000 7fee21ffb700 1 -- 172.21.15.59:0/2865578698 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fee14019550 con 0x7fee24108ee0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.270+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee2407d510 con 0x7fee24108ee0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.270+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee2407d9f0 con 0x7fee24108ee0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.271+0000 7fee21ffb700 1 -- 172.21.15.59:0/2865578698 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7fee1401a8e0 con 0x7fee24108ee0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.271+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee24064b00 con 0x7fee24108ee0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.271+0000 7fee21ffb700 1 --2- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fee1003e4e0 0x7fee100409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.271+0000 7fee21ffb700 1 -- 172.21.15.59:0/2865578698 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fee14053790 con 0x7fee24108ee0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.271+0000 7fee23fff700 1 --2- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fee1003e4e0 0x7fee100409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.272+0000 7fee23fff700 1 --2- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fee1003e4e0 0x7fee100409a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00ad30 tx=0x7fee0c0093f0 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:27.272 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.276+0000 7fee21ffb700 1 -- 172.21.15.59:0/2865578698 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fee14008040 con 0x7fee24108ee0 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.490+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fee2407df90 con 0x7fee24108ee0 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.491+0000 7fee21ffb700 1 -- 172.21.15.59:0/2865578698 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7fee1400b3e0 con 0x7fee24108ee0 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.493+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fee1003e4e0 msgr2=0x7fee100409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.493+0000 7fee2ad82700 1 --2- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fee1003e4e0 0x7fee100409a0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7fee0c00ad30 tx=0x7fee0c0093f0 comp rx=0 tx=0).stop 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.493+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 msgr2=0x7fee2407ed30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.493+0000 7fee2ad82700 1 --2- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee2407ed30 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fee14009fb0 tx=0x7fee14011940 comp rx=0 tx=0).stop 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.493+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 shutdown_connections 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.493+0000 7fee2ad82700 1 --2- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fee1003e4e0 0x7fee100409a0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.493+0000 7fee2ad82700 1 --2- 172.21.15.59:0/2865578698 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fee24108ee0 0x7fee2407ed30 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.493+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 >> 172.21.15.59:0/2865578698 conn(0x7fee241044a0 msgr2=0x7fee24104f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.494+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 shutdown_connections 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:22.494+0000 7fee2ad82700 1 -- 172.21.15.59:0/2865578698 wait complete. 2024-05-08T05:17:27.273 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling cephadm module... 2024-05-08T05:17:28.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:27 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/313040021' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-05-08T05:17:28.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: ignoring --setuser ceph since I am not root 2024-05-08T05:17:28.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: ignoring --setgroup ceph since I am not root 2024-05-08T05:17:28.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:28.169+0000 7f9d5da05200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:17:28.811 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:28.363+0000 7f9d5da05200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:17:28.811 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:28.471+0000 7f9d5da05200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:17:28.811 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:28.793+0000 7f9d5da05200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:17:28.997 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.655+0000 7f620d3cf700 1 Processor -- start 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.655+0000 7f620d3cf700 1 -- start start 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.655+0000 7f620d3cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620807db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.655+0000 7f620d3cf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f620807e140 con 0x7f620807f710 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.655+0000 7f6206ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620807db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.655+0000 7f6206ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620807db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38802/0 (socket says 172.21.15.59:38802) 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.655+0000 7f6206ffd700 1 -- 172.21.15.59:0/694117182 learned_addr learned my addr 172.21.15.59:0/694117182 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.656+0000 7f6206ffd700 1 -- 172.21.15.59:0/694117182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620807e2c0 con 0x7f620807f710 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.656+0000 7f6206ffd700 1 --2- 172.21.15.59:0/694117182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620807db70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61f80098a0 tx=0x7f61f8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb995132aee15f5d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.656+0000 7f6205ffb700 1 -- 172.21.15.59:0/694117182 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61f8004160 con 0x7f620807f710 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.656+0000 7f6205ffb700 1 -- 172.21.15.59:0/694117182 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f61f8004300 con 0x7f620807f710 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.656+0000 7f6205ffb700 1 -- 172.21.15.59:0/694117182 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61f8011af0 con 0x7f620807f710 2024-05-08T05:17:28.998 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.656+0000 7f620d3cf700 1 -- 172.21.15.59:0/694117182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 msgr2=0x7f620807db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.656+0000 7f620d3cf700 1 --2- 172.21.15.59:0/694117182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620807db70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f61f80098a0 tx=0x7f61f8009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.657+0000 7f620d3cf700 1 -- 172.21.15.59:0/694117182 shutdown_connections 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.657+0000 7f620d3cf700 1 --2- 172.21.15.59:0/694117182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620807db70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.657+0000 7f620d3cf700 1 -- 172.21.15.59:0/694117182 >> 172.21.15.59:0/694117182 conn(0x7f6208104410 msgr2=0x7f6208106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.657+0000 7f620d3cf700 1 -- 172.21.15.59:0/694117182 shutdown_connections 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.657+0000 7f620d3cf700 1 -- 172.21.15.59:0/694117182 wait complete. 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.657+0000 7f620d3cf700 1 Processor -- start 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.657+0000 7f620d3cf700 1 -- start start 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.658+0000 7f620d3cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620819efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.658+0000 7f620d3cf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f620810f3e0 con 0x7f620807f710 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.658+0000 7f6206ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620819efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.658+0000 7f6206ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620819efa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38810/0 (socket says 172.21.15.59:38810) 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.658+0000 7f6206ffd700 1 -- 172.21.15.59:0/313040021 learned_addr learned my addr 172.21.15.59:0/313040021 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.658+0000 7f6206ffd700 1 -- 172.21.15.59:0/313040021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f620819f4e0 con 0x7f620807f710 2024-05-08T05:17:28.999 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.658+0000 7f6206ffd700 1 --2- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620819efa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f61f8000c00 tx=0x7f61f8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.658+0000 7f61effff700 1 -- 172.21.15.59:0/313040021 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61f8003b20 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.658+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f620819f770 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.659+0000 7f61effff700 1 -- 172.21.15.59:0/313040021 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f61f801a8c0 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.659+0000 7f61effff700 1 -- 172.21.15.59:0/313040021 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61f8024940 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.659+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62081a0290 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.660+0000 7f61effff700 1 -- 172.21.15.59:0/313040021 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 4) ==== 50211+0+0 (secure 0 0 0) 0x7f61f8022020 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.660+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6208064b00 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.660+0000 7f61effff700 1 --2- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f61f003e4e0 0x7f61f00409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.660+0000 7f61effff700 1 -- 172.21.15.59:0/313040021 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f61f80533f0 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.660+0000 7f62067fc700 1 --2- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f61f003e4e0 0x7f61f00409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.660+0000 7f62067fc700 1 --2- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f61f003e4e0 0x7f61f00409a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61fc006fd0 tx=0x7f61fc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.665+0000 7f61effff700 1 -- 172.21.15.59:0/313040021 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f61f80212c0 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.941+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f62081a07b0 con 0x7f620807f710 2024-05-08T05:17:29.000 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.993+0000 7f61effff700 1 -- 172.21.15.59:0/313040021 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f61f80042c0 con 0x7f620807f710 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.994+0000 7f61effff700 1 -- 172.21.15.59:0/313040021 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f61f801aa70 con 0x7f620807f710 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f61f003e4e0 msgr2=0x7f61f00409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 --2- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f61f003e4e0 0x7f61f00409a0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f61fc006fd0 tx=0x7f61fc006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 msgr2=0x7f620819efa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 --2- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620819efa0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f61f8000c00 tx=0x7f61f8011b70 comp rx=0 tx=0).stop 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 shutdown_connections 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 --2- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f61f003e4e0 0x7f61f00409a0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 --2- 172.21.15.59:0/313040021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f620807f710 0x7f620819efa0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 >> 172.21.15.59:0/313040021 conn(0x7f6208104410 msgr2=0x7f6208104f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 shutdown_connections 2024-05-08T05:17:29.001 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:27.996+0000 7f620d3cf700 1 -- 172.21.15.59:0/313040021 wait complete. 2024-05-08T05:17:29.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:28 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/313040021' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-05-08T05:17:29.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:28 smithi059 ceph-mon[30304]: mgrmap e5: x(active, since 13s) 2024-05-08T05:17:29.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:28.882+0000 7f9d5da05200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:17:30.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:29 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3237239644' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T05:17:30.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:30.167+0000 7f9d5da05200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:17:30.362 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:17:30.362 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-05-08T05:17:30.362 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:17:30.362 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T05:17:30.362 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T05:17:30.362 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:17:30.362 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.453+0000 7fb27b19e700 1 Processor -- start 2024-05-08T05:17:30.362 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.453+0000 7fb27b19e700 1 -- start start 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.453+0000 7fb27b19e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb2741092b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.453+0000 7fb27b19e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2741097f0 con 0x7fb274108e90 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.453+0000 7fb278f3a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb2741092b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.453+0000 7fb278f3a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb2741092b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39482/0 (socket says 172.21.15.59:39482) 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.453+0000 7fb278f3a700 1 -- 172.21.15.59:0/1495861438 learned_addr learned my addr 172.21.15.59:0/1495861438 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.453+0000 7fb278f3a700 1 -- 172.21.15.59:0/1495861438 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb274109ff0 con 0x7fb274108e90 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.454+0000 7fb278f3a700 1 --2- 172.21.15.59:0/1495861438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb2741092b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb2680098a0 tx=0x7fb268009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0da563e39415a1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.454+0000 7fb2737fe700 1 -- 172.21.15.59:0/1495861438 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb268004160 con 0x7fb274108e90 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.454+0000 7fb2737fe700 1 -- 172.21.15.59:0/1495861438 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb268004300 con 0x7fb274108e90 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.454+0000 7fb2737fe700 1 -- 172.21.15.59:0/1495861438 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb268011af0 con 0x7fb274108e90 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.454+0000 7fb27b19e700 1 -- 172.21.15.59:0/1495861438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 msgr2=0x7fb2741092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.454+0000 7fb27b19e700 1 --2- 172.21.15.59:0/1495861438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb2741092b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fb2680098a0 tx=0x7fb268009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.454+0000 7fb27b19e700 1 -- 172.21.15.59:0/1495861438 shutdown_connections 2024-05-08T05:17:30.363 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.454+0000 7fb27b19e700 1 --2- 172.21.15.59:0/1495861438 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb2741092b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.454+0000 7fb27b19e700 1 -- 172.21.15.59:0/1495861438 >> 172.21.15.59:0/1495861438 conn(0x7fb274104490 msgr2=0x7fb2741068b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.455+0000 7fb27b19e700 1 -- 172.21.15.59:0/1495861438 shutdown_connections 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.455+0000 7fb27b19e700 1 -- 172.21.15.59:0/1495861438 wait complete. 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.455+0000 7fb27b19e700 1 Processor -- start 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.455+0000 7fb27b19e700 1 -- start start 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.455+0000 7fb27b19e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb27419ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.455+0000 7fb27b19e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb27410d430 con 0x7fb274108e90 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.455+0000 7fb278f3a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb27419ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.456+0000 7fb278f3a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb27419ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39484/0 (socket says 172.21.15.59:39484) 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.456+0000 7fb278f3a700 1 -- 172.21.15.59:0/3237239644 learned_addr learned my addr 172.21.15.59:0/3237239644 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.456+0000 7fb278f3a700 1 -- 172.21.15.59:0/3237239644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb27419d410 con 0x7fb274108e90 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.456+0000 7fb278f3a700 1 --2- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb27419ced0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb268011ac0 tx=0x7fb268011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.456+0000 7fb271ffb700 1 -- 172.21.15.59:0/3237239644 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb268003970 con 0x7fb274108e90 2024-05-08T05:17:30.364 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.456+0000 7fb271ffb700 1 -- 172.21.15.59:0/3237239644 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb268003b10 con 0x7fb274108e90 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.456+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb27419d6a0 con 0x7fb274108e90 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.456+0000 7fb271ffb700 1 -- 172.21.15.59:0/3237239644 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb268024610 con 0x7fb274108e90 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.456+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb27419e080 con 0x7fb274108e90 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.457+0000 7fb271ffb700 1 -- 172.21.15.59:0/3237239644 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7fb268024890 con 0x7fb274108e90 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.458+0000 7fb271ffb700 1 --2- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fb26403e530 0x7fb2640409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.458+0000 7fb271ffb700 1 -- 172.21.15.59:0/3237239644 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fb2680538b0 con 0x7fb274108e90 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.458+0000 7fb273fff700 1 -- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fb26403e530 msgr2=0x7fb2640409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817743540 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.458+0000 7fb273fff700 1 --2- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fb26403e530 0x7fb2640409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.458+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb274064b00 con 0x7fb274108e90 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.463+0000 7fb271ffb700 1 -- 172.21.15.59:0/3237239644 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb26802c350 con 0x7fb274108e90 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.659+0000 7fb273fff700 1 -- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fb26403e530 msgr2=0x7fb2640409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817743540 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.659+0000 7fb273fff700 1 --2- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fb26403e530 0x7fb2640409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:17:30.365 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.751+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fb27419e710 con 0x7fb274108e90 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.751+0000 7fb271ffb700 1 -- 172.21.15.59:0/3237239644 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fb2680172b0 con 0x7fb274108e90 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fb26403e530 msgr2=0x7fb2640409f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 --2- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fb26403e530 0x7fb2640409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 msgr2=0x7fb27419ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 --2- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb27419ced0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb268011ac0 tx=0x7fb268011ba0 comp rx=0 tx=0).stop 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 shutdown_connections 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 --2- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7fb26403e530 0x7fb2640409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 --2- 172.21.15.59:0/3237239644 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb274108e90 0x7fb27419ced0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 >> 172.21.15.59:0/3237239644 conn(0x7fb274104490 msgr2=0x7fb274104ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 shutdown_connections 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:29.754+0000 7fb27b19e700 1 -- 172.21.15.59:0/3237239644 wait complete. 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for the mgr to restart... 2024-05-08T05:17:30.366 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr epoch 5... 2024-05-08T05:17:30.619 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:30.345+0000 7f9d5da05200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:17:30.620 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:30.525+0000 7f9d5da05200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:17:30.933 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:30.835+0000 7f9d5da05200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:17:30.933 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:30.932+0000 7f9d5da05200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:17:31.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:31.142+0000 7f9d5da05200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:17:31.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:31.343+0000 7f9d5da05200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:17:32.227 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:31.976+0000 7f9d5da05200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:17:32.228 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:32.211+0000 7f9d5da05200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:17:33.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:32.654+0000 7f9d5da05200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:17:33.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:32.755+0000 7f9d5da05200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:17:33.514 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:33.139+0000 7f9d5da05200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:17:33.829 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:33.513+0000 7f9d5da05200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:17:33.829 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:33.611+0000 7f9d5da05200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:17:34.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:33.870+0000 7f9d5da05200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:17:34.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:34.011+0000 7f9d5da05200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:17:34.452 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:34.113+0000 7f9d5da05200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:17:34.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:34.452+0000 7f9d5da05200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:17:34.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:34.549+0000 7f9d5da05200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:17:35.464 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:35 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:35.184+0000 7f9d5da05200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:17:35.464 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:35 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:35.410+0000 7f9d5da05200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:17:35.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: Active manager daemon x restarted 2024-05-08T05:17:35.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: Activating manager daemon x 2024-05-08T05:17:35.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: osdmap e2: 0 total, 0 up, 0 in 2024-05-08T05:17:35.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: mgrmap e6: x(active, starting, since 0.00339224s) 2024-05-08T05:17:35.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:17:35.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:17:35.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:17:35.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:17:35.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:17:35.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: Manager daemon x is now available 2024-05-08T05:17:35.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:35.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:35.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:17:35.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:35 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:17:36.785 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:36 smithi059 ceph-mon[30304]: Found migration_current of "None". Setting to last migration. 2024-05-08T05:17:36.785 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:36 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:17:36.785 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:36 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:17:36.785 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:36 smithi059 ceph-mon[30304]: mgrmap e7: x(active, since 1.0061s) 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.819+0000 7f8778b91700 1 Processor -- start 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.819+0000 7f8778b91700 1 -- start start 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.819+0000 7f8778b91700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f877410bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.819+0000 7f8778b91700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f877407cdb0 con 0x7f87741099b0 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.820+0000 7f877259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f877410bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:37.048 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.820+0000 7f877259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f877410bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39488/0 (socket says 172.21.15.59:39488) 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.820+0000 7f877259c700 1 -- 172.21.15.59:0/2380808708 learned_addr learned my addr 172.21.15.59:0/2380808708 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.820+0000 7f877259c700 1 -- 172.21.15.59:0/2380808708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f877410c2e0 con 0x7f87741099b0 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.820+0000 7f877259c700 1 --2- 172.21.15.59:0/2380808708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f877410bda0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f875c0098a0 tx=0x7f875c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc195d1659fb40bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.820+0000 7f877159a700 1 -- 172.21.15.59:0/2380808708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f875c004160 con 0x7f87741099b0 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.820+0000 7f877159a700 1 -- 172.21.15.59:0/2380808708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f875c004300 con 0x7f87741099b0 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.820+0000 7f877159a700 1 -- 172.21.15.59:0/2380808708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f875c011af0 con 0x7f87741099b0 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.821+0000 7f8778b91700 1 -- 172.21.15.59:0/2380808708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 msgr2=0x7f877410bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.821+0000 7f8778b91700 1 --2- 172.21.15.59:0/2380808708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f877410bda0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f875c0098a0 tx=0x7f875c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.821+0000 7f8778b91700 1 -- 172.21.15.59:0/2380808708 shutdown_connections 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.821+0000 7f8778b91700 1 --2- 172.21.15.59:0/2380808708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f877410bda0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.821+0000 7f8778b91700 1 -- 172.21.15.59:0/2380808708 >> 172.21.15.59:0/2380808708 conn(0x7f87741032c0 msgr2=0x7f8774105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.821+0000 7f8778b91700 1 -- 172.21.15.59:0/2380808708 shutdown_connections 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.821+0000 7f8778b91700 1 -- 172.21.15.59:0/2380808708 wait complete. 2024-05-08T05:17:37.049 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.822+0000 7f8778b91700 1 Processor -- start 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.822+0000 7f8778b91700 1 -- start start 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.822+0000 7f8778b91700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f8774109230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.822+0000 7f8778b91700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f877410d1e0 con 0x7f87741099b0 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.822+0000 7f877259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f8774109230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.822+0000 7f877259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f8774109230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39490/0 (socket says 172.21.15.59:39490) 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.822+0000 7f877259c700 1 -- 172.21.15.59:0/820059889 learned_addr learned my addr 172.21.15.59:0/820059889 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.823+0000 7f877259c700 1 -- 172.21.15.59:0/820059889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8774109770 con 0x7f87741099b0 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.823+0000 7f877259c700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f8774109230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f875c011ac0 tx=0x7f875c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.823+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f875c003970 con 0x7f87741099b0 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.823+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f875c003b10 con 0x7f87741099b0 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.823+0000 7f8778b91700 1 -- 172.21.15.59:0/820059889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8774107960 con 0x7f87741099b0 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.823+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f875c024610 con 0x7f87741099b0 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.823+0000 7f8778b91700 1 -- 172.21.15.59:0/820059889 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8774107ea0 con 0x7f87741099b0 2024-05-08T05:17:37.050 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.824+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 5) ==== 50222+0+0 (secure 0 0 0) 0x7f875c024840 con 0x7f87741099b0 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.824+0000 7f876b7fe700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 0x7f8760040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.824+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 --> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f8760041150 con 0x7f876003e580 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.824+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f875c0539a0 con 0x7f87741099b0 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.824+0000 7f8771d9b700 1 -- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 msgr2=0x7f8760040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817743540 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:30.824+0000 7f8771d9b700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 0x7f8760040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:31.025+0000 7f8771d9b700 1 -- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 msgr2=0x7f8760040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817743540 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:31.025+0000 7f8771d9b700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 0x7f8760040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:31.426+0000 7f8771d9b700 1 -- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 msgr2=0x7f8760040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817743540 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:31.426+0000 7f8771d9b700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 0x7f8760040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:32.227+0000 7f8771d9b700 1 -- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 msgr2=0x7f8760040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817743540 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:32.227+0000 7f8771d9b700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 0x7f8760040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:33.828+0000 7f8771d9b700 1 -- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 msgr2=0x7f8760040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/817743540 2024-05-08T05:17:37.051 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:33.828+0000 7f8771d9b700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 0x7f8760040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:35.413+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 6) ==== 49989+0+0 (secure 0 0 0) 0x7f875c024b30 con 0x7f87741099b0 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:35.413+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 msgr2=0x7f8760040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:35.413+0000 7f876b7fe700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 0x7f8760040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.415+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f875c055880 con 0x7f87741099b0 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.415+0000 7f876b7fe700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f8760041e70 0x7f8760044260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.415+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f875c021e70 con 0x7f8760041e70 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.416+0000 7f8771d9b700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f8760041e70 0x7f8760044260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.416+0000 7f8771d9b700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f8760041e70 0x7f8760044260 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8764003a10 tx=0x7f87640092b0 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.417+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 <== mgr.14120 v2:172.21.15.59:6800/95297313 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f875c021e70 con 0x7f8760041e70 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.425+0000 7f8778b91700 1 -- 172.21.15.59:0/820059889 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f877406a5d0 con 0x7f8760041e70 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.425+0000 7f876b7fe700 1 -- 172.21.15.59:0/820059889 <== mgr.14120 v2:172.21.15.59:6800/95297313 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f877406a5d0 con 0x7f8760041e70 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 -- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f8760041e70 msgr2=0x7f8760044260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f8760041e70 0x7f8760044260 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f8764003a10 tx=0x7f87640092b0 comp rx=0 tx=0).stop 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 -- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 msgr2=0x7f8774109230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:37.052 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f8774109230 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f875c011ac0 tx=0x7f875c011ba0 comp rx=0 tx=0).stop 2024-05-08T05:17:37.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 -- 172.21.15.59:0/820059889 shutdown_connections 2024-05-08T05:17:37.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f8760041e70 0x7f8760044260 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:37.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87741099b0 0x7f8774109230 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:37.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 --2- 172.21.15.59:0/820059889 >> [v2:172.21.15.59:6800/817743540,v1:172.21.15.59:6801/817743540] conn(0x7f876003e580 0x7f8760040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:37.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 -- 172.21.15.59:0/820059889 >> 172.21.15.59:0/820059889 conn(0x7f87741032c0 msgr2=0x7f8774103e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:37.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 -- 172.21.15.59:0/820059889 shutdown_connections 2024-05-08T05:17:37.053 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:36.426+0000 7f8778b91700 1 -- 172.21.15.59:0/820059889 wait complete. 2024-05-08T05:17:37.053 INFO:teuthology.orchestra.run.smithi059.stdout:mgr epoch 5 is available 2024-05-08T05:17:37.053 INFO:teuthology.orchestra.run.smithi059.stdout:Setting orchestrator backend to cephadm... 2024-05-08T05:17:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:37 smithi059 ceph-mon[30304]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-05-08T05:17:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:37 smithi059 ceph-mon[30304]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-05-08T05:17:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:37 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:37.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:37 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.424+0000 7f297901b700 1 Processor -- start 2024-05-08T05:17:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.424+0000 7f297901b700 1 -- start start 2024-05-08T05:17:38.224 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.425+0000 7f297901b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974107560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.425+0000 7f297901b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2974107b30 con 0x7f2974107140 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.425+0000 7f2972d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974107560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.425+0000 7f2972d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974107560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39566/0 (socket says 172.21.15.59:39566) 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.425+0000 7f2972d9d700 1 -- 172.21.15.59:0/3766903149 learned_addr learned my addr 172.21.15.59:0/3766903149 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.425+0000 7f2972d9d700 1 -- 172.21.15.59:0/3766903149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f297410a0d0 con 0x7f2974107140 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.425+0000 7f2972d9d700 1 --2- 172.21.15.59:0/3766903149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974107560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29640098a0 tx=0x7f2964009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea1484a329d1825c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.425+0000 7f2971d9b700 1 -- 172.21.15.59:0/3766903149 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2964004160 con 0x7f2974107140 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.426+0000 7f2971d9b700 1 -- 172.21.15.59:0/3766903149 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2964004300 con 0x7f2974107140 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.426+0000 7f2971d9b700 1 -- 172.21.15.59:0/3766903149 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2964011af0 con 0x7f2974107140 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.426+0000 7f297901b700 1 -- 172.21.15.59:0/3766903149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 msgr2=0x7f2974107560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.426+0000 7f297901b700 1 --2- 172.21.15.59:0/3766903149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974107560 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f29640098a0 tx=0x7f2964009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.426+0000 7f297901b700 1 -- 172.21.15.59:0/3766903149 shutdown_connections 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.426+0000 7f297901b700 1 --2- 172.21.15.59:0/3766903149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974107560 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:38.225 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.426+0000 7f297901b700 1 -- 172.21.15.59:0/3766903149 >> 172.21.15.59:0/3766903149 conn(0x7f2974102a30 msgr2=0x7f2974104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.426+0000 7f297901b700 1 -- 172.21.15.59:0/3766903149 shutdown_connections 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.426+0000 7f297901b700 1 -- 172.21.15.59:0/3766903149 wait complete. 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.427+0000 7f297901b700 1 Processor -- start 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.427+0000 7f297901b700 1 -- start start 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.427+0000 7f297901b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974109b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.427+0000 7f297901b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f297410d480 con 0x7f2974107140 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.427+0000 7f2972d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974109b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.427+0000 7f2972d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974109b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39570/0 (socket says 172.21.15.59:39570) 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.427+0000 7f2972d9d700 1 -- 172.21.15.59:0/523020555 learned_addr learned my addr 172.21.15.59:0/523020555 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.428+0000 7f2972d9d700 1 -- 172.21.15.59:0/523020555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2974108080 con 0x7f2974107140 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.428+0000 7f2972d9d700 1 --2- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974109b30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29640117b0 tx=0x7f2964011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.428+0000 7f296bfff700 1 -- 172.21.15.59:0/523020555 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2964003af0 con 0x7f2974107140 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.428+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2974108310 con 0x7f2974107140 2024-05-08T05:17:38.226 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.428+0000 7f296bfff700 1 -- 172.21.15.59:0/523020555 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2964020070 con 0x7f2974107140 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.428+0000 7f296bfff700 1 -- 172.21.15.59:0/523020555 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2964019630 con 0x7f2974107140 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.428+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29741087f0 con 0x7f2974107140 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.429+0000 7f296bfff700 1 -- 172.21.15.59:0/523020555 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 7) ==== 50116+0+0 (secure 0 0 0) 0x7f296401a430 con 0x7f2974107140 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.429+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2974064b00 con 0x7f2974107140 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.429+0000 7f296bfff700 1 --2- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f296003e460 0x7f2960040920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.430+0000 7f296bfff700 1 -- 172.21.15.59:0/523020555 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f2964053910 con 0x7f2974107140 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.430+0000 7f297259c700 1 --2- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f296003e460 0x7f2960040920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.430+0000 7f297259c700 1 --2- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f296003e460 0x7f2960040920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f295c006fd0 tx=0x7f295c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.435+0000 7f296bfff700 1 -- 172.21.15.59:0/523020555 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29640042a0 con 0x7f2974107140 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.690+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f2974108e10 con 0x7f296003e460 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.698+0000 7f296bfff700 1 -- 172.21.15.59:0/523020555 <== mgr.14120 v2:172.21.15.59:6800/95297313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f2974108e10 con 0x7f296003e460 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.700+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f296003e460 msgr2=0x7f2960040920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.700+0000 7f297901b700 1 --2- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f296003e460 0x7f2960040920 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f295c006fd0 tx=0x7f295c006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.700+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 msgr2=0x7f2974109b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.700+0000 7f297901b700 1 --2- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974109b30 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f29640117b0 tx=0x7f2964011b70 comp rx=0 tx=0).stop 2024-05-08T05:17:38.227 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.700+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 shutdown_connections 2024-05-08T05:17:38.228 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.700+0000 7f297901b700 1 --2- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f296003e460 0x7f2960040920 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:38.228 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.700+0000 7f297901b700 1 --2- 172.21.15.59:0/523020555 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2974107140 0x7f2974109b30 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:38.228 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.700+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 >> 172.21.15.59:0/523020555 conn(0x7f2974102a30 msgr2=0x7f2974103560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:38.228 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.700+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 shutdown_connections 2024-05-08T05:17:38.228 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:37.701+0000 7f297901b700 1 -- 172.21.15.59:0/523020555 wait complete. 2024-05-08T05:17:39.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:38 smithi059 ceph-mon[30304]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:17:39.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:38 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:39.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:38 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:17:39.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:38 smithi059 ceph-mon[30304]: mgrmap e8: x(active, since 2s) 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout value unchanged 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.594+0000 7f49d77e0700 1 Processor -- start 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.595+0000 7f49d77e0700 1 -- start start 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.595+0000 7f49d77e0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d0107b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.595+0000 7f49d77e0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f49d007ce30 con 0x7f49d0107720 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.595+0000 7f49d557c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d0107b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.595+0000 7f49d557c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d0107b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39572/0 (socket says 172.21.15.59:39572) 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.595+0000 7f49d557c700 1 -- 172.21.15.59:0/3273865958 learned_addr learned my addr 172.21.15.59:0/3273865958 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.595+0000 7f49d557c700 1 -- 172.21.15.59:0/3273865958 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d0108080 con 0x7f49d0107720 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.596+0000 7f49d557c700 1 --2- 172.21.15.59:0/3273865958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d0107b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0098a0 tx=0x7f49cc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=697129dce323e549 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:39.423 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.596+0000 7f49c7fff700 1 -- 172.21.15.59:0/3273865958 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49cc004160 con 0x7f49d0107720 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.596+0000 7f49c7fff700 1 -- 172.21.15.59:0/3273865958 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49cc004300 con 0x7f49d0107720 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.596+0000 7f49c7fff700 1 -- 172.21.15.59:0/3273865958 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49cc011af0 con 0x7f49d0107720 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.596+0000 7f49d77e0700 1 -- 172.21.15.59:0/3273865958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 msgr2=0x7f49d0107b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.596+0000 7f49d77e0700 1 --2- 172.21.15.59:0/3273865958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d0107b40 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f49cc0098a0 tx=0x7f49cc009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.597+0000 7f49d77e0700 1 -- 172.21.15.59:0/3273865958 shutdown_connections 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.597+0000 7f49d77e0700 1 --2- 172.21.15.59:0/3273865958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d0107b40 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.597+0000 7f49d77e0700 1 -- 172.21.15.59:0/3273865958 >> 172.21.15.59:0/3273865958 conn(0x7f49d01032c0 msgr2=0x7f49d0105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.597+0000 7f49d77e0700 1 -- 172.21.15.59:0/3273865958 shutdown_connections 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.597+0000 7f49d77e0700 1 -- 172.21.15.59:0/3273865958 wait complete. 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.597+0000 7f49d77e0700 1 Processor -- start 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.597+0000 7f49d77e0700 1 -- start start 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49d77e0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d019ef90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:39.424 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49d77e0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f49d01090b0 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49d557c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d019ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49d557c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d019ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39574/0 (socket says 172.21.15.59:39574) 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49d557c700 1 -- 172.21.15.59:0/1613028989 learned_addr learned my addr 172.21.15.59:0/1613028989 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49d557c700 1 -- 172.21.15.59:0/1613028989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49d019f4d0 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49d557c700 1 --2- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d019ef90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009fb0 tx=0x7f49cc011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49c67fc700 1 -- 172.21.15.59:0/1613028989 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49cc0036a0 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49c67fc700 1 -- 172.21.15.59:0/1613028989 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f49cc01a430 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49c67fc700 1 -- 172.21.15.59:0/1613028989 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49cc019550 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49d019f760 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.598+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49d01a2450 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.600+0000 7f49c67fc700 1 -- 172.21.15.59:0/1613028989 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f49cc019800 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.600+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49d006a5d0 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.600+0000 7f49c67fc700 1 --2- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f49bc03e530 0x7f49bc0409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.600+0000 7f49c67fc700 1 -- 172.21.15.59:0/1613028989 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f49cc053990 con 0x7f49d0107720 2024-05-08T05:17:39.425 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.600+0000 7f49d4d7b700 1 --2- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f49bc03e530 0x7f49bc0409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.600+0000 7f49d4d7b700 1 --2- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f49bc03e530 0x7f49bc0409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006fd0 tx=0x7f49c0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.605+0000 7f49c67fc700 1 -- 172.21.15.59:0/1613028989 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f49cc018d00 con 0x7f49d0107720 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.828+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f49d00008d0 con 0x7f49bc03e530 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.828+0000 7f49c67fc700 1 -- 172.21.15.59:0/1613028989 <== mgr.14120 v2:172.21.15.59:6800/95297313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f49d00008d0 con 0x7f49bc03e530 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.830+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f49bc03e530 msgr2=0x7f49bc0409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.830+0000 7f49d77e0700 1 --2- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f49bc03e530 0x7f49bc0409f0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f49c0006fd0 tx=0x7f49c0006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.830+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 msgr2=0x7f49d019ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.830+0000 7f49d77e0700 1 --2- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d019ef90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f49cc009fb0 tx=0x7f49cc011940 comp rx=0 tx=0).stop 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.831+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 shutdown_connections 2024-05-08T05:17:39.426 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.831+0000 7f49d77e0700 1 --2- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f49bc03e530 0x7f49bc0409f0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:39.427 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.831+0000 7f49d77e0700 1 --2- 172.21.15.59:0/1613028989 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49d0107720 0x7f49d019ef90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:39.427 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.831+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 >> 172.21.15.59:0/1613028989 conn(0x7f49d01032c0 msgr2=0x7f49d0195c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:39.427 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.831+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 shutdown_connections 2024-05-08T05:17:39.427 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:38.831+0000 7f49d77e0700 1 -- 172.21.15.59:0/1613028989 wait complete. 2024-05-08T05:17:39.427 INFO:teuthology.orchestra.run.smithi059.stdout:Generating ssh key... 2024-05-08T05:17:40.080 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:39 smithi059 ceph-mon[30304]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:17:40.080 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:39 smithi059 ceph-mon[30304]: [08/May/2024:05:17:38] ENGINE Bus STARTING 2024-05-08T05:17:40.080 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:39 smithi059 ceph-mon[30304]: [08/May/2024:05:17:38] ENGINE Serving on https://172.21.15.59:7150 2024-05-08T05:17:40.080 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:39 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:17:40.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: Generating public/private rsa key pair. 2024-05-08T05:17:40.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: Your identification has been saved in /tmp/tmpentqpd6l/key. 2024-05-08T05:17:40.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: Your public key has been saved in /tmp/tmpentqpd6l/key.pub. 2024-05-08T05:17:40.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: The key fingerprint is: 2024-05-08T05:17:40.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: SHA256:qlKAr4w0lXPQw5SKsqJ88vq5WXkh38yJhFfmk6NhAB0 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:17:40.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: The key's randomart image is: 2024-05-08T05:17:40.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: +---[RSA 3072]----+ 2024-05-08T05:17:40.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: | .=Eo | 2024-05-08T05:17:40.338 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: | ..* | 2024-05-08T05:17:40.338 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: | .. +.. o | 2024-05-08T05:17:40.338 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: |o..= .o + . | 2024-05-08T05:17:40.338 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: |.o..oo *S= | 2024-05-08T05:17:40.338 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: |oo. . B.O + | 2024-05-08T05:17:40.338 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: |B... o.= = | 2024-05-08T05:17:40.338 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: |++..+.. | 2024-05-08T05:17:40.338 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: | .*Bo | 2024-05-08T05:17:40.338 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: +----[SHA256]-----+ 2024-05-08T05:17:40.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.849+0000 7f840a7d5700 1 Processor -- start 2024-05-08T05:17:40.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.849+0000 7f840a7d5700 1 -- start start 2024-05-08T05:17:40.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.850+0000 7f840a7d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840410bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:40.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.850+0000 7f840a7d5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f840407ce70 con 0x7f84041098e0 2024-05-08T05:17:40.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.850+0000 7f8403fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840410bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:40.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.850+0000 7f8403fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840410bcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34348/0 (socket says 172.21.15.59:34348) 2024-05-08T05:17:40.837 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.850+0000 7f8403fff700 1 -- 172.21.15.59:0/3557863517 learned_addr learned my addr 172.21.15.59:0/3557863517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.850+0000 7f8403fff700 1 -- 172.21.15.59:0/3557863517 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840410c210 con 0x7f84041098e0 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.850+0000 7f8403fff700 1 --2- 172.21.15.59:0/3557863517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840410bcd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098a0 tx=0x7f83f4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e9a4029c76e785f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f8402ffd700 1 -- 172.21.15.59:0/3557863517 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83f4004160 con 0x7f84041098e0 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f8402ffd700 1 -- 172.21.15.59:0/3557863517 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f83f4004300 con 0x7f84041098e0 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f8402ffd700 1 -- 172.21.15.59:0/3557863517 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83f4011af0 con 0x7f84041098e0 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f840a7d5700 1 -- 172.21.15.59:0/3557863517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 msgr2=0x7f840410bcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f840a7d5700 1 --2- 172.21.15.59:0/3557863517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840410bcd0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f83f40098a0 tx=0x7f83f4009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f840a7d5700 1 -- 172.21.15.59:0/3557863517 shutdown_connections 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f840a7d5700 1 --2- 172.21.15.59:0/3557863517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840410bcd0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f840a7d5700 1 -- 172.21.15.59:0/3557863517 >> 172.21.15.59:0/3557863517 conn(0x7f84041032b0 msgr2=0x7f8404105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f840a7d5700 1 -- 172.21.15.59:0/3557863517 shutdown_connections 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.851+0000 7f840a7d5700 1 -- 172.21.15.59:0/3557863517 wait complete. 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.852+0000 7f840a7d5700 1 Processor -- start 2024-05-08T05:17:40.838 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.852+0000 7f840a7d5700 1 -- start start 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.852+0000 7f840a7d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840419ce10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.852+0000 7f840a7d5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f840410d160 con 0x7f84041098e0 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.852+0000 7f8403fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840419ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.852+0000 7f8403fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840419ce10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34356/0 (socket says 172.21.15.59:34356) 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.852+0000 7f8403fff700 1 -- 172.21.15.59:0/3768102050 learned_addr learned my addr 172.21.15.59:0/3768102050 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.853+0000 7f8403fff700 1 -- 172.21.15.59:0/3768102050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f840419d350 con 0x7f84041098e0 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.853+0000 7f8403fff700 1 --2- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840419ce10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009fb0 tx=0x7f83f4011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.853+0000 7f84017fa700 1 -- 172.21.15.59:0/3768102050 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83f40036a0 con 0x7f84041098e0 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.853+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f840419d640 con 0x7f84041098e0 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.853+0000 7f84017fa700 1 -- 172.21.15.59:0/3768102050 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f83f401a430 con 0x7f84041098e0 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.853+0000 7f84017fa700 1 -- 172.21.15.59:0/3768102050 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83f4019550 con 0x7f84041098e0 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.853+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f840419e2a0 con 0x7f84041098e0 2024-05-08T05:17:40.839 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.854+0000 7f84017fa700 1 -- 172.21.15.59:0/3768102050 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f83f4003840 con 0x7f84041098e0 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.854+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8404064b00 con 0x7f84041098e0 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.854+0000 7f84017fa700 1 --2- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f83f003e4e0 0x7f83f00409a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.854+0000 7f84017fa700 1 -- 172.21.15.59:0/3768102050 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f83f40537c0 con 0x7f84041098e0 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.855+0000 7f84037fe700 1 --2- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f83f003e4e0 0x7f83f00409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.855+0000 7f84037fe700 1 --2- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f83f003e4e0 0x7f83f00409a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f83ec006fd0 tx=0x7f83ec006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:39.860+0000 7f84017fa700 1 -- 172.21.15.59:0/3768102050 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f83f4003b30 con 0x7f84041098e0 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.079+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f84041044d0 con 0x7f83f003e4e0 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.225+0000 7f84017fa700 1 -- 172.21.15.59:0/3768102050 <== mgr.14120 v2:172.21.15.59:6800/95297313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f84041044d0 con 0x7f83f003e4e0 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f83f003e4e0 msgr2=0x7f83f00409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:40.840 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 --2- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f83f003e4e0 0x7f83f00409a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f83ec006fd0 tx=0x7f83ec006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:40.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 msgr2=0x7f840419ce10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:40.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 --2- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840419ce10 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f83f4009fb0 tx=0x7f83f4011940 comp rx=0 tx=0).stop 2024-05-08T05:17:40.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 shutdown_connections 2024-05-08T05:17:40.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 --2- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f83f003e4e0 0x7f83f00409a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:40.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 --2- 172.21.15.59:0/3768102050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84041098e0 0x7f840419ce10 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:40.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 >> 172.21.15.59:0/3768102050 conn(0x7f84041032b0 msgr2=0x7f8404103dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:40.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 shutdown_connections 2024-05-08T05:17:40.841 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:40.228+0000 7f840a7d5700 1 -- 172.21.15.59:0/3768102050 wait complete. 2024-05-08T05:17:40.954 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:40 smithi059 ceph-mon[30304]: [08/May/2024:05:17:39] ENGINE Serving on http://172.21.15.59:8765 2024-05-08T05:17:40.954 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:40 smithi059 ceph-mon[30304]: [08/May/2024:05:17:39] ENGINE Bus STARTED 2024-05-08T05:17:40.954 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:40 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:40.954 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:40 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:41.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:41 smithi059 ceph-mon[30304]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:17:41.957 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:41 smithi059 ceph-mon[30304]: Generating ssh key... 2024-05-08T05:17:42.042 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/LY86J0TNyKyLtTYInAC8Sz4DMgi1qJDt7LzAXVoDYvBOGdTPXDeXmnE/HlHjvKXYdJP06z1+0MOBi9uvsrv5bZKN2E9SCiHW0s+cjJEqXTpGJotrpR/iQFyAoWXSjUFgrd9h3WBDlwzXETsKFAkPVR3ZuP3YR+PS8Zxe4wfFrkypkw4JxfVnnwERS5EKpR+LizRpJF40s3iM+0vKcvv+oaGJ7keE4uER2zPSPOGwfRwmuJy6ropjt2uZl8y5Zq5HKCGETDk6DOFMLJP1xANSUYCcqiHUyU8vJX9U05FyWosecHewxzQcuq6u7FIyztMwXKVHRjZ9WyZ8p3RHklcNk2NZpMxzT1IxaApEhjPYk/MInCHxcjZpxRvb5trko8LJW2MbUC2SzFnhk8pMwp4+GoLuuE+6YF9rLfCjLQOKT/AcDq0tFkM+mxA0SWk+IogaYpzcG+5bdCd6LT8fkAKqYAQIjMrtTshbVNGYI9kgzbta5tiLgDJsddw/APnYwlM= ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.243+0000 7fe49734a700 1 Processor -- start 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.243+0000 7fe49734a700 1 -- start start 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.243+0000 7fe49734a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49007db50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.243+0000 7fe49734a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe49007e120 con 0x7fe49007f6f0 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.243+0000 7fe4950e6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49007db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.243+0000 7fe4950e6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49007db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34358/0 (socket says 172.21.15.59:34358) 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.244+0000 7fe4950e6700 1 -- 172.21.15.59:0/3566918308 learned_addr learned my addr 172.21.15.59:0/3566918308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.244+0000 7fe4950e6700 1 -- 172.21.15.59:0/3566918308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49007e2a0 con 0x7fe49007f6f0 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.244+0000 7fe4950e6700 1 --2- 172.21.15.59:0/3566918308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49007db50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0098a0 tx=0x7fe48c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f8cf0853de94d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.244+0000 7fe487fff700 1 -- 172.21.15.59:0/3566918308 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c004160 con 0x7fe49007f6f0 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.244+0000 7fe487fff700 1 -- 172.21.15.59:0/3566918308 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe48c004300 con 0x7fe49007f6f0 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.244+0000 7fe487fff700 1 -- 172.21.15.59:0/3566918308 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c011af0 con 0x7fe49007f6f0 2024-05-08T05:17:42.043 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.245+0000 7fe49734a700 1 -- 172.21.15.59:0/3566918308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 msgr2=0x7fe49007db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.245+0000 7fe49734a700 1 --2- 172.21.15.59:0/3566918308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49007db50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0098a0 tx=0x7fe48c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.245+0000 7fe49734a700 1 -- 172.21.15.59:0/3566918308 shutdown_connections 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.245+0000 7fe49734a700 1 --2- 172.21.15.59:0/3566918308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49007db50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.245+0000 7fe49734a700 1 -- 172.21.15.59:0/3566918308 >> 172.21.15.59:0/3566918308 conn(0x7fe490104410 msgr2=0x7fe490106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.245+0000 7fe49734a700 1 -- 172.21.15.59:0/3566918308 shutdown_connections 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.245+0000 7fe49734a700 1 -- 172.21.15.59:0/3566918308 wait complete. 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.245+0000 7fe49734a700 1 Processor -- start 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe49734a700 1 -- start start 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe49734a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49019ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe49734a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe49010f400 con 0x7fe49007f6f0 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe4950e6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49019ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe4950e6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49019ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34368/0 (socket says 172.21.15.59:34368) 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe4950e6700 1 -- 172.21.15.59:0/2823640756 learned_addr learned my addr 172.21.15.59:0/2823640756 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe4950e6700 1 -- 172.21.15.59:0/2823640756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe49019f4a0 con 0x7fe49007f6f0 2024-05-08T05:17:42.044 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe4950e6700 1 --2- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49019ef60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe48c011ac0 tx=0x7fe48c011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe4867fc700 1 -- 172.21.15.59:0/2823640756 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c003b20 con 0x7fe49007f6f0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe4867fc700 1 -- 172.21.15.59:0/2823640756 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe48c01a8c0 con 0x7fe49007f6f0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.246+0000 7fe4867fc700 1 -- 172.21.15.59:0/2823640756 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe48c0248d0 con 0x7fe49007f6f0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.247+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe49019f730 con 0x7fe49007f6f0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.247+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4901a0250 con 0x7fe49007f6f0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.248+0000 7fe4867fc700 1 -- 172.21.15.59:0/2823640756 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fe48c022020 con 0x7fe49007f6f0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.248+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe490064b00 con 0x7fe49007f6f0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.248+0000 7fe4867fc700 1 --2- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fe47c03e580 0x7fe47c040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.248+0000 7fe4867fc700 1 -- 172.21.15.59:0/2823640756 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe48c053810 con 0x7fe49007f6f0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.248+0000 7fe4948e5700 1 --2- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fe47c03e580 0x7fe47c040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.248+0000 7fe4948e5700 1 --2- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fe47c03e580 0x7fe47c040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe480006fd0 tx=0x7fe480006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.253+0000 7fe4867fc700 1 -- 172.21.15.59:0/2823640756 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe48c01ab20 con 0x7fe49007f6f0 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.472+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7fe490105590 con 0x7fe47c03e580 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.472+0000 7fe4867fc700 1 -- 172.21.15.59:0/2823640756 <== mgr.14120 v2:172.21.15.59:6800/95297313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7fe490105590 con 0x7fe47c03e580 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.474+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fe47c03e580 msgr2=0x7fe47c040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:42.045 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.474+0000 7fe49734a700 1 --2- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fe47c03e580 0x7fe47c040a40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fe480006fd0 tx=0x7fe480006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.474+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 msgr2=0x7fe49019ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.474+0000 7fe49734a700 1 --2- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49019ef60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe48c011ac0 tx=0x7fe48c011ba0 comp rx=0 tx=0).stop 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.474+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 shutdown_connections 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.474+0000 7fe49734a700 1 --2- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fe47c03e580 0x7fe47c040a40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.474+0000 7fe49734a700 1 --2- 172.21.15.59:0/2823640756 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe49007f6f0 0x7fe49019ef60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.474+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 >> 172.21.15.59:0/2823640756 conn(0x7fe490104410 msgr2=0x7fe490104e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.475+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 shutdown_connections 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:41.475+0000 7fe49734a700 1 -- 172.21.15.59:0/2823640756 wait complete. 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:Adding key to root@localhost authorized_keys... 2024-05-08T05:17:42.046 INFO:teuthology.orchestra.run.smithi059.stdout:Adding host smithi059... 2024-05-08T05:17:43.019 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:42 smithi059 ceph-mon[30304]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:17:44.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:43 smithi059 ceph-mon[30304]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:17:45.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:44 smithi059 ceph-mon[30304]: Deploying cephadm binary to smithi059 2024-05-08T05:17:47.584 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Added host 'smithi059' with addr '172.21.15.59' 2024-05-08T05:17:47.584 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.456+0000 7f312526a700 1 Processor -- start 2024-05-08T05:17:47.584 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.456+0000 7f312526a700 1 -- start start 2024-05-08T05:17:47.584 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.456+0000 7f312526a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f3120109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.456+0000 7f312526a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3120109790 con 0x7f3120108e30 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.456+0000 7f311effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f3120109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.456+0000 7f311effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f3120109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34380/0 (socket says 172.21.15.59:34380) 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.456+0000 7f311effd700 1 -- 172.21.15.59:0/3608349737 learned_addr learned my addr 172.21.15.59:0/3608349737 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.457+0000 7f311effd700 1 -- 172.21.15.59:0/3608349737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3120109f90 con 0x7f3120108e30 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.457+0000 7f311effd700 1 --2- 172.21.15.59:0/3608349737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f3120109250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3110009e80 tx=0x7f311000b330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efccafb28f16e935 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.457+0000 7f311e7fc700 1 -- 172.21.15.59:0/3608349737 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3110017070 con 0x7f3120108e30 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.457+0000 7f311e7fc700 1 -- 172.21.15.59:0/3608349737 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f311000b8c0 con 0x7f3120108e30 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.457+0000 7f311e7fc700 1 -- 172.21.15.59:0/3608349737 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3110004490 con 0x7f3120108e30 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.458+0000 7f312526a700 1 -- 172.21.15.59:0/3608349737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 msgr2=0x7f3120109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.458+0000 7f312526a700 1 --2- 172.21.15.59:0/3608349737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f3120109250 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f3110009e80 tx=0x7f311000b330 comp rx=0 tx=0).stop 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.458+0000 7f312526a700 1 -- 172.21.15.59:0/3608349737 shutdown_connections 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.458+0000 7f312526a700 1 --2- 172.21.15.59:0/3608349737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f3120109250 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:47.585 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.458+0000 7f312526a700 1 -- 172.21.15.59:0/3608349737 >> 172.21.15.59:0/3608349737 conn(0x7f3120104410 msgr2=0x7f3120106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.458+0000 7f312526a700 1 -- 172.21.15.59:0/3608349737 shutdown_connections 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.458+0000 7f312526a700 1 -- 172.21.15.59:0/3608349737 wait complete. 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.458+0000 7f312526a700 1 Processor -- start 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.459+0000 7f312526a700 1 -- start start 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.459+0000 7f312526a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f312019cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.459+0000 7f312526a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f312010d3d0 con 0x7f3120108e30 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.459+0000 7f311effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f312019cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.459+0000 7f311effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f312019cdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34382/0 (socket says 172.21.15.59:34382) 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.459+0000 7f311effd700 1 -- 172.21.15.59:0/2805880372 learned_addr learned my addr 172.21.15.59:0/2805880372 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.459+0000 7f311effd700 1 -- 172.21.15.59:0/2805880372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f312019d2f0 con 0x7f3120108e30 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.459+0000 7f311effd700 1 --2- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f312019cdb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f311000b7b0 tx=0x7f311000be20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.460+0000 7f311cff9700 1 -- 172.21.15.59:0/2805880372 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3110020020 con 0x7f3120108e30 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.460+0000 7f311cff9700 1 -- 172.21.15.59:0/2805880372 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3110012430 con 0x7f3120108e30 2024-05-08T05:17:47.586 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.460+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f312019d580 con 0x7f3120108e30 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.460+0000 7f311cff9700 1 -- 172.21.15.59:0/2805880372 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3110017040 con 0x7f3120108e30 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.460+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f312019e1b0 con 0x7f3120108e30 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.461+0000 7f311cff9700 1 -- 172.21.15.59:0/2805880372 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3110023020 con 0x7f3120108e30 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.461+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3120064b00 con 0x7f3120108e30 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.461+0000 7f311cff9700 1 --2- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f310c03e530 0x7f310c0409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.461+0000 7f311cff9700 1 -- 172.21.15.59:0/2805880372 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f311000f050 con 0x7f3120108e30 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.461+0000 7f31167fc700 1 --2- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f310c03e530 0x7f310c0409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.461+0000 7f31167fc700 1 --2- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f310c03e530 0x7f310c0409f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3108006fd0 tx=0x7f3108006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.466+0000 7f311cff9700 1 -- 172.21.15.59:0/2805880372 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f311002a020 con 0x7f3120108e30 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:42.686+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi059", "addr": "172.21.15.59", "target": ["mon-mgr", ""]}) -- 0x7f31201054f0 con 0x7f310c03e530 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.979+0000 7f311cff9700 1 -- 172.21.15.59:0/2805880372 <== mgr.14120 v2:172.21.15.59:6800/95297313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f31201054f0 con 0x7f310c03e530 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f310c03e530 msgr2=0x7f310c0409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 --2- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f310c03e530 0x7f310c0409f0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f3108006fd0 tx=0x7f3108006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 msgr2=0x7f312019cdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:47.587 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 --2- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f312019cdb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f311000b7b0 tx=0x7f311000be20 comp rx=0 tx=0).stop 2024-05-08T05:17:47.588 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 shutdown_connections 2024-05-08T05:17:47.588 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 --2- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f310c03e530 0x7f310c0409f0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:47.588 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 --2- 172.21.15.59:0/2805880372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3120108e30 0x7f312019cdb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:47.588 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 >> 172.21.15.59:0/2805880372 conn(0x7f3120104410 msgr2=0x7f3120104de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:47.588 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 shutdown_connections 2024-05-08T05:17:47.588 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:46.982+0000 7f312526a700 1 -- 172.21.15.59:0/2805880372 wait complete. 2024-05-08T05:17:47.588 INFO:teuthology.orchestra.run.smithi059.stdout:Deploying unmanaged mon service... 2024-05-08T05:17:48.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:47 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:48.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:47 smithi059 ceph-mon[30304]: Added host smithi059 2024-05-08T05:17:48.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:47 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:17:49.457 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:49 smithi059 ceph-mon[30304]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:17:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:49 smithi059 ceph-mon[30304]: Saving service mon spec with placement count:5 2024-05-08T05:17:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:49 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:49.458 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:49 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.111+0000 7f431dd7a700 1 Processor -- start 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.111+0000 7f431dd7a700 1 -- start start 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.111+0000 7f431dd7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f4318109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.111+0000 7f431dd7a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4318109790 con 0x7f4318108e30 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.111+0000 7f430f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f4318109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.111+0000 7f430f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f4318109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34396/0 (socket says 172.21.15.59:34396) 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.111+0000 7f430f7fe700 1 -- 172.21.15.59:0/3529680342 learned_addr learned my addr 172.21.15.59:0/3529680342 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.111+0000 7f430f7fe700 1 -- 172.21.15.59:0/3529680342 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4318109f90 con 0x7f4318108e30 2024-05-08T05:17:49.716 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.111+0000 7f430f7fe700 1 --2- 172.21.15.59:0/3529680342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f4318109250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f43080098a0 tx=0x7f4308009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e8be16f88d7345 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f4316ffd700 1 -- 172.21.15.59:0/3529680342 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4308004160 con 0x7f4318108e30 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f4316ffd700 1 -- 172.21.15.59:0/3529680342 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4308004300 con 0x7f4318108e30 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f4316ffd700 1 -- 172.21.15.59:0/3529680342 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4308011af0 con 0x7f4318108e30 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f431dd7a700 1 -- 172.21.15.59:0/3529680342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 msgr2=0x7f4318109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f431dd7a700 1 --2- 172.21.15.59:0/3529680342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f4318109250 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f43080098a0 tx=0x7f4308009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f431dd7a700 1 -- 172.21.15.59:0/3529680342 shutdown_connections 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f431dd7a700 1 --2- 172.21.15.59:0/3529680342 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f4318109250 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f431dd7a700 1 -- 172.21.15.59:0/3529680342 >> 172.21.15.59:0/3529680342 conn(0x7f4318104410 msgr2=0x7f4318106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f431dd7a700 1 -- 172.21.15.59:0/3529680342 shutdown_connections 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.112+0000 7f431dd7a700 1 -- 172.21.15.59:0/3529680342 wait complete. 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.113+0000 7f431dd7a700 1 Processor -- start 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.113+0000 7f431dd7a700 1 -- start start 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.113+0000 7f431dd7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f431819ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.113+0000 7f431dd7a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f431807d950 con 0x7f4318108e30 2024-05-08T05:17:49.717 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.113+0000 7f430f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f431819ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.113+0000 7f430f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f431819ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34398/0 (socket says 172.21.15.59:34398) 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.113+0000 7f430f7fe700 1 -- 172.21.15.59:0/2233305229 learned_addr learned my addr 172.21.15.59:0/2233305229 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.114+0000 7f430f7fe700 1 -- 172.21.15.59:0/2233305229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f431819f4c0 con 0x7f4318108e30 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.114+0000 7f430f7fe700 1 --2- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f431819ef80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4308011ac0 tx=0x7f4308011ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.114+0000 7f43157fa700 1 -- 172.21.15.59:0/2233305229 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4308003b20 con 0x7f4318108e30 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.114+0000 7f43157fa700 1 -- 172.21.15.59:0/2233305229 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f430801a8c0 con 0x7f4318108e30 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.114+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f431819f750 con 0x7f4318108e30 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.114+0000 7f43157fa700 1 -- 172.21.15.59:0/2233305229 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f43080248d0 con 0x7f4318108e30 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.114+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43181a0270 con 0x7f4318108e30 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.115+0000 7f43157fa700 1 -- 172.21.15.59:0/2233305229 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f4308022020 con 0x7f4318108e30 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.115+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4318064b00 con 0x7f4318108e30 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.115+0000 7f43157fa700 1 --2- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f430403e530 0x7f43040409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.115+0000 7f43157fa700 1 -- 172.21.15.59:0/2233305229 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f43080537f0 con 0x7f4318108e30 2024-05-08T05:17:49.718 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.115+0000 7f43177fe700 1 --2- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f430403e530 0x7f43040409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.116+0000 7f43177fe700 1 --2- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f430403e530 0x7f43040409f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4300006fd0 tx=0x7f4300006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.121+0000 7f43157fa700 1 -- 172.21.15.59:0/2233305229 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f430801aa70 con 0x7f4318108e30 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.359+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f43181a0fb0 con 0x7f430403e530 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.366+0000 7f43157fa700 1 -- 172.21.15.59:0/2233305229 <== mgr.14120 v2:172.21.15.59:6800/95297313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f43181a0fb0 con 0x7f430403e530 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f430403e530 msgr2=0x7f43040409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 --2- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f430403e530 0x7f43040409f0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4300006fd0 tx=0x7f4300006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 msgr2=0x7f431819ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 --2- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f431819ef80 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f4308011ac0 tx=0x7f4308011ba0 comp rx=0 tx=0).stop 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 shutdown_connections 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 --2- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f430403e530 0x7f43040409f0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 --2- 172.21.15.59:0/2233305229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4318108e30 0x7f431819ef80 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 >> 172.21.15.59:0/2233305229 conn(0x7f4318104410 msgr2=0x7f4318106800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 shutdown_connections 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:48.369+0000 7f431dd7a700 1 -- 172.21.15.59:0/2233305229 wait complete. 2024-05-08T05:17:49.719 INFO:teuthology.orchestra.run.smithi059.stdout:Deploying unmanaged mgr service... 2024-05-08T05:17:50.932 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.139+0000 7f3a1597a700 1 Processor -- start 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.139+0000 7f3a1597a700 1 -- start start 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.139+0000 7f3a1597a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a10109300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.139+0000 7f3a1597a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a10109840 con 0x7f3a10108ee0 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.139+0000 7f3a0effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a10109300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.139+0000 7f3a0effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a10109300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45008/0 (socket says 172.21.15.59:45008) 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.139+0000 7f3a0effd700 1 -- 172.21.15.59:0/3711663408 learned_addr learned my addr 172.21.15.59:0/3711663408 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.140+0000 7f3a0effd700 1 -- 172.21.15.59:0/3711663408 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a10109fa0 con 0x7f3a10108ee0 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.140+0000 7f3a0effd700 1 --2- 172.21.15.59:0/3711663408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a10109300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3a000098a0 tx=0x7f3a00009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f93671cae6017d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.140+0000 7f3a0dffb700 1 -- 172.21.15.59:0/3711663408 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a00004160 con 0x7f3a10108ee0 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.140+0000 7f3a0dffb700 1 -- 172.21.15.59:0/3711663408 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a00004300 con 0x7f3a10108ee0 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.140+0000 7f3a0dffb700 1 -- 172.21.15.59:0/3711663408 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a00011a20 con 0x7f3a10108ee0 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.141+0000 7f3a1597a700 1 -- 172.21.15.59:0/3711663408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 msgr2=0x7f3a10109300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.141+0000 7f3a1597a700 1 --2- 172.21.15.59:0/3711663408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a10109300 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f3a000098a0 tx=0x7f3a00009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:50.933 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.141+0000 7f3a1597a700 1 -- 172.21.15.59:0/3711663408 shutdown_connections 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.141+0000 7f3a1597a700 1 --2- 172.21.15.59:0/3711663408 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a10109300 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.141+0000 7f3a1597a700 1 -- 172.21.15.59:0/3711663408 >> 172.21.15.59:0/3711663408 conn(0x7f3a101044a0 msgr2=0x7f3a101068c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.141+0000 7f3a1597a700 1 -- 172.21.15.59:0/3711663408 shutdown_connections 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.141+0000 7f3a1597a700 1 -- 172.21.15.59:0/3711663408 wait complete. 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.142+0000 7f3a1597a700 1 Processor -- start 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.142+0000 7f3a1597a700 1 -- start start 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.142+0000 7f3a1597a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a1019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.142+0000 7f3a1597a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3a1010d3e0 con 0x7f3a10108ee0 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.142+0000 7f3a0effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a1019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.142+0000 7f3a0effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a1019ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45018/0 (socket says 172.21.15.59:45018) 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.143+0000 7f3a0effd700 1 -- 172.21.15.59:0/480086074 learned_addr learned my addr 172.21.15.59:0/480086074 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.143+0000 7f3a0effd700 1 -- 172.21.15.59:0/480086074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3a1019d380 con 0x7f3a10108ee0 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.143+0000 7f3a0effd700 1 --2- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a1019ce40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a00004660 tx=0x7f3a00011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:50.934 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.143+0000 7f3a14978700 1 -- 172.21.15.59:0/480086074 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a00003aa0 con 0x7f3a10108ee0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.143+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3a1019d610 con 0x7f3a10108ee0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.143+0000 7f3a14978700 1 -- 172.21.15.59:0/480086074 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3a0001b8f0 con 0x7f3a10108ee0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.143+0000 7f3a14978700 1 -- 172.21.15.59:0/480086074 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3a00023900 con 0x7f3a10108ee0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.143+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3a1019dff0 con 0x7f3a10108ee0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.144+0000 7f3a14978700 1 -- 172.21.15.59:0/480086074 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7f3a00023bc0 con 0x7f3a10108ee0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.145+0000 7f3a14978700 1 --2- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f39f803e530 0x7f39f80409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.145+0000 7f3a14978700 1 -- 172.21.15.59:0/480086074 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f3a00053eb0 con 0x7f3a10108ee0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.145+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3a10064b00 con 0x7f3a10108ee0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.145+0000 7f3a0e7fc700 1 --2- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f39f803e530 0x7f39f80409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.145+0000 7f3a0e7fc700 1 --2- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f39f803e530 0x7f39f80409f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a04006fd0 tx=0x7f3a04006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.150+0000 7f3a14978700 1 -- 172.21.15.59:0/480086074 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3a0001f030 con 0x7f3a10108ee0 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.409+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7f3a10105600 con 0x7f39f803e530 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.414+0000 7f3a14978700 1 -- 172.21.15.59:0/480086074 <== mgr.14120 v2:172.21.15.59:6800/95297313 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f3a10105600 con 0x7f39f803e530 2024-05-08T05:17:50.935 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f39f803e530 msgr2=0x7f39f80409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 --2- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f39f803e530 0x7f39f80409f0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f3a04006fd0 tx=0x7f3a04006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 msgr2=0x7f3a1019ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 --2- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a1019ce40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3a00004660 tx=0x7f3a00011aa0 comp rx=0 tx=0).stop 2024-05-08T05:17:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 shutdown_connections 2024-05-08T05:17:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 --2- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f39f803e530 0x7f39f80409f0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 --2- 172.21.15.59:0/480086074 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3a10108ee0 0x7f3a1019ce40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 >> 172.21.15.59:0/480086074 conn(0x7f3a101044a0 msgr2=0x7f3a10104ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 shutdown_connections 2024-05-08T05:17:50.936 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:50.417+0000 7f3a1597a700 1 -- 172.21.15.59:0/480086074 wait complete. 2024-05-08T05:17:51.052 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:50 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:51.053 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:50 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:52.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:51 smithi059 ceph-mon[30304]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:17:52.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:51 smithi059 ceph-mon[30304]: Saving service mgr spec with placement count:2 2024-05-08T05:17:52.006 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:51 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:52.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.041+0000 7fd414e64700 1 Processor -- start 2024-05-08T05:17:52.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.041+0000 7fd414e64700 1 -- start start 2024-05-08T05:17:52.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.041+0000 7fd414e64700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd410109770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:52.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.041+0000 7fd414e64700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd410109d40 con 0x7fd410109350 2024-05-08T05:17:52.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.042+0000 7fd40e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd410109770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:52.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.042+0000 7fd40e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd410109770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45032/0 (socket says 172.21.15.59:45032) 2024-05-08T05:17:52.850 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.042+0000 7fd40e59c700 1 -- 172.21.15.59:0/565350369 learned_addr learned my addr 172.21.15.59:0/565350369 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.042+0000 7fd40e59c700 1 -- 172.21.15.59:0/565350369 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd410108000 con 0x7fd410109350 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.042+0000 7fd40e59c700 1 --2- 172.21.15.59:0/565350369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd410109770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4000098a0 tx=0x7fd400009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4c3fc4b8ac55d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.042+0000 7fd40d59a700 1 -- 172.21.15.59:0/565350369 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd400004160 con 0x7fd410109350 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.042+0000 7fd40d59a700 1 -- 172.21.15.59:0/565350369 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd400004300 con 0x7fd410109350 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.042+0000 7fd40d59a700 1 -- 172.21.15.59:0/565350369 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd400011a20 con 0x7fd410109350 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.043+0000 7fd414e64700 1 -- 172.21.15.59:0/565350369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 msgr2=0x7fd410109770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.043+0000 7fd414e64700 1 --2- 172.21.15.59:0/565350369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd410109770 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd4000098a0 tx=0x7fd400009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.043+0000 7fd414e64700 1 -- 172.21.15.59:0/565350369 shutdown_connections 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.043+0000 7fd414e64700 1 --2- 172.21.15.59:0/565350369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd410109770 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.043+0000 7fd414e64700 1 -- 172.21.15.59:0/565350369 >> 172.21.15.59:0/565350369 conn(0x7fd410102a30 msgr2=0x7fd410104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.043+0000 7fd414e64700 1 -- 172.21.15.59:0/565350369 shutdown_connections 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.043+0000 7fd414e64700 1 -- 172.21.15.59:0/565350369 wait complete. 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.044+0000 7fd414e64700 1 Processor -- start 2024-05-08T05:17:52.851 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.044+0000 7fd414e64700 1 -- start start 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.044+0000 7fd414e64700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd41019cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.044+0000 7fd414e64700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd41010c3c0 con 0x7fd410109350 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.044+0000 7fd40e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd41019cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.044+0000 7fd40e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd41019cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45038/0 (socket says 172.21.15.59:45038) 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.044+0000 7fd40e59c700 1 -- 172.21.15.59:0/1566188250 learned_addr learned my addr 172.21.15.59:0/1566188250 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.044+0000 7fd40e59c700 1 -- 172.21.15.59:0/1566188250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd41019d420 con 0x7fd410109350 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.045+0000 7fd40e59c700 1 --2- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd41019cee0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd400000c00 tx=0x7fd400011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.045+0000 7fd3ff7fe700 1 -- 172.21.15.59:0/1566188250 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd400003aa0 con 0x7fd410109350 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.045+0000 7fd3ff7fe700 1 -- 172.21.15.59:0/1566188250 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd40001b8f0 con 0x7fd410109350 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.045+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd41019d6b0 con 0x7fd410109350 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.045+0000 7fd3ff7fe700 1 -- 172.21.15.59:0/1566188250 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd400023890 con 0x7fd410109350 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.045+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd41019e1d0 con 0x7fd410109350 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.046+0000 7fd3ff7fe700 1 -- 172.21.15.59:0/1566188250 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fd40002c3f0 con 0x7fd410109350 2024-05-08T05:17:52.852 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.046+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd410064b00 con 0x7fd410109350 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.046+0000 7fd3ff7fe700 1 --2- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fd3f803e530 0x7fd3f80409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.046+0000 7fd3ff7fe700 1 -- 172.21.15.59:0/1566188250 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd400053ea0 con 0x7fd410109350 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.046+0000 7fd40dd9b700 1 --2- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fd3f803e530 0x7fd3f80409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.047+0000 7fd40dd9b700 1 --2- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fd3f803e530 0x7fd3f80409f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd404006fd0 tx=0x7fd404006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.052+0000 7fd3ff7fe700 1 -- 172.21.15.59:0/1566188250 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd40001f030 con 0x7fd410109350 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.273+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7fd41019ee00 con 0x7fd410109350 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.276+0000 7fd3ff7fe700 1 -- 172.21.15.59:0/1566188250 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7fd400017e90 con 0x7fd410109350 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fd3f803e530 msgr2=0x7fd3f80409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 --2- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fd3f803e530 0x7fd3f80409f0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7fd404006fd0 tx=0x7fd404006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 msgr2=0x7fd41019cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:52.853 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 --2- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd41019cee0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fd400000c00 tx=0x7fd400011aa0 comp rx=0 tx=0).stop 2024-05-08T05:17:52.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 shutdown_connections 2024-05-08T05:17:52.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 --2- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fd3f803e530 0x7fd3f80409f0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:52.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 --2- 172.21.15.59:0/1566188250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd410109350 0x7fd41019cee0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:52.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 >> 172.21.15.59:0/1566188250 conn(0x7fd410102a30 msgr2=0x7fd4101038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:52.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 shutdown_connections 2024-05-08T05:17:52.854 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:52.279+0000 7fd414e64700 1 -- 172.21.15.59:0/1566188250 wait complete. 2024-05-08T05:17:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:53 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1566188250' entity='client.admin' 2024-05-08T05:17:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:53 smithi059 ceph-mon[30304]: from='mgr.14120 172.21.15.59:0/886324536' entity='mgr.x' 2024-05-08T05:17:54.752 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.421+0000 7fca61bb0700 1 Processor -- start 2024-05-08T05:17:54.752 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.421+0000 7fca61bb0700 1 -- start start 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.421+0000 7fca61bb0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c10bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.421+0000 7fca61bb0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c07ce50 con 0x7fca5c109900 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.422+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c10bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.422+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c10bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45042/0 (socket says 172.21.15.59:45042) 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.422+0000 7fca5b7fe700 1 -- 172.21.15.59:0/462876902 learned_addr learned my addr 172.21.15.59:0/462876902 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.422+0000 7fca5b7fe700 1 -- 172.21.15.59:0/462876902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c10c230 con 0x7fca5c109900 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.422+0000 7fca5b7fe700 1 --2- 172.21.15.59:0/462876902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c10bcf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0098a0 tx=0x7fca4c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a98edee29f5d1016 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.422+0000 7fca5a7fc700 1 -- 172.21.15.59:0/462876902 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca4c004160 con 0x7fca5c109900 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.423+0000 7fca5a7fc700 1 -- 172.21.15.59:0/462876902 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c004300 con 0x7fca5c109900 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.423+0000 7fca5a7fc700 1 -- 172.21.15.59:0/462876902 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca4c011a20 con 0x7fca5c109900 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.423+0000 7fca61bb0700 1 -- 172.21.15.59:0/462876902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 msgr2=0x7fca5c10bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.423+0000 7fca61bb0700 1 --2- 172.21.15.59:0/462876902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c10bcf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0098a0 tx=0x7fca4c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.423+0000 7fca61bb0700 1 -- 172.21.15.59:0/462876902 shutdown_connections 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.423+0000 7fca61bb0700 1 --2- 172.21.15.59:0/462876902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c10bcf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:54.753 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.423+0000 7fca61bb0700 1 -- 172.21.15.59:0/462876902 >> 172.21.15.59:0/462876902 conn(0x7fca5c1032b0 msgr2=0x7fca5c105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.423+0000 7fca61bb0700 1 -- 172.21.15.59:0/462876902 shutdown_connections 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.423+0000 7fca61bb0700 1 -- 172.21.15.59:0/462876902 wait complete. 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.424+0000 7fca61bb0700 1 Processor -- start 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.424+0000 7fca61bb0700 1 -- start start 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.424+0000 7fca61bb0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c19ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.424+0000 7fca61bb0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca5c10d230 con 0x7fca5c109900 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c19ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca5b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c19ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45054/0 (socket says 172.21.15.59:45054) 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca5b7fe700 1 -- 172.21.15.59:0/1968691681 learned_addr learned my addr 172.21.15.59:0/1968691681 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca5b7fe700 1 -- 172.21.15.59:0/1968691681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca5c19d340 con 0x7fca5c109900 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca5b7fe700 1 --2- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c19ce00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0116e0 tx=0x7fca4c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca58ff9700 1 -- 172.21.15.59:0/1968691681 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca4c003b20 con 0x7fca5c109900 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca58ff9700 1 -- 172.21.15.59:0/1968691681 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca4c01b8f0 con 0x7fca5c109900 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca58ff9700 1 -- 172.21.15.59:0/1968691681 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fca4c0238c0 con 0x7fca5c109900 2024-05-08T05:17:54.754 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca5c19d5d0 con 0x7fca5c109900 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.425+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca5c19e040 con 0x7fca5c109900 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.426+0000 7fca58ff9700 1 -- 172.21.15.59:0/1968691681 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7fca4c023a60 con 0x7fca5c109900 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.426+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca5c064b00 con 0x7fca5c109900 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.427+0000 7fca58ff9700 1 --2- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fca4403e530 0x7fca440409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.427+0000 7fca58ff9700 1 -- 172.21.15.59:0/1968691681 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fca4c053f50 con 0x7fca5c109900 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.427+0000 7fca5affd700 1 --2- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fca4403e530 0x7fca440409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.427+0000 7fca5affd700 1 --2- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fca4403e530 0x7fca440409f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fca50006fd0 tx=0x7fca50006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.432+0000 7fca58ff9700 1 -- 172.21.15.59:0/1968691681 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca4c01f030 con 0x7fca5c109900 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.654+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7fca5c19e5a0 con 0x7fca5c109900 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.657+0000 7fca58ff9700 1 -- 172.21.15.59:0/1968691681 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7fca4c00b3e0 con 0x7fca5c109900 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fca4403e530 msgr2=0x7fca440409f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 --2- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fca4403e530 0x7fca440409f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fca50006fd0 tx=0x7fca50006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 msgr2=0x7fca5c19ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 --2- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c19ce00 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fca4c0116e0 tx=0x7fca4c011aa0 comp rx=0 tx=0).stop 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 shutdown_connections 2024-05-08T05:17:54.755 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 --2- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fca4403e530 0x7fca440409f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:54.756 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 --2- 172.21.15.59:0/1968691681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca5c109900 0x7fca5c19ce00 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:54.756 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 >> 172.21.15.59:0/1968691681 conn(0x7fca5c1032b0 msgr2=0x7fca5c103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:54.756 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 shutdown_connections 2024-05-08T05:17:54.756 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:53.660+0000 7fca61bb0700 1 -- 172.21.15.59:0/1968691681 wait complete. 2024-05-08T05:17:54.756 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling the dashboard module... 2024-05-08T05:17:54.974 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:54 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1968691681' entity='client.admin' 2024-05-08T05:17:56.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:56 smithi059 ceph-mon[30304]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:17:56.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:56 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/51404636' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-05-08T05:17:56.769 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: ignoring --setuser ceph since I am not root 2024-05-08T05:17:56.769 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: ignoring --setgroup ceph since I am not root 2024-05-08T05:17:56.769 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:56.614+0000 7f4d0bbfd200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:17:57.027 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:56.808+0000 7f4d0bbfd200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:17:57.027 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:56.914+0000 7f4d0bbfd200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:17:57.035 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.192+0000 7ff2c2fc2700 1 Processor -- start 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.192+0000 7ff2c2fc2700 1 -- start start 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.192+0000 7ff2c2fc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc10bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.192+0000 7ff2c2fc2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc07d6a0 con 0x7ff2bc109950 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.192+0000 7ff2c0d5e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc10bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.192+0000 7ff2c0d5e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc10bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45062/0 (socket says 172.21.15.59:45062) 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.192+0000 7ff2c0d5e700 1 -- 172.21.15.59:0/3782641992 learned_addr learned my addr 172.21.15.59:0/3782641992 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2c0d5e700 1 -- 172.21.15.59:0/3782641992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc10c280 con 0x7ff2bc109950 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2c0d5e700 1 --2- 172.21.15.59:0/3782641992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc10bd40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00098a0 tx=0x7ff2b0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f437e21fb0e189e5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2bb7fe700 1 -- 172.21.15.59:0/3782641992 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0004160 con 0x7ff2bc109950 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2bb7fe700 1 -- 172.21.15.59:0/3782641992 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b0004300 con 0x7ff2bc109950 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2bb7fe700 1 -- 172.21.15.59:0/3782641992 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0011a20 con 0x7ff2bc109950 2024-05-08T05:17:57.036 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/3782641992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 msgr2=0x7ff2bc10bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2c2fc2700 1 --2- 172.21.15.59:0/3782641992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc10bd40 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff2b00098a0 tx=0x7ff2b0009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/3782641992 shutdown_connections 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2c2fc2700 1 --2- 172.21.15.59:0/3782641992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc10bd40 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.193+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/3782641992 >> 172.21.15.59:0/3782641992 conn(0x7ff2bc103320 msgr2=0x7ff2bc105740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.194+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/3782641992 shutdown_connections 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.194+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/3782641992 wait complete. 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.194+0000 7ff2c2fc2700 1 Processor -- start 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.194+0000 7ff2c2fc2700 1 -- start start 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.194+0000 7ff2c2fc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc1093b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.194+0000 7ff2c2fc2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2bc10d220 con 0x7ff2bc109950 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2c0d5e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc1093b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2c0d5e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc1093b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45068/0 (socket says 172.21.15.59:45068) 2024-05-08T05:17:57.037 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2c0d5e700 1 -- 172.21.15.59:0/51404636 learned_addr learned my addr 172.21.15.59:0/51404636 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2c0d5e700 1 -- 172.21.15.59:0/51404636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2bc107900 con 0x7ff2bc109950 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2c0d5e700 1 --2- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc1093b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0004660 tx=0x7ff2b0011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2b9ffb700 1 -- 172.21.15.59:0/51404636 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0003aa0 con 0x7ff2bc109950 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2b9ffb700 1 -- 172.21.15.59:0/51404636 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2b001b8f0 con 0x7ff2bc109950 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2bc107b90 con 0x7ff2bc109950 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2b9ffb700 1 -- 172.21.15.59:0/51404636 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff2b0023900 con 0x7ff2bc109950 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.195+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2bc108070 con 0x7ff2bc109950 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.196+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2bc064b00 con 0x7ff2bc109950 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.197+0000 7ff2b9ffb700 1 -- 172.21.15.59:0/51404636 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 8) ==== 50222+0+0 (secure 0 0 0) 0x7ff2b0023b70 con 0x7ff2bc109950 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.197+0000 7ff2b9ffb700 1 --2- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7ff2ac03e580 0x7ff2ac040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.197+0000 7ff2b9ffb700 1 -- 172.21.15.59:0/51404636 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7ff2b0053d70 con 0x7ff2bc109950 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.197+0000 7ff2bbfff700 1 --2- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7ff2ac03e580 0x7ff2ac040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:57.038 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.197+0000 7ff2bbfff700 1 --2- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7ff2ac03e580 0x7ff2ac040a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8006fd0 tx=0x7ff2a8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.202+0000 7ff2b9ffb700 1 -- 172.21.15.59:0/51404636 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2b001f030 con 0x7ff2bc109950 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:55.481+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7ff2bc1085b0 con 0x7ff2bc109950 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.423+0000 7ff2b9ffb700 1 -- 172.21.15.59:0/51404636 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7ff2b0016260 con 0x7ff2bc109950 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.425+0000 7ff2b9ffb700 1 -- 172.21.15.59:0/51404636 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7ff2b001acb0 con 0x7ff2bc109950 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.427+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7ff2ac03e580 msgr2=0x7ff2ac040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.427+0000 7ff2c2fc2700 1 --2- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7ff2ac03e580 0x7ff2ac040a40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7ff2a8006fd0 tx=0x7ff2a8006e40 comp rx=0 tx=0).stop 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.427+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 msgr2=0x7ff2bc1093b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.427+0000 7ff2c2fc2700 1 --2- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc1093b0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff2b0004660 tx=0x7ff2b0011aa0 comp rx=0 tx=0).stop 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.427+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 shutdown_connections 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.427+0000 7ff2c2fc2700 1 --2- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7ff2ac03e580 0x7ff2ac040a40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.427+0000 7ff2c2fc2700 1 --2- 172.21.15.59:0/51404636 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2bc109950 0x7ff2bc1093b0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.428+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 >> 172.21.15.59:0/51404636 conn(0x7ff2bc103320 msgr2=0x7ff2bc103d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.428+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 shutdown_connections 2024-05-08T05:17:57.039 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:56.428+0000 7ff2c2fc2700 1 -- 172.21.15.59:0/51404636 wait complete. 2024-05-08T05:17:57.280 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:57 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:57.253+0000 7f4d0bbfd200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:17:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:57 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/51404636' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-05-08T05:17:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:57 smithi059 ceph-mon[30304]: mgrmap e9: x(active, since 21s) 2024-05-08T05:17:57.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:57 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:57.346+0000 7f4d0bbfd200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "available": true, 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.469+0000 7fcceaca5700 1 Processor -- start 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.469+0000 7fcceaca5700 1 -- start start 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.469+0000 7fcceaca5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce4109780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.469+0000 7fcceaca5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4109d50 con 0x7fcce4109360 2024-05-08T05:17:58.384 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.469+0000 7fcce8a41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce4109780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.470+0000 7fcce8a41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce4109780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45116/0 (socket says 172.21.15.59:45116) 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.470+0000 7fcce8a41700 1 -- 172.21.15.59:0/763049050 learned_addr learned my addr 172.21.15.59:0/763049050 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.470+0000 7fcce8a41700 1 -- 172.21.15.59:0/763049050 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce4109ed0 con 0x7fcce4109360 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.470+0000 7fcce8a41700 1 --2- 172.21.15.59:0/763049050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce4109780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccd80098a0 tx=0x7fccd8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b715d65119164a6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.470+0000 7fcce37fe700 1 -- 172.21.15.59:0/763049050 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd8004160 con 0x7fcce4109360 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.470+0000 7fcce37fe700 1 -- 172.21.15.59:0/763049050 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd8004300 con 0x7fcce4109360 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.470+0000 7fcce37fe700 1 -- 172.21.15.59:0/763049050 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd8011a20 con 0x7fcce4109360 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.471+0000 7fcceaca5700 1 -- 172.21.15.59:0/763049050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 msgr2=0x7fcce4109780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.471+0000 7fcceaca5700 1 --2- 172.21.15.59:0/763049050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce4109780 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fccd80098a0 tx=0x7fccd8009bb0 comp rx=0 tx=0).stop 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.471+0000 7fcceaca5700 1 -- 172.21.15.59:0/763049050 shutdown_connections 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.471+0000 7fcceaca5700 1 --2- 172.21.15.59:0/763049050 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce4109780 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.471+0000 7fcceaca5700 1 -- 172.21.15.59:0/763049050 >> 172.21.15.59:0/763049050 conn(0x7fcce4102a30 msgr2=0x7fcce4104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.471+0000 7fcceaca5700 1 -- 172.21.15.59:0/763049050 shutdown_connections 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.471+0000 7fcceaca5700 1 -- 172.21.15.59:0/763049050 wait complete. 2024-05-08T05:17:58.385 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.471+0000 7fcceaca5700 1 Processor -- start 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcceaca5700 1 -- start start 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcceaca5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce419ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcceaca5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcce4107d20 con 0x7fcce4109360 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcce8a41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce419ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcce8a41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce419ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45128/0 (socket says 172.21.15.59:45128) 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcce8a41700 1 -- 172.21.15.59:0/1545781929 learned_addr learned my addr 172.21.15.59:0/1545781929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcce8a41700 1 -- 172.21.15.59:0/1545781929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcce419f4a0 con 0x7fcce4109360 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcce8a41700 1 --2- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce419ef60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fccd80119f0 tx=0x7fccd8011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcce1ffb700 1 -- 172.21.15.59:0/1545781929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd80039e0 con 0x7fcce4109360 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcce1ffb700 1 -- 172.21.15.59:0/1545781929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fccd801b8f0 con 0x7fcce4109360 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.472+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcce419f790 con 0x7fcce4109360 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.473+0000 7fcce1ffb700 1 -- 172.21.15.59:0/1545781929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fccd80237a0 con 0x7fcce4109360 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.473+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcce41a0250 con 0x7fcce4109360 2024-05-08T05:17:58.386 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.474+0000 7fcce1ffb700 1 -- 172.21.15.59:0/1545781929 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7fccd8023aa0 con 0x7fcce4109360 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.474+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcce4064b00 con 0x7fcce4109360 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.474+0000 7fcce1ffb700 1 --2- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fccd403e530 0x7fccd40409f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.474+0000 7fcce3fff700 1 -- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fccd403e530 msgr2=0x7fccd40409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/95297313 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.474+0000 7fcce1ffb700 1 -- 172.21.15.59:0/1545781929 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fccd8054080 con 0x7fcce4109360 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.474+0000 7fcce3fff700 1 --2- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fccd403e530 0x7fccd40409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.479+0000 7fcce1ffb700 1 -- 172.21.15.59:0/1545781929 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fccd801f030 con 0x7fcce4109360 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.675+0000 7fcce3fff700 1 -- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fccd403e530 msgr2=0x7fccd40409f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/95297313 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.675+0000 7fcce3fff700 1 --2- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fccd403e530 0x7fccd40409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.788+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fcce4197dc0 con 0x7fcce4109360 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.789+0000 7fcce1ffb700 1 -- 172.21.15.59:0/1545781929 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fccd80094f0 con 0x7fcce4109360 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.793+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fccd403e530 msgr2=0x7fccd40409f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.793+0000 7fcceaca5700 1 --2- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fccd403e530 0x7fccd40409f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.793+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 msgr2=0x7fcce419ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:17:58.387 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.793+0000 7fcceaca5700 1 --2- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce419ef60 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fccd80119f0 tx=0x7fccd8011ad0 comp rx=0 tx=0).stop 2024-05-08T05:17:58.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.793+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 shutdown_connections 2024-05-08T05:17:58.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.793+0000 7fcceaca5700 1 --2- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7fccd403e530 0x7fccd40409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:58.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.793+0000 7fcceaca5700 1 --2- 172.21.15.59:0/1545781929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcce4109360 0x7fcce419ef60 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:17:58.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.793+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 >> 172.21.15.59:0/1545781929 conn(0x7fcce4102a30 msgr2=0x7fcce41954a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:17:58.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.794+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 shutdown_connections 2024-05-08T05:17:58.388 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:57.794+0000 7fcceaca5700 1 -- 172.21.15.59:0/1545781929 wait complete. 2024-05-08T05:17:58.388 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for the mgr to restart... 2024-05-08T05:17:58.388 INFO:teuthology.orchestra.run.smithi059.stdout:Waiting for mgr epoch 9... 2024-05-08T05:17:58.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:17:58 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1545781929' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-05-08T05:17:58.990 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:58 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:58.628+0000 7f4d0bbfd200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:17:58.990 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:58 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:58.811+0000 7f4d0bbfd200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:17:58.991 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:58 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:58.990+0000 7f4d0bbfd200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:17:59.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:59 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:59.295+0000 7f4d0bbfd200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:17:59.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:59 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:59.392+0000 7f4d0bbfd200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:18:00.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:59 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:59.604+0000 7f4d0bbfd200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:18:00.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:17:59 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:17:59.805+0000 7f4d0bbfd200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:18:00.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:00.436+0000 7f4d0bbfd200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:18:01.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:00.667+0000 7f4d0bbfd200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:18:01.402 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:01 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:01.116+0000 7f4d0bbfd200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:18:01.403 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:01 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:01.217+0000 7f4d0bbfd200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:18:01.826 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:01 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:01.612+0000 7f4d0bbfd200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:18:02.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:01 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:01.988+0000 7f4d0bbfd200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:18:02.343 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:02.086+0000 7f4d0bbfd200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:18:02.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:02.343+0000 7f4d0bbfd200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:18:02.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:02.484+0000 7f4d0bbfd200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:18:02.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:02.584+0000 7f4d0bbfd200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:18:03.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:02.926+0000 7f4d0bbfd200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:18:03.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:03 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:03.024+0000 7f4d0bbfd200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:18:03.940 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:03 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:03.657+0000 7f4d0bbfd200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:18:03.941 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:18:03 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:18:03.886+0000 7f4d0bbfd200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:18:04.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: Active manager daemon x restarted 2024-05-08T05:18:04.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: Activating manager daemon x 2024-05-08T05:18:04.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: osdmap e3: 0 total, 0 up, 0 in 2024-05-08T05:18:04.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: mgrmap e10: x(active, starting, since 0.00429288s) 2024-05-08T05:18:04.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:18:04.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:18:04.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:18:04.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:18:04.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:18:04.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: Manager daemon x is now available 2024-05-08T05:18:04.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:18:05.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:18:05.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:18:05.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:04 smithi059 ceph-mon[30304]: mgrmap e11: x(active, since 1.00643s) 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout { 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout "initialized": true 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout } 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.816+0000 7f21af197700 1 Processor -- start 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.816+0000 7f21af197700 1 -- start start 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.816+0000 7f21af197700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a8109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.816+0000 7f21af197700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21a8109790 con 0x7f21a8108e30 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.816+0000 7f21acf33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a8109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.816+0000 7f21acf33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a8109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59626/0 (socket says 172.21.15.59:59626) 2024-05-08T05:18:05.946 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.816+0000 7f21acf33700 1 -- 172.21.15.59:0/384905667 learned_addr learned my addr 172.21.15.59:0/384905667 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.817+0000 7f21acf33700 1 -- 172.21.15.59:0/384905667 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a8109f90 con 0x7f21a8108e30 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.817+0000 7f21acf33700 1 --2- 172.21.15.59:0/384905667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a8109250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f219c0098a0 tx=0x7f219c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c99d1fbaaeaafab4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.817+0000 7f21a77fe700 1 -- 172.21.15.59:0/384905667 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f219c004160 con 0x7f21a8108e30 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.817+0000 7f21a77fe700 1 -- 172.21.15.59:0/384905667 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f219c004300 con 0x7f21a8108e30 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.817+0000 7f21a77fe700 1 -- 172.21.15.59:0/384905667 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f219c011a20 con 0x7f21a8108e30 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.818+0000 7f21af197700 1 -- 172.21.15.59:0/384905667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 msgr2=0x7f21a8109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.818+0000 7f21af197700 1 --2- 172.21.15.59:0/384905667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a8109250 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f219c0098a0 tx=0x7f219c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.818+0000 7f21af197700 1 -- 172.21.15.59:0/384905667 shutdown_connections 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.818+0000 7f21af197700 1 --2- 172.21.15.59:0/384905667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a8109250 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.818+0000 7f21af197700 1 -- 172.21.15.59:0/384905667 >> 172.21.15.59:0/384905667 conn(0x7f21a8104410 msgr2=0x7f21a8106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.818+0000 7f21af197700 1 -- 172.21.15.59:0/384905667 shutdown_connections 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.818+0000 7f21af197700 1 -- 172.21.15.59:0/384905667 wait complete. 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.818+0000 7f21af197700 1 Processor -- start 2024-05-08T05:18:05.947 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.818+0000 7f21af197700 1 -- start start 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21af197700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21af197700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f21a810d3d0 con 0x7f21a8108e30 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21acf33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21acf33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59636/0 (socket says 172.21.15.59:59636) 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21acf33700 1 -- 172.21.15.59:0/4106739585 learned_addr learned my addr 172.21.15.59:0/4106739585 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21acf33700 1 -- 172.21.15.59:0/4106739585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21a819d150 con 0x7f21a8108e30 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21acf33700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a819cc10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f219c000c00 tx=0x7f219c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f219c003aa0 con 0x7f21a8108e30 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21af197700 1 -- 172.21.15.59:0/4106739585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21a819d3e0 con 0x7f21a8108e30 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.819+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f219c01b8f0 con 0x7f21a8108e30 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.820+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f219c023870 con 0x7f21a8108e30 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.820+0000 7f21af197700 1 -- 172.21.15.59:0/4106739585 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21a81a00d0 con 0x7f21a8108e30 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.821+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 9) ==== 50235+0+0 (secure 0 0 0) 0x7f219c023b70 con 0x7f21a8108e30 2024-05-08T05:18:05.948 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.821+0000 7f21a5ffb700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 0x7f2198040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.821+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 --> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f2198041150 con 0x7f219803e580 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.821+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f219c053e50 con 0x7f21a8108e30 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.821+0000 7f21a7fff700 1 -- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 msgr2=0x7f2198040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/95297313 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:58.821+0000 7f21a7fff700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 0x7f2198040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:59.022+0000 7f21a7fff700 1 -- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 msgr2=0x7f2198040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/95297313 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:59.022+0000 7f21a7fff700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 0x7f2198040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:59.423+0000 7f21a7fff700 1 -- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 msgr2=0x7f2198040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/95297313 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:17:59.423+0000 7f21a7fff700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 0x7f2198040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:00.224+0000 7f21a7fff700 1 -- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 msgr2=0x7f2198040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/95297313 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:00.224+0000 7f21a7fff700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 0x7f2198040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:01.825+0000 7f21a7fff700 1 -- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 msgr2=0x7f2198040a40 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/95297313 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:01.825+0000 7f21a7fff700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 0x7f2198040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:03.891+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mgrmap(e 10) ==== 50002+0+0 (secure 0 0 0) 0x7f219c02c3f0 con 0x7f21a8108e30 2024-05-08T05:18:05.949 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:03.891+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 msgr2=0x7f2198040a40 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:03.891+0000 7f21a5ffb700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 0x7f2198040a40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.892+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f219c055be0 con 0x7f21a8108e30 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.892+0000 7f21a5ffb700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2198041fb0 0x7f21980443a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.892+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f219c024e70 con 0x7f2198041fb0 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.893+0000 7f21a7fff700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2198041fb0 0x7f21980443a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.893+0000 7f21a7fff700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2198041fb0 0x7f21980443a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2194003a10 tx=0x7f21940092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.894+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== command_reply(tid 0: 0 ) ==== 8+0+9564 (secure 0 0 0) 0x7f219c024e70 con 0x7f2198041fb0 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.902+0000 7f21af197700 1 -- 172.21.15.59:0/4106739585 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f21a810d550 con 0x7f2198041fb0 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.902+0000 7f21a5ffb700 1 -- 172.21.15.59:0/4106739585 <== mgr.14152 v2:172.21.15.59:6800/546809822 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f21a810d550 con 0x7f2198041fb0 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 -- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2198041fb0 msgr2=0x7f21980443a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2198041fb0 0x7f21980443a0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2194003a10 tx=0x7f21940092b0 comp rx=0 tx=0).stop 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 -- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 msgr2=0x7f21a819cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a819cc10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f219c000c00 tx=0x7f219c011aa0 comp rx=0 tx=0).stop 2024-05-08T05:18:05.950 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 -- 172.21.15.59:0/4106739585 shutdown_connections 2024-05-08T05:18:05.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2198041fb0 0x7f21980443a0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:05.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f21a8108e30 0x7f21a819cc10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:05.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 --2- 172.21.15.59:0/4106739585 >> [v2:172.21.15.59:6800/95297313,v1:172.21.15.59:6801/95297313] conn(0x7f219803e580 0x7f2198040a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:05.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 -- 172.21.15.59:0/4106739585 >> 172.21.15.59:0/4106739585 conn(0x7f21a8104410 msgr2=0x7f21a8104dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:05.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 -- 172.21.15.59:0/4106739585 shutdown_connections 2024-05-08T05:18:05.951 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:04.903+0000 7f21af197700 1 -- 172.21.15.59:0/4106739585 wait complete. 2024-05-08T05:18:05.951 INFO:teuthology.orchestra.run.smithi059.stdout:mgr epoch 9 is available 2024-05-08T05:18:05.951 INFO:teuthology.orchestra.run.smithi059.stdout:Generating a dashboard self-signed certificate... 2024-05-08T05:18:06.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:05 smithi059 ceph-mon[30304]: [08/May/2024:05:18:05] ENGINE Bus STARTING 2024-05-08T05:18:06.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:05 smithi059 ceph-mon[30304]: [08/May/2024:05:18:05] ENGINE Serving on https://172.21.15.59:7150 2024-05-08T05:18:06.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:05 smithi059 ceph-mon[30304]: [08/May/2024:05:18:05] ENGINE Serving on http://172.21.15.59:8765 2024-05-08T05:18:06.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:05 smithi059 ceph-mon[30304]: [08/May/2024:05:18:05] ENGINE Bus STARTED 2024-05-08T05:18:06.247 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.362+0000 7f784d02f700 1 Processor -- start 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.362+0000 7f784d02f700 1 -- start start 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.363+0000 7f784d02f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f7848108170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.363+0000 7f784d02f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7848108740 con 0x7f7848107d50 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.363+0000 7f7846d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f7848108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.363+0000 7f7846d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f7848108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59704/0 (socket says 172.21.15.59:59704) 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.363+0000 7f7846d9d700 1 -- 172.21.15.59:0/1206502868 learned_addr learned my addr 172.21.15.59:0/1206502868 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.363+0000 7f7846d9d700 1 -- 172.21.15.59:0/1206502868 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7848108f50 con 0x7f7848107d50 2024-05-08T05:18:07.251 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.363+0000 7f7846d9d700 1 --2- 172.21.15.59:0/1206502868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f7848108170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78380098a0 tx=0x7f7838009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95f99b1f148a1957 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784659c700 1 -- 172.21.15.59:0/1206502868 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7838004160 con 0x7f7848107d50 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784659c700 1 -- 172.21.15.59:0/1206502868 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7838004300 con 0x7f7848107d50 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784659c700 1 -- 172.21.15.59:0/1206502868 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7838011a20 con 0x7f7848107d50 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784d02f700 1 -- 172.21.15.59:0/1206502868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 msgr2=0x7f7848108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784d02f700 1 --2- 172.21.15.59:0/1206502868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f7848108170 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f78380098a0 tx=0x7f7838009bb0 comp rx=0 tx=0).stop 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784d02f700 1 -- 172.21.15.59:0/1206502868 shutdown_connections 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784d02f700 1 --2- 172.21.15.59:0/1206502868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f7848108170 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784d02f700 1 -- 172.21.15.59:0/1206502868 >> 172.21.15.59:0/1206502868 conn(0x7f78481032b0 msgr2=0x7f7848105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784d02f700 1 -- 172.21.15.59:0/1206502868 shutdown_connections 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.364+0000 7f784d02f700 1 -- 172.21.15.59:0/1206502868 wait complete. 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.365+0000 7f784d02f700 1 Processor -- start 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.365+0000 7f784d02f700 1 -- start start 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.365+0000 7f784d02f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f784819fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.365+0000 7f784d02f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f784810c300 con 0x7f7848107d50 2024-05-08T05:18:07.252 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f7846d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f784819fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f7846d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f784819fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59718/0 (socket says 172.21.15.59:59718) 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f7846d9d700 1 -- 172.21.15.59:0/1822536288 learned_addr learned my addr 172.21.15.59:0/1822536288 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f7846d9d700 1 -- 172.21.15.59:0/1822536288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784807cc10 con 0x7f7848107d50 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f7846d9d700 1 --2- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f784819fa70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78380119f0 tx=0x7f7838011ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f7844d99700 1 -- 172.21.15.59:0/1822536288 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78380039e0 con 0x7f7848107d50 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f7844d99700 1 -- 172.21.15.59:0/1822536288 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f783801b8f0 con 0x7f7848107d50 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784807cea0 con 0x7f7848107d50 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f7844d99700 1 -- 172.21.15.59:0/1822536288 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f78380237a0 con 0x7f7848107d50 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.366+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78481a0200 con 0x7f7848107d50 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.367+0000 7f7844d99700 1 -- 172.21.15.59:0/1822536288 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 11) ==== 50129+0+0 (secure 0 0 0) 0x7f783802c3f0 con 0x7f7848107d50 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.367+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7848064b00 con 0x7f7848107d50 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.368+0000 7f7844d99700 1 --2- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f783403e470 0x7f7834040930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.368+0000 7f7844d99700 1 -- 172.21.15.59:0/1822536288 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7838053d00 con 0x7f7848107d50 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.368+0000 7f783ffff700 1 --2- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f783403e470 0x7f7834040930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.368+0000 7f783ffff700 1 --2- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f783403e470 0x7f7834040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7830006fd0 tx=0x7f7830006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:07.253 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.373+0000 7f7844d99700 1 -- 172.21.15.59:0/1822536288 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f783801f030 con 0x7f7848107d50 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.599+0000 7f7844d99700 1 -- 172.21.15.59:0/1822536288 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f783802cab0 con 0x7f7848107d50 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.602+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f7848104480 con 0x7f783403e470 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.664+0000 7f7844d99700 1 -- 172.21.15.59:0/1822536288 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f7848104480 con 0x7f783403e470 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f783403e470 msgr2=0x7f7834040930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 --2- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f783403e470 0x7f7834040930 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f7830006fd0 tx=0x7f7830006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 msgr2=0x7f784819fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 --2- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f784819fa70 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f78380119f0 tx=0x7f7838011ad0 comp rx=0 tx=0).stop 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 shutdown_connections 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 --2- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f783403e470 0x7f7834040930 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 --2- 172.21.15.59:0/1822536288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7848107d50 0x7f784819fa70 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 >> 172.21.15.59:0/1822536288 conn(0x7f78481032b0 msgr2=0x7f7848103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 shutdown_connections 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:06.667+0000 7f784d02f700 1 -- 172.21.15.59:0/1822536288 wait complete. 2024-05-08T05:18:07.254 INFO:teuthology.orchestra.run.smithi059.stdout:Creating initial admin user... 2024-05-08T05:18:07.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:07 smithi059 ceph-mon[30304]: mgrmap e12: x(active, since 2s) 2024-05-08T05:18:07.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:07 smithi059 ceph-mon[30304]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:18:07.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:07 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:07.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:07 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:07.645 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:07 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:08.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$kfEGvLZaEJc5UayCUIxzi.I4IzAzxyYVELHrdpIMUmXEYr0pXyaVu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1715145488, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-05-08T05:18:08.810 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.678+0000 7ff799212700 1 Processor -- start 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.678+0000 7ff799212700 1 -- start start 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.679+0000 7ff799212700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff794108170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.679+0000 7ff799212700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff794108740 con 0x7ff794107d50 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.679+0000 7ff792d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff794108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.679+0000 7ff792d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff794108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59726/0 (socket says 172.21.15.59:59726) 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.679+0000 7ff792d9d700 1 -- 172.21.15.59:0/3537981881 learned_addr learned my addr 172.21.15.59:0/3537981881 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.679+0000 7ff792d9d700 1 -- 172.21.15.59:0/3537981881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff794108f50 con 0x7ff794107d50 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.679+0000 7ff792d9d700 1 --2- 172.21.15.59:0/3537981881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff794108170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7840098a0 tx=0x7ff784009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c54e0ffafac16a6e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.679+0000 7ff79259c700 1 -- 172.21.15.59:0/3537981881 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff784004160 con 0x7ff794107d50 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.679+0000 7ff79259c700 1 -- 172.21.15.59:0/3537981881 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff784004300 con 0x7ff794107d50 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.680+0000 7ff79259c700 1 -- 172.21.15.59:0/3537981881 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff784011a20 con 0x7ff794107d50 2024-05-08T05:18:08.811 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.680+0000 7ff799212700 1 -- 172.21.15.59:0/3537981881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 msgr2=0x7ff794108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.680+0000 7ff799212700 1 --2- 172.21.15.59:0/3537981881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff794108170 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7ff7840098a0 tx=0x7ff784009bb0 comp rx=0 tx=0).stop 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.680+0000 7ff799212700 1 -- 172.21.15.59:0/3537981881 shutdown_connections 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.680+0000 7ff799212700 1 --2- 172.21.15.59:0/3537981881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff794108170 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.680+0000 7ff799212700 1 -- 172.21.15.59:0/3537981881 >> 172.21.15.59:0/3537981881 conn(0x7ff7941032b0 msgr2=0x7ff794105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.680+0000 7ff799212700 1 -- 172.21.15.59:0/3537981881 shutdown_connections 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.680+0000 7ff799212700 1 -- 172.21.15.59:0/3537981881 wait complete. 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.681+0000 7ff799212700 1 Processor -- start 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.681+0000 7ff799212700 1 -- start start 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.681+0000 7ff799212700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff79419f080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.681+0000 7ff799212700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff79410c300 con 0x7ff794107d50 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff792d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff79419f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff792d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff79419f080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59740/0 (socket says 172.21.15.59:59740) 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff792d9d700 1 -- 172.21.15.59:0/4067850341 learned_addr learned my addr 172.21.15.59:0/4067850341 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:08.812 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff792d9d700 1 -- 172.21.15.59:0/4067850341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff79419f5c0 con 0x7ff794107d50 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff792d9d700 1 --2- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff79419f080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff784000c00 tx=0x7ff784011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff790d99700 1 -- 172.21.15.59:0/4067850341 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff784003aa0 con 0x7ff794107d50 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff790d99700 1 -- 172.21.15.59:0/4067850341 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff78401b8f0 con 0x7ff794107d50 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff790d99700 1 -- 172.21.15.59:0/4067850341 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff784023890 con 0x7ff794107d50 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff79419f850 con 0x7ff794107d50 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.682+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7941a0370 con 0x7ff794107d50 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.683+0000 7ff790d99700 1 -- 172.21.15.59:0/4067850341 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7ff78402c3f0 con 0x7ff794107d50 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.683+0000 7ff790d99700 1 --2- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff78003e580 0x7ff780040a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.684+0000 7ff790d99700 1 -- 172.21.15.59:0/4067850341 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff784054040 con 0x7ff794107d50 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.684+0000 7ff78bfff700 1 --2- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff78003e580 0x7ff780040a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.684+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff794064b00 con 0x7ff794107d50 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.684+0000 7ff78bfff700 1 --2- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff78003e580 0x7ff780040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff77c006fd0 tx=0x7ff77c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:08.813 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.689+0000 7ff790d99700 1 -- 172.21.15.59:0/4067850341 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff78401f030 con 0x7ff794107d50 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:07.924+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7ff79410c150 con 0x7ff78003e580 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.171+0000 7ff790d99700 1 -- 172.21.15.59:0/4067850341 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7ff79410c150 con 0x7ff78003e580 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff78003e580 msgr2=0x7ff780040a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 --2- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff78003e580 0x7ff780040a40 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7ff77c006fd0 tx=0x7ff77c006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 msgr2=0x7ff79419f080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 --2- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff79419f080 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff784000c00 tx=0x7ff784011aa0 comp rx=0 tx=0).stop 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 shutdown_connections 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 --2- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff78003e580 0x7ff780040a40 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 --2- 172.21.15.59:0/4067850341 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff794107d50 0x7ff79419f080 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 >> 172.21.15.59:0/4067850341 conn(0x7ff7941032b0 msgr2=0x7ff794104140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 shutdown_connections 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:08.173+0000 7ff799212700 1 -- 172.21.15.59:0/4067850341 wait complete. 2024-05-08T05:18:08.814 INFO:teuthology.orchestra.run.smithi059.stdout:Fetching dashboard port number... 2024-05-08T05:18:09.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:09 smithi059 ceph-mon[30304]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:18:09.399 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:09 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:10.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stdout 8443 2024-05-08T05:18:10.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.251+0000 7f2f4a199700 1 Processor -- start 2024-05-08T05:18:10.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.251+0000 7f2f4a199700 1 -- start start 2024-05-08T05:18:10.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.251+0000 7f2f4a199700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4407db70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:10.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.251+0000 7f2f4a199700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4407e140 con 0x7f2f4407f710 2024-05-08T05:18:10.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.251+0000 7f2f437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4407db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:10.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.251+0000 7f2f437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4407db70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40764/0 (socket says 172.21.15.59:40764) 2024-05-08T05:18:10.195 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.251+0000 7f2f437fe700 1 -- 172.21.15.59:0/245154499 learned_addr learned my addr 172.21.15.59:0/245154499 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.252+0000 7f2f437fe700 1 -- 172.21.15.59:0/245154499 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4407e2c0 con 0x7f2f4407f710 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.252+0000 7f2f437fe700 1 --2- 172.21.15.59:0/245154499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4407db70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f340098a0 tx=0x7f2f34009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efbc25e493db14 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.252+0000 7f2f427fc700 1 -- 172.21.15.59:0/245154499 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f34004160 con 0x7f2f4407f710 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.252+0000 7f2f427fc700 1 -- 172.21.15.59:0/245154499 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f34004300 con 0x7f2f4407f710 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.252+0000 7f2f427fc700 1 -- 172.21.15.59:0/245154499 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f34011a20 con 0x7f2f4407f710 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.253+0000 7f2f4a199700 1 -- 172.21.15.59:0/245154499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 msgr2=0x7f2f4407db70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.253+0000 7f2f4a199700 1 --2- 172.21.15.59:0/245154499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4407db70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f2f340098a0 tx=0x7f2f34009bb0 comp rx=0 tx=0).stop 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.253+0000 7f2f4a199700 1 -- 172.21.15.59:0/245154499 shutdown_connections 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.253+0000 7f2f4a199700 1 --2- 172.21.15.59:0/245154499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4407db70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.253+0000 7f2f4a199700 1 -- 172.21.15.59:0/245154499 >> 172.21.15.59:0/245154499 conn(0x7f2f44104410 msgr2=0x7f2f44106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.253+0000 7f2f4a199700 1 -- 172.21.15.59:0/245154499 shutdown_connections 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.253+0000 7f2f4a199700 1 -- 172.21.15.59:0/245154499 wait complete. 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.253+0000 7f2f4a199700 1 Processor -- start 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f4a199700 1 -- start start 2024-05-08T05:18:10.196 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f4a199700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4419ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f4a199700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2f4410f430 con 0x7f2f4407f710 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4419ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f437fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4419ef40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40768/0 (socket says 172.21.15.59:40768) 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f437fe700 1 -- 172.21.15.59:0/3959362911 learned_addr learned my addr 172.21.15.59:0/3959362911 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f437fe700 1 -- 172.21.15.59:0/3959362911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f4419f480 con 0x7f2f4407f710 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f437fe700 1 --2- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4419ef40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009520 tx=0x7f2f34011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f40ff9700 1 -- 172.21.15.59:0/3959362911 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f340039e0 con 0x7f2f4407f710 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.254+0000 7f2f40ff9700 1 -- 172.21.15.59:0/3959362911 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f3401b8f0 con 0x7f2f4407f710 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.255+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f4419f710 con 0x7f2f4407f710 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.255+0000 7f2f40ff9700 1 -- 172.21.15.59:0/3959362911 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f34019770 con 0x7f2f4407f710 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.255+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f441a0230 con 0x7f2f4407f710 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.256+0000 7f2f40ff9700 1 -- 172.21.15.59:0/3959362911 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 12) ==== 50235+0+0 (secure 0 0 0) 0x7f2f3401b440 con 0x7f2f4407f710 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.256+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f44064b00 con 0x7f2f4407f710 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.256+0000 7f2f40ff9700 1 --2- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f2c03e5d0 0x7f2f2c040a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:10.197 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.256+0000 7f2f40ff9700 1 -- 172.21.15.59:0/3959362911 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f2f34053f40 con 0x7f2f4407f710 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.256+0000 7f2f42ffd700 1 --2- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f2c03e5d0 0x7f2f2c040a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.256+0000 7f2f42ffd700 1 --2- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f2c03e5d0 0x7f2f2c040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.261+0000 7f2f40ff9700 1 -- 172.21.15.59:0/3959362911 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f3401f030 con 0x7f2f4407f710 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.478+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7f2f441a06e0 con 0x7f2f4407f710 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.478+0000 7f2f40ff9700 1 -- 172.21.15.59:0/3959362911 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7f2f34017e90 con 0x7f2f4407f710 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f2c03e5d0 msgr2=0x7f2f2c040a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 --2- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f2c03e5d0 0x7f2f2c040a90 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f2f38006fd0 tx=0x7f2f38006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 msgr2=0x7f2f4419ef40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 --2- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4419ef40 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f2f34009520 tx=0x7f2f34011aa0 comp rx=0 tx=0).stop 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 shutdown_connections 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 --2- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f2c03e5d0 0x7f2f2c040a90 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 --2- 172.21.15.59:0/3959362911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2f4407f710 0x7f2f4419ef40 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 >> 172.21.15.59:0/3959362911 conn(0x7f2f44104410 msgr2=0x7f2f44104e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:10.198 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 shutdown_connections 2024-05-08T05:18:10.199 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:09.481+0000 7f2f4a199700 1 -- 172.21.15.59:0/3959362911 wait complete. 2024-05-08T05:18:10.204 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-05-08T05:18:10.204 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout disabled 2024-05-08T05:18:10.213 INFO:teuthology.orchestra.run.smithi059.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-05-08T05:18:10.213 INFO:teuthology.orchestra.run.smithi059.stdout:systemctl: stdout inactive 2024-05-08T05:18:10.213 INFO:teuthology.orchestra.run.smithi059.stdout:firewalld.service is not enabled 2024-05-08T05:18:10.214 INFO:teuthology.orchestra.run.smithi059.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-05-08T05:18:10.215 INFO:teuthology.orchestra.run.smithi059.stdout:Ceph Dashboard is now available at: 2024-05-08T05:18:10.216 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:10.216 INFO:teuthology.orchestra.run.smithi059.stdout: URL: https://smithi059.front.sepia.ceph.com:8443/ 2024-05-08T05:18:10.216 INFO:teuthology.orchestra.run.smithi059.stdout: User: admin 2024-05-08T05:18:10.216 INFO:teuthology.orchestra.run.smithi059.stdout: Password: 0wcaog1f5b 2024-05-08T05:18:10.216 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:10.216 INFO:teuthology.orchestra.run.smithi059.stdout:Saving cluster configuration to /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config directory 2024-05-08T05:18:10.217 INFO:teuthology.orchestra.run.smithi059.stdout:Enabling autotune for osd_memory_target 2024-05-08T05:18:10.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:10 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3959362911' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-05-08T05:18:11.393 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:11 smithi059 ceph-mon[30304]: mgrmap e13: x(active, since 6s) 2024-05-08T05:18:11.400 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.617+0000 7f502d3e7700 1 Processor -- start 2024-05-08T05:18:11.400 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.617+0000 7f502d3e7700 1 -- start start 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.617+0000 7f502d3e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f5028108170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.617+0000 7f502d3e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5028108740 con 0x7f5028107d50 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.617+0000 7f5026ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f5028108170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.617+0000 7f5026ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f5028108170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40774/0 (socket says 172.21.15.59:40774) 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.617+0000 7f5026ffd700 1 -- 172.21.15.59:0/2866557896 learned_addr learned my addr 172.21.15.59:0/2866557896 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.618+0000 7f5026ffd700 1 -- 172.21.15.59:0/2866557896 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5028108f50 con 0x7f5028107d50 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.618+0000 7f5026ffd700 1 --2- 172.21.15.59:0/2866557896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f5028108170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f501800bb40 tx=0x7f501800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bae9d3c6aef8d971 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.618+0000 7f5025ffb700 1 -- 172.21.15.59:0/2866557896 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f501800dbe0 con 0x7f5028107d50 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.618+0000 7f5025ffb700 1 -- 172.21.15.59:0/2866557896 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50180044e0 con 0x7f5028107d50 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.618+0000 7f5025ffb700 1 -- 172.21.15.59:0/2866557896 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5018010470 con 0x7f5028107d50 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.619+0000 7f502d3e7700 1 -- 172.21.15.59:0/2866557896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 msgr2=0x7f5028108170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.619+0000 7f502d3e7700 1 --2- 172.21.15.59:0/2866557896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f5028108170 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f501800bb40 tx=0x7f501800be50 comp rx=0 tx=0).stop 2024-05-08T05:18:11.401 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.619+0000 7f502d3e7700 1 -- 172.21.15.59:0/2866557896 shutdown_connections 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.619+0000 7f502d3e7700 1 --2- 172.21.15.59:0/2866557896 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f5028108170 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.619+0000 7f502d3e7700 1 -- 172.21.15.59:0/2866557896 >> 172.21.15.59:0/2866557896 conn(0x7f50281032b0 msgr2=0x7f5028105730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.619+0000 7f502d3e7700 1 -- 172.21.15.59:0/2866557896 shutdown_connections 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.619+0000 7f502d3e7700 1 -- 172.21.15.59:0/2866557896 wait complete. 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.620+0000 7f502d3e7700 1 Processor -- start 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.620+0000 7f502d3e7700 1 -- start start 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.620+0000 7f502d3e7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f502819f020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.620+0000 7f502d3e7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f502807bfb0 con 0x7f5028107d50 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.620+0000 7f5026ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f502819f020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.620+0000 7f5026ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f502819f020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40780/0 (socket says 172.21.15.59:40780) 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.620+0000 7f5026ffd700 1 -- 172.21.15.59:0/41493776 learned_addr learned my addr 172.21.15.59:0/41493776 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.620+0000 7f5026ffd700 1 -- 172.21.15.59:0/41493776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f502819f560 con 0x7f5028107d50 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.621+0000 7f5026ffd700 1 --2- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f502819f020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5018003c30 tx=0x7f5018003ff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:11.402 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.621+0000 7f501ffff700 1 -- 172.21.15.59:0/41493776 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5018010720 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.621+0000 7f501ffff700 1 -- 172.21.15.59:0/41493776 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501801d070 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.621+0000 7f501ffff700 1 -- 172.21.15.59:0/41493776 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5018021aa0 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.621+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f502819f7f0 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.621+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50281a0310 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.622+0000 7f501ffff700 1 -- 172.21.15.59:0/41493776 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f501801f030 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.622+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5028064b00 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.622+0000 7f501ffff700 1 --2- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f501403e620 0x7f5014040ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.622+0000 7f501ffff700 1 -- 172.21.15.59:0/41493776 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f5018053e40 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.622+0000 7f50267fc700 1 --2- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f501403e620 0x7f5014040ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.623+0000 7f50267fc700 1 --2- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f501403e620 0x7f5014040ae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5010006fd0 tx=0x7f5010006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.627+0000 7f501ffff700 1 -- 172.21.15.59:0/41493776 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5018015070 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.839+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7f502806bcb0 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.840+0000 7f501ffff700 1 -- 172.21.15.59:0/41493776 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7f50180183a0 con 0x7f5028107d50 2024-05-08T05:18:11.403 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f501403e620 msgr2=0x7f5014040ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:11.404 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 --2- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f501403e620 0x7f5014040ae0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f5010006fd0 tx=0x7f5010006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:11.404 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 msgr2=0x7f502819f020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:11.404 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 --2- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f502819f020 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f5018003c30 tx=0x7f5018003ff0 comp rx=0 tx=0).stop 2024-05-08T05:18:11.404 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 shutdown_connections 2024-05-08T05:18:11.404 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 --2- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f501403e620 0x7f5014040ae0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:11.404 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 --2- 172.21.15.59:0/41493776 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5028107d50 0x7f502819f020 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:11.404 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 >> 172.21.15.59:0/41493776 conn(0x7f50281032b0 msgr2=0x7f5028195580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:11.404 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 shutdown_connections 2024-05-08T05:18:11.404 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:10.842+0000 7f502d3e7700 1 -- 172.21.15.59:0/41493776 wait complete. 2024-05-08T05:18:12.708 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.824+0000 7fd23eb92700 1 Processor -- start 2024-05-08T05:18:12.708 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.824+0000 7fd23eb92700 1 -- start start 2024-05-08T05:18:12.708 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.824+0000 7fd23eb92700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd238109250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:12.708 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.824+0000 7fd23eb92700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd238109790 con 0x7fd238108e30 2024-05-08T05:18:12.708 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.824+0000 7fd23c92e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd238109250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:12.708 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.824+0000 7fd23c92e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd238109250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40786/0 (socket says 172.21.15.59:40786) 2024-05-08T05:18:12.708 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.824+0000 7fd23c92e700 1 -- 172.21.15.59:0/2242188109 learned_addr learned my addr 172.21.15.59:0/2242188109 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:12.708 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.825+0000 7fd23c92e700 1 -- 172.21.15.59:0/2242188109 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd238109f90 con 0x7fd238108e30 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.825+0000 7fd23c92e700 1 --2- 172.21.15.59:0/2242188109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd238109250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0098a0 tx=0x7fd22c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d7696ba336b6e79 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.825+0000 7fd2377fe700 1 -- 172.21.15.59:0/2242188109 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd22c004160 con 0x7fd238108e30 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.825+0000 7fd2377fe700 1 -- 172.21.15.59:0/2242188109 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd22c004300 con 0x7fd238108e30 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.825+0000 7fd2377fe700 1 -- 172.21.15.59:0/2242188109 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd22c011a20 con 0x7fd238108e30 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.826+0000 7fd23eb92700 1 -- 172.21.15.59:0/2242188109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 msgr2=0x7fd238109250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.826+0000 7fd23eb92700 1 --2- 172.21.15.59:0/2242188109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd238109250 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0098a0 tx=0x7fd22c009bb0 comp rx=0 tx=0).stop 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.826+0000 7fd23eb92700 1 -- 172.21.15.59:0/2242188109 shutdown_connections 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.826+0000 7fd23eb92700 1 --2- 172.21.15.59:0/2242188109 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd238109250 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.826+0000 7fd23eb92700 1 -- 172.21.15.59:0/2242188109 >> 172.21.15.59:0/2242188109 conn(0x7fd238104410 msgr2=0x7fd238106830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.826+0000 7fd23eb92700 1 -- 172.21.15.59:0/2242188109 shutdown_connections 2024-05-08T05:18:12.709 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.826+0000 7fd23eb92700 1 -- 172.21.15.59:0/2242188109 wait complete. 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.826+0000 7fd23eb92700 1 Processor -- start 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.826+0000 7fd23eb92700 1 -- start start 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd23eb92700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd23819cce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd23eb92700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd23810d3d0 con 0x7fd238108e30 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd23c92e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd23819cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd23c92e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd23819cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40800/0 (socket says 172.21.15.59:40800) 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd23c92e700 1 -- 172.21.15.59:0/4223593669 learned_addr learned my addr 172.21.15.59:0/4223593669 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd23c92e700 1 -- 172.21.15.59:0/4223593669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd23819d220 con 0x7fd238108e30 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd23c92e700 1 --2- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd23819cce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd22c000c00 tx=0x7fd22c011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd235ffb700 1 -- 172.21.15.59:0/4223593669 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd22c003aa0 con 0x7fd238108e30 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd23819d4b0 con 0x7fd238108e30 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd235ffb700 1 -- 172.21.15.59:0/4223593669 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd22c01b8f0 con 0x7fd238108e30 2024-05-08T05:18:12.710 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.827+0000 7fd235ffb700 1 -- 172.21.15.59:0/4223593669 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd22c023870 con 0x7fd238108e30 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.828+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd23819dfd0 con 0x7fd238108e30 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.828+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd23810d550 con 0x7fd238108e30 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.828+0000 7fd235ffb700 1 -- 172.21.15.59:0/4223593669 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fd22c023b70 con 0x7fd238108e30 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.829+0000 7fd235ffb700 1 --2- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd22803e620 0x7fd228040ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.829+0000 7fd235ffb700 1 -- 172.21.15.59:0/4223593669 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd22c053fe0 con 0x7fd238108e30 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.829+0000 7fd237fff700 1 --2- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd22803e620 0x7fd228040ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.829+0000 7fd237fff700 1 --2- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd22803e620 0x7fd228040ae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd224006fd0 tx=0x7fd224006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:11.834+0000 7fd235ffb700 1 -- 172.21.15.59:0/4223593669 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd22c01f030 con 0x7fd238108e30 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.168+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7fd238064b00 con 0x7fd238108e30 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.171+0000 7fd235ffb700 1 -- 172.21.15.59:0/4223593669 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v28)=0 set mgr/dashboard/cluster/status v28) ==== 153+0+0 (secure 0 0 0) 0x7fd238064b00 con 0x7fd238108e30 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd22803e620 msgr2=0x7fd228040ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 --2- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd22803e620 0x7fd228040ae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fd224006fd0 tx=0x7fd224006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:12.711 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 msgr2=0x7fd23819cce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 --2- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd23819cce0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd22c000c00 tx=0x7fd22c011aa0 comp rx=0 tx=0).stop 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 shutdown_connections 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 --2- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd22803e620 0x7fd228040ae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 --2- 172.21.15.59:0/4223593669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd238108e30 0x7fd23819cce0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 >> 172.21.15.59:0/4223593669 conn(0x7fd238104410 msgr2=0x7fd238104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 shutdown_connections 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr 2024-05-08T05:18:12.173+0000 7fd23eb92700 1 -- 172.21.15.59:0/4223593669 wait complete. 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:12.712 INFO:teuthology.orchestra.run.smithi059.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout:Or, if you are only running a single cluster on this host: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout:Please consider enabling telemetry to help improve Ceph: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: ceph telemetry on 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout:For more information see: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:12.713 INFO:teuthology.orchestra.run.smithi059.stdout:Bootstrap complete. 2024-05-08T05:18:12.748 INFO:tasks.cephadm:Fetching config... 2024-05-08T05:18:12.749 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:18:12.749 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-05-08T05:18:12.769 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-05-08T05:18:12.769 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:18:12.769 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-05-08T05:18:12.823 INFO:tasks.cephadm:Fetching mon keyring... 2024-05-08T05:18:12.823 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:18:12.823 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/keyring of=/dev/stdout 2024-05-08T05:18:12.889 INFO:tasks.cephadm:Fetching pub ssh key... 2024-05-08T05:18:12.889 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:18:12.889 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-05-08T05:18:12.945 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-05-08T05:18:12.946 DEBUG:teuthology.orchestra.run.smithi059:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/LY86J0TNyKyLtTYInAC8Sz4DMgi1qJDt7LzAXVoDYvBOGdTPXDeXmnE/HlHjvKXYdJP06z1+0MOBi9uvsrv5bZKN2E9SCiHW0s+cjJEqXTpGJotrpR/iQFyAoWXSjUFgrd9h3WBDlwzXETsKFAkPVR3ZuP3YR+PS8Zxe4wfFrkypkw4JxfVnnwERS5EKpR+LizRpJF40s3iM+0vKcvv+oaGJ7keE4uER2zPSPOGwfRwmuJy6ropjt2uZl8y5Zq5HKCGETDk6DOFMLJP1xANSUYCcqiHUyU8vJX9U05FyWosecHewxzQcuq6u7FIyztMwXKVHRjZ9WyZ8p3RHklcNk2NZpMxzT1IxaApEhjPYk/MInCHxcjZpxRvb5trko8LJW2MbUC2SzFnhk8pMwp4+GoLuuE+6YF9rLfCjLQOKT/AcDq0tFkM+mxA0SWk+IogaYpzcG+5bdCd6LT8fkAKqYAQIjMrtTshbVNGYI9kgzbta5tiLgDJsddw/APnYwlM= ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:18:13.036 INFO:teuthology.orchestra.run.smithi059.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/LY86J0TNyKyLtTYInAC8Sz4DMgi1qJDt7LzAXVoDYvBOGdTPXDeXmnE/HlHjvKXYdJP06z1+0MOBi9uvsrv5bZKN2E9SCiHW0s+cjJEqXTpGJotrpR/iQFyAoWXSjUFgrd9h3WBDlwzXETsKFAkPVR3ZuP3YR+PS8Zxe4wfFrkypkw4JxfVnnwERS5EKpR+LizRpJF40s3iM+0vKcvv+oaGJ7keE4uER2zPSPOGwfRwmuJy6ropjt2uZl8y5Zq5HKCGETDk6DOFMLJP1xANSUYCcqiHUyU8vJX9U05FyWosecHewxzQcuq6u7FIyztMwXKVHRjZ9WyZ8p3RHklcNk2NZpMxzT1IxaApEhjPYk/MInCHxcjZpxRvb5trko8LJW2MbUC2SzFnhk8pMwp4+GoLuuE+6YF9rLfCjLQOKT/AcDq0tFkM+mxA0SWk+IogaYpzcG+5bdCd6LT8fkAKqYAQIjMrtTshbVNGYI9kgzbta5tiLgDJsddw/APnYwlM= ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:18:13.049 DEBUG:teuthology.orchestra.run.smithi096:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/LY86J0TNyKyLtTYInAC8Sz4DMgi1qJDt7LzAXVoDYvBOGdTPXDeXmnE/HlHjvKXYdJP06z1+0MOBi9uvsrv5bZKN2E9SCiHW0s+cjJEqXTpGJotrpR/iQFyAoWXSjUFgrd9h3WBDlwzXETsKFAkPVR3ZuP3YR+PS8Zxe4wfFrkypkw4JxfVnnwERS5EKpR+LizRpJF40s3iM+0vKcvv+oaGJ7keE4uER2zPSPOGwfRwmuJy6ropjt2uZl8y5Zq5HKCGETDk6DOFMLJP1xANSUYCcqiHUyU8vJX9U05FyWosecHewxzQcuq6u7FIyztMwXKVHRjZ9WyZ8p3RHklcNk2NZpMxzT1IxaApEhjPYk/MInCHxcjZpxRvb5trko8LJW2MbUC2SzFnhk8pMwp4+GoLuuE+6YF9rLfCjLQOKT/AcDq0tFkM+mxA0SWk+IogaYpzcG+5bdCd6LT8fkAKqYAQIjMrtTshbVNGYI9kgzbta5tiLgDJsddw/APnYwlM= ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:18:13.088 INFO:teuthology.orchestra.run.smithi096.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/LY86J0TNyKyLtTYInAC8Sz4DMgi1qJDt7LzAXVoDYvBOGdTPXDeXmnE/HlHjvKXYdJP06z1+0MOBi9uvsrv5bZKN2E9SCiHW0s+cjJEqXTpGJotrpR/iQFyAoWXSjUFgrd9h3WBDlwzXETsKFAkPVR3ZuP3YR+PS8Zxe4wfFrkypkw4JxfVnnwERS5EKpR+LizRpJF40s3iM+0vKcvv+oaGJ7keE4uER2zPSPOGwfRwmuJy6ropjt2uZl8y5Zq5HKCGETDk6DOFMLJP1xANSUYCcqiHUyU8vJX9U05FyWosecHewxzQcuq6u7FIyztMwXKVHRjZ9WyZ8p3RHklcNk2NZpMxzT1IxaApEhjPYk/MInCHxcjZpxRvb5trko8LJW2MbUC2SzFnhk8pMwp4+GoLuuE+6YF9rLfCjLQOKT/AcDq0tFkM+mxA0SWk+IogaYpzcG+5bdCd6LT8fkAKqYAQIjMrtTshbVNGYI9kgzbta5tiLgDJsddw/APnYwlM= ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:18:13.100 DEBUG:teuthology.orchestra.run.smithi143:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/LY86J0TNyKyLtTYInAC8Sz4DMgi1qJDt7LzAXVoDYvBOGdTPXDeXmnE/HlHjvKXYdJP06z1+0MOBi9uvsrv5bZKN2E9SCiHW0s+cjJEqXTpGJotrpR/iQFyAoWXSjUFgrd9h3WBDlwzXETsKFAkPVR3ZuP3YR+PS8Zxe4wfFrkypkw4JxfVnnwERS5EKpR+LizRpJF40s3iM+0vKcvv+oaGJ7keE4uER2zPSPOGwfRwmuJy6ropjt2uZl8y5Zq5HKCGETDk6DOFMLJP1xANSUYCcqiHUyU8vJX9U05FyWosecHewxzQcuq6u7FIyztMwXKVHRjZ9WyZ8p3RHklcNk2NZpMxzT1IxaApEhjPYk/MInCHxcjZpxRvb5trko8LJW2MbUC2SzFnhk8pMwp4+GoLuuE+6YF9rLfCjLQOKT/AcDq0tFkM+mxA0SWk+IogaYpzcG+5bdCd6LT8fkAKqYAQIjMrtTshbVNGYI9kgzbta5tiLgDJsddw/APnYwlM= ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-05-08T05:18:13.140 INFO:teuthology.orchestra.run.smithi143.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC/LY86J0TNyKyLtTYInAC8Sz4DMgi1qJDt7LzAXVoDYvBOGdTPXDeXmnE/HlHjvKXYdJP06z1+0MOBi9uvsrv5bZKN2E9SCiHW0s+cjJEqXTpGJotrpR/iQFyAoWXSjUFgrd9h3WBDlwzXETsKFAkPVR3ZuP3YR+PS8Zxe4wfFrkypkw4JxfVnnwERS5EKpR+LizRpJF40s3iM+0vKcvv+oaGJ7keE4uER2zPSPOGwfRwmuJy6ropjt2uZl8y5Zq5HKCGETDk6DOFMLJP1xANSUYCcqiHUyU8vJX9U05FyWosecHewxzQcuq6u7FIyztMwXKVHRjZ9WyZ8p3RHklcNk2NZpMxzT1IxaApEhjPYk/MInCHxcjZpxRvb5trko8LJW2MbUC2SzFnhk8pMwp4+GoLuuE+6YF9rLfCjLQOKT/AcDq0tFkM+mxA0SWk+IogaYpzcG+5bdCd6LT8fkAKqYAQIjMrtTshbVNGYI9kgzbta5tiLgDJsddw/APnYwlM= ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:18:13.152 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-05-08T05:18:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:13 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/4223593669' entity='client.admin' 2024-05-08T05:18:13.400 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:18:15.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:15 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:15.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:15 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:15 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi059", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:18:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:15 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:15 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:18:15.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:15 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.906+0000 7f3e1329b700 1 -- 172.21.15.59:0/1483427193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 msgr2=0x7f3e0c103340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.906+0000 7f3e1329b700 1 --2- 172.21.15.59:0/1483427193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 0x7f3e0c103340 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3e08009910 tx=0x7f3e08009c20 comp rx=0 tx=0).stop 2024-05-08T05:18:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.906+0000 7f3e1329b700 1 -- 172.21.15.59:0/1483427193 shutdown_connections 2024-05-08T05:18:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.906+0000 7f3e1329b700 1 --2- 172.21.15.59:0/1483427193 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 0x7f3e0c103340 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.906+0000 7f3e1329b700 1 -- 172.21.15.59:0/1483427193 >> 172.21.15.59:0/1483427193 conn(0x7f3e0c0fe4c0 msgr2=0x7f3e0c100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.906+0000 7f3e1329b700 1 -- 172.21.15.59:0/1483427193 shutdown_connections 2024-05-08T05:18:15.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.906+0000 7f3e1329b700 1 -- 172.21.15.59:0/1483427193 wait complete. 2024-05-08T05:18:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.907+0000 7f3e1329b700 1 Processor -- start 2024-05-08T05:18:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.907+0000 7f3e1329b700 1 -- start start 2024-05-08T05:18:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.907+0000 7f3e1329b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 0x7f3e0c077160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.907+0000 7f3e1329b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3e0c108de0 con 0x7f3e0c102f20 2024-05-08T05:18:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.907+0000 7f3e11037700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 0x7f3e0c077160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.907+0000 7f3e11037700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 0x7f3e0c077160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40818/0 (socket says 172.21.15.59:40818) 2024-05-08T05:18:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.907+0000 7f3e11037700 1 -- 172.21.15.59:0/3309624984 learned_addr learned my addr 172.21.15.59:0/3309624984 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.908+0000 7f3e11037700 1 -- 172.21.15.59:0/3309624984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3e0c0776a0 con 0x7f3e0c102f20 2024-05-08T05:18:15.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.908+0000 7f3e11037700 1 --2- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 0x7f3e0c077160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005b40 tx=0x7f3e08004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.908+0000 7f3e027fc700 1 -- 172.21.15.59:0/3309624984 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e08027070 con 0x7f3e0c102f20 2024-05-08T05:18:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.908+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3e0c075890 con 0x7f3e0c102f20 2024-05-08T05:18:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.908+0000 7f3e027fc700 1 -- 172.21.15.59:0/3309624984 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3e0800be10 con 0x7f3e0c102f20 2024-05-08T05:18:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.908+0000 7f3e027fc700 1 -- 172.21.15.59:0/3309624984 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3e08022de0 con 0x7f3e0c102f20 2024-05-08T05:18:15.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.908+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3e0c075dd0 con 0x7f3e0c102f20 2024-05-08T05:18:15.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.909+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3e0c109560 con 0x7f3e0c102f20 2024-05-08T05:18:15.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.909+0000 7f3e027fc700 1 -- 172.21.15.59:0/3309624984 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f3e0802a070 con 0x7f3e0c102f20 2024-05-08T05:18:15.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.909+0000 7f3e027fc700 1 --2- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3df803e510 0x7f3df80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:15.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.909+0000 7f3e027fc700 1 -- 172.21.15.59:0/3309624984 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3e0801e030 con 0x7f3e0c102f20 2024-05-08T05:18:15.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.910+0000 7f3e10836700 1 --2- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3df803e510 0x7f3df80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:15.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.910+0000 7f3e10836700 1 --2- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3df803e510 0x7f3df80409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006fd0 tx=0x7f3dfc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:15.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:15.915+0000 7f3e027fc700 1 -- 172.21.15.59:0/3309624984 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3e08013070 con 0x7f3e0c102f20 2024-05-08T05:18:16.156 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.154+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7f3e0c191860 con 0x7f3e0c102f20 2024-05-08T05:18:16.161 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.159+0000 7f3e027fc700 1 -- 172.21.15.59:0/3309624984 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7f3e08031020 con 0x7f3e0c102f20 2024-05-08T05:18:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3df803e510 msgr2=0x7f3df80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 --2- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3df803e510 0x7f3df80409d0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f3dfc006fd0 tx=0x7f3dfc006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 msgr2=0x7f3e0c077160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 --2- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 0x7f3e0c077160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3e08005b40 tx=0x7f3e08004f80 comp rx=0 tx=0).stop 2024-05-08T05:18:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 shutdown_connections 2024-05-08T05:18:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 --2- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3df803e510 0x7f3df80409d0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 --2- 172.21.15.59:0/3309624984 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3e0c102f20 0x7f3e0c077160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 >> 172.21.15.59:0/3309624984 conn(0x7f3e0c0fe4c0 msgr2=0x7f3e0c0fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:16.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 shutdown_connections 2024-05-08T05:18:16.165 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:16.162+0000 7f3e1329b700 1 -- 172.21.15.59:0/3309624984 wait complete. 2024-05-08T05:18:17.245 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-05-08T05:18:17.245 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-05-08T05:18:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:18:17.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:17 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3309624984' entity='client.admin' 2024-05-08T05:18:17.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:18:17.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.023+0000 7f959399d700 1 -- 172.21.15.59:0/4211296982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 msgr2=0x7f958c102220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.023+0000 7f959399d700 1 --2- 172.21.15.59:0/4211296982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 0x7f958c102220 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f9588009910 tx=0x7f9588009c20 comp rx=0 tx=0).stop 2024-05-08T05:18:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.023+0000 7f959399d700 1 -- 172.21.15.59:0/4211296982 shutdown_connections 2024-05-08T05:18:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.023+0000 7f959399d700 1 --2- 172.21.15.59:0/4211296982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 0x7f958c102220 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.023+0000 7f959399d700 1 -- 172.21.15.59:0/4211296982 >> 172.21.15.59:0/4211296982 conn(0x7f958c0fd3a0 msgr2=0x7f958c0ff7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.023+0000 7f959399d700 1 -- 172.21.15.59:0/4211296982 shutdown_connections 2024-05-08T05:18:19.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.023+0000 7f959399d700 1 -- 172.21.15.59:0/4211296982 wait complete. 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.024+0000 7f959399d700 1 Processor -- start 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.024+0000 7f959399d700 1 -- start start 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.024+0000 7f959399d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 0x7f958c0ff260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.024+0000 7f959399d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f958c107cc0 con 0x7f958c101e00 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.024+0000 7f9591739700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 0x7f958c0ff260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.024+0000 7f9591739700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 0x7f958c0ff260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45564/0 (socket says 172.21.15.59:45564) 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.024+0000 7f9591739700 1 -- 172.21.15.59:0/1452268198 learned_addr learned my addr 172.21.15.59:0/1452268198 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.025+0000 7f9591739700 1 -- 172.21.15.59:0/1452268198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f958c0ff7a0 con 0x7f958c101e00 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.025+0000 7f9591739700 1 --2- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 0x7f958c0ff260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9588005190 tx=0x7f9588004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.025+0000 7f95827fc700 1 -- 172.21.15.59:0/1452268198 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9588027070 con 0x7f958c101e00 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.025+0000 7f95827fc700 1 -- 172.21.15.59:0/1452268198 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9588022470 con 0x7f958c101e00 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.025+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f958c0fd990 con 0x7f958c101e00 2024-05-08T05:18:19.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.025+0000 7f95827fc700 1 -- 172.21.15.59:0/1452268198 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f958800f5f0 con 0x7f958c101e00 2024-05-08T05:18:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.025+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f958c0fded0 con 0x7f958c101e00 2024-05-08T05:18:19.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.026+0000 7f95827fc700 1 -- 172.21.15.59:0/1452268198 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f958802a070 con 0x7f958c101e00 2024-05-08T05:18:19.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.026+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f958c108440 con 0x7f958c101e00 2024-05-08T05:18:19.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.026+0000 7f95827fc700 1 --2- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f957803e570 0x7f9578040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:19.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.026+0000 7f95827fc700 1 -- 172.21.15.59:0/1452268198 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f958801e030 con 0x7f958c101e00 2024-05-08T05:18:19.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.026+0000 7f9590f38700 1 --2- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f957803e570 0x7f9578040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:19.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.027+0000 7f9590f38700 1 --2- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f957803e570 0x7f9578040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f957c006fd0 tx=0x7f957c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:19.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.031+0000 7f95827fc700 1 -- 172.21.15.59:0/1452268198 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9588013070 con 0x7f958c101e00 2024-05-08T05:18:19.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.253+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f958c1919e0 con 0x7f957803e570 2024-05-08T05:18:19.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.257+0000 7f95827fc700 1 -- 172.21.15.59:0/1452268198 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f958c1919e0 con 0x7f957803e570 2024-05-08T05:18:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f957803e570 msgr2=0x7f9578040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 --2- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f957803e570 0x7f9578040a30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f957c006fd0 tx=0x7f957c006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 msgr2=0x7f958c0ff260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 --2- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 0x7f958c0ff260 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f9588005190 tx=0x7f9588004f80 comp rx=0 tx=0).stop 2024-05-08T05:18:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 shutdown_connections 2024-05-08T05:18:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 --2- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f957803e570 0x7f9578040a30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 --2- 172.21.15.59:0/1452268198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f958c101e00 0x7f958c0ff260 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 >> 172.21.15.59:0/1452268198 conn(0x7f958c0fd3a0 msgr2=0x7f958c06ad80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:19.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 shutdown_connections 2024-05-08T05:18:19.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:19.259+0000 7f959399d700 1 -- 172.21.15.59:0/1452268198 wait complete. 2024-05-08T05:18:19.990 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi096 2024-05-08T05:18:19.990 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:18:19.990 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.conf 2024-05-08T05:18:20.008 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:18:20.008 DEBUG:teuthology.orchestra.run.smithi096:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:18:20.064 INFO:tasks.cephadm:Adding host smithi096 to orchestrator... 2024-05-08T05:18:20.065 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch host add smithi096 2024-05-08T05:18:20.327 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:18:20.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:20 smithi059 ceph-mon[30304]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:18:20.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:20.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:18:20.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:18:20.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:18:20.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:20 smithi059 ceph-mon[30304]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-08T05:18:21.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:21 smithi059 ceph-mon[30304]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:18:21.686 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:21 smithi059 ceph-mon[30304]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:18:21.686 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:21 smithi059 ceph-mon[30304]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:18:21.686 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:21.686 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:21.686 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.797+0000 7ff140a45700 1 -- 172.21.15.59:0/1959269116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 msgr2=0x7ff13c104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.797+0000 7ff140a45700 1 --2- 172.21.15.59:0/1959269116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 0x7ff13c104490 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7ff12c009910 tx=0x7ff12c009c20 comp rx=0 tx=0).stop 2024-05-08T05:18:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.797+0000 7ff140a45700 1 -- 172.21.15.59:0/1959269116 shutdown_connections 2024-05-08T05:18:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.797+0000 7ff140a45700 1 --2- 172.21.15.59:0/1959269116 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 0x7ff13c104490 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.797+0000 7ff140a45700 1 -- 172.21.15.59:0/1959269116 >> 172.21.15.59:0/1959269116 conn(0x7ff13c0fd900 msgr2=0x7ff13c0ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.797+0000 7ff140a45700 1 -- 172.21.15.59:0/1959269116 shutdown_connections 2024-05-08T05:18:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.797+0000 7ff140a45700 1 -- 172.21.15.59:0/1959269116 wait complete. 2024-05-08T05:18:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.798+0000 7ff140a45700 1 Processor -- start 2024-05-08T05:18:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.798+0000 7ff140a45700 1 -- start start 2024-05-08T05:18:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.798+0000 7ff140a45700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 0x7ff13c19e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.798+0000 7ff140a45700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff13c10d430 con 0x7ff13c1040b0 2024-05-08T05:18:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.798+0000 7ff13a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 0x7ff13c19e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.798+0000 7ff13a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 0x7ff13c19e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45570/0 (socket says 172.21.15.59:45570) 2024-05-08T05:18:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.798+0000 7ff13a59c700 1 -- 172.21.15.59:0/2374912288 learned_addr learned my addr 172.21.15.59:0/2374912288 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:21.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.798+0000 7ff13a59c700 1 -- 172.21.15.59:0/2374912288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff13c19ebc0 con 0x7ff13c1040b0 2024-05-08T05:18:21.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.799+0000 7ff13a59c700 1 --2- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 0x7ff13c19e680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0056d0 tx=0x7ff12c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:21.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.799+0000 7ff12b7fe700 1 -- 172.21.15.59:0/2374912288 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff12c026070 con 0x7ff13c1040b0 2024-05-08T05:18:21.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.799+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff13c19ee50 con 0x7ff13c1040b0 2024-05-08T05:18:21.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.799+0000 7ff12b7fe700 1 -- 172.21.15.59:0/2374912288 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff12c004370 con 0x7ff13c1040b0 2024-05-08T05:18:21.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.799+0000 7ff12b7fe700 1 -- 172.21.15.59:0/2374912288 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff12c0039b0 con 0x7ff13c1040b0 2024-05-08T05:18:21.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.799+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff13c198800 con 0x7ff13c1040b0 2024-05-08T05:18:21.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.800+0000 7ff12b7fe700 1 -- 172.21.15.59:0/2374912288 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7ff12c029070 con 0x7ff13c1040b0 2024-05-08T05:18:21.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.800+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff13c06bbf0 con 0x7ff13c1040b0 2024-05-08T05:18:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.800+0000 7ff12b7fe700 1 --2- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff12403e510 0x7ff1240409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.800+0000 7ff12b7fe700 1 -- 172.21.15.59:0/2374912288 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7ff12c01e070 con 0x7ff13c1040b0 2024-05-08T05:18:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.800+0000 7ff139d9b700 1 --2- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff12403e510 0x7ff1240409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.801+0000 7ff139d9b700 1 --2- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff12403e510 0x7ff1240409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff130006fd0 tx=0x7ff130006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:21.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:21.805+0000 7ff12b7fe700 1 -- 172.21.15.59:0/2374912288 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff12c013070 con 0x7ff13c1040b0 2024-05-08T05:18:22.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:22.023+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}) -- 0x7ff13c064b00 con 0x7ff12403e510 2024-05-08T05:18:22.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:22 smithi059 ceph-mon[30304]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi096", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:18:23.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:23 smithi059 ceph-mon[30304]: Deploying cephadm binary to smithi096 2024-05-08T05:18:24.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:24 smithi059 ceph-mon[30304]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:26.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.374+0000 7ff12b7fe700 1 -- 172.21.15.59:0/2374912288 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7ff13c064b00 con 0x7ff12403e510 2024-05-08T05:18:26.376 INFO:teuthology.orchestra.run.smithi059.stdout:Added host 'smithi096' with addr '172.21.15.96' 2024-05-08T05:18:26.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.377+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff12403e510 msgr2=0x7ff1240409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:26.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.377+0000 7ff140a45700 1 --2- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff12403e510 0x7ff1240409d0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7ff130006fd0 tx=0x7ff130006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:26.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.377+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 msgr2=0x7ff13c19e680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:26.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.377+0000 7ff140a45700 1 --2- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 0x7ff13c19e680 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff12c0056d0 tx=0x7ff12c0057b0 comp rx=0 tx=0).stop 2024-05-08T05:18:26.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.377+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 shutdown_connections 2024-05-08T05:18:26.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.377+0000 7ff140a45700 1 --2- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff12403e510 0x7ff1240409d0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:26.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.377+0000 7ff140a45700 1 --2- 172.21.15.59:0/2374912288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff13c1040b0 0x7ff13c19e680 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:26.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.377+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 >> 172.21.15.59:0/2374912288 conn(0x7ff13c0fd900 msgr2=0x7ff13c104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:26.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.378+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 shutdown_connections 2024-05-08T05:18:26.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:26.378+0000 7ff140a45700 1 -- 172.21.15.59:0/2374912288 wait complete. 2024-05-08T05:18:26.975 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T05:18:27.227 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:18:27.616 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:27 smithi059 ceph-mon[30304]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:27.616 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:27 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:27.616 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:27 smithi059 ceph-mon[30304]: Added host smithi096 2024-05-08T05:18:27.616 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:27 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:18:27.616 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:27 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:28.638 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:28 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:28.639 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:28 smithi059 ceph-mon[30304]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:28.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.686+0000 7f541c02d700 1 -- 172.21.15.59:0/131130530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 msgr2=0x7f54141034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:28.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.686+0000 7f541c02d700 1 --2- 172.21.15.59:0/131130530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 0x7f54141034f0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f5410009910 tx=0x7f5410009c20 comp rx=0 tx=0).stop 2024-05-08T05:18:28.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.686+0000 7f541c02d700 1 -- 172.21.15.59:0/131130530 shutdown_connections 2024-05-08T05:18:28.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.686+0000 7f541c02d700 1 --2- 172.21.15.59:0/131130530 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 0x7f54141034f0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:28.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.686+0000 7f541c02d700 1 -- 172.21.15.59:0/131130530 >> 172.21.15.59:0/131130530 conn(0x7f54140fe9e0 msgr2=0x7f5414100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:28.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.686+0000 7f541c02d700 1 -- 172.21.15.59:0/131130530 shutdown_connections 2024-05-08T05:18:28.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.686+0000 7f541c02d700 1 -- 172.21.15.59:0/131130530 wait complete. 2024-05-08T05:18:28.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.687+0000 7f541c02d700 1 Processor -- start 2024-05-08T05:18:28.689 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.687+0000 7f541c02d700 1 -- start start 2024-05-08T05:18:28.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.687+0000 7f541c02d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 0x7f54140757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.687+0000 7f541c02d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f541410d390 con 0x7f5414103110 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.687+0000 7f5419dc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 0x7f54140757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.687+0000 7f5419dc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 0x7f54140757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33418/0 (socket says 172.21.15.59:33418) 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.687+0000 7f5419dc9700 1 -- 172.21.15.59:0/3322986739 learned_addr learned my addr 172.21.15.59:0/3322986739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.688+0000 7f5419dc9700 1 -- 172.21.15.59:0/3322986739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5414077690 con 0x7f5414103110 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.688+0000 7f5419dc9700 1 --2- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 0x7f54140757a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5410006010 tx=0x7f5410005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.688+0000 7f5406ffd700 1 -- 172.21.15.59:0/3322986739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5410026070 con 0x7f5414103110 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.688+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5414075d40 con 0x7f5414103110 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.688+0000 7f5406ffd700 1 -- 172.21.15.59:0/3322986739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541000bc00 con 0x7f5414103110 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.688+0000 7f5406ffd700 1 -- 172.21.15.59:0/3322986739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5410003910 con 0x7f5414103110 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.688+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5414076200 con 0x7f5414103110 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.689+0000 7f5406ffd700 1 -- 172.21.15.59:0/3322986739 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f5410029070 con 0x7f5414103110 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.689+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54141045f0 con 0x7f5414103110 2024-05-08T05:18:28.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.689+0000 7f5406ffd700 1 --2- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f540003e510 0x7f54000409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:28.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.689+0000 7f5406ffd700 1 -- 172.21.15.59:0/3322986739 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f541001e070 con 0x7f5414103110 2024-05-08T05:18:28.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.689+0000 7f54195c8700 1 --2- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f540003e510 0x7f54000409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:28.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.690+0000 7f54195c8700 1 --2- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f540003e510 0x7f54000409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5408006fd0 tx=0x7f5408006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:28.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.695+0000 7f5406ffd700 1 -- 172.21.15.59:0/3322986739 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5410013070 con 0x7f5414103110 2024-05-08T05:18:28.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.913+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f5414076cd0 con 0x7f540003e510 2024-05-08T05:18:28.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.914+0000 7f5406ffd700 1 -- 172.21.15.59:0/3322986739 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f5414076cd0 con 0x7f540003e510 2024-05-08T05:18:28.916 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:28.916 INFO:teuthology.orchestra.run.smithi059.stdout:[{"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}] 2024-05-08T05:18:28.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f540003e510 msgr2=0x7f54000409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:28.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 --2- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f540003e510 0x7f54000409d0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5408006fd0 tx=0x7f5408006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:28.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 msgr2=0x7f54140757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:28.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 --2- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 0x7f54140757a0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f5410006010 tx=0x7f5410005780 comp rx=0 tx=0).stop 2024-05-08T05:18:28.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 shutdown_connections 2024-05-08T05:18:28.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 --2- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f540003e510 0x7f54000409d0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:28.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 --2- 172.21.15.59:0/3322986739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5414103110 0x7f54140757a0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:28.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 >> 172.21.15.59:0/3322986739 conn(0x7f54140fe9e0 msgr2=0x7f5414104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:28.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 shutdown_connections 2024-05-08T05:18:28.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:28.918+0000 7f541c02d700 1 -- 172.21.15.59:0/3322986739 wait complete. 2024-05-08T05:18:29.515 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi143 2024-05-08T05:18:29.515 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:18:29.516 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/etc/ceph/ceph.conf 2024-05-08T05:18:29.534 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:18:29.534 DEBUG:teuthology.orchestra.run.smithi143:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:18:29.591 INFO:tasks.cephadm:Adding host smithi143 to orchestrator... 2024-05-08T05:18:29.592 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch host add smithi143 2024-05-08T05:18:29.843 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:18:30.295 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:29 smithi059 ceph-mon[30304]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:18:30.295 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:31.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.228+0000 7fbc2b02d700 1 -- 172.21.15.59:0/666515856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 msgr2=0x7fbc24106920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.228+0000 7fbc2b02d700 1 --2- 172.21.15.59:0/666515856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 0x7fbc24106920 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fbc1400aa20 tx=0x7fbc1400ad30 comp rx=0 tx=0).stop 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.228+0000 7fbc2b02d700 1 -- 172.21.15.59:0/666515856 shutdown_connections 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.228+0000 7fbc2b02d700 1 --2- 172.21.15.59:0/666515856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 0x7fbc24106920 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.228+0000 7fbc2b02d700 1 -- 172.21.15.59:0/666515856 >> 172.21.15.59:0/666515856 conn(0x7fbc240793d0 msgr2=0x7fbc240797e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.228+0000 7fbc2b02d700 1 -- 172.21.15.59:0/666515856 shutdown_connections 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.228+0000 7fbc2b02d700 1 -- 172.21.15.59:0/666515856 wait complete. 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.229+0000 7fbc2b02d700 1 Processor -- start 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.229+0000 7fbc2b02d700 1 -- start start 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.229+0000 7fbc2b02d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 0x7fbc241031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.229+0000 7fbc2b02d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbc2410b1d0 con 0x7fbc24106540 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.229+0000 7fbc28dc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 0x7fbc241031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.229+0000 7fbc28dc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 0x7fbc241031d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33422/0 (socket says 172.21.15.59:33422) 2024-05-08T05:18:31.230 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.229+0000 7fbc28dc9700 1 -- 172.21.15.59:0/2883966948 learned_addr learned my addr 172.21.15.59:0/2883966948 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:31.231 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.229+0000 7fbc28dc9700 1 -- 172.21.15.59:0/2883966948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbc24105070 con 0x7fbc24106540 2024-05-08T05:18:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.229+0000 7fbc28dc9700 1 --2- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 0x7fbc241031d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbc14005b40 tx=0x7fbc14005050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.230+0000 7fbc21ffb700 1 -- 172.21.15.59:0/2883966948 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc14005400 con 0x7fbc24106540 2024-05-08T05:18:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.230+0000 7fbc21ffb700 1 -- 172.21.15.59:0/2883966948 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbc140055a0 con 0x7fbc24106540 2024-05-08T05:18:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.230+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbc24103710 con 0x7fbc24106540 2024-05-08T05:18:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.230+0000 7fbc21ffb700 1 -- 172.21.15.59:0/2883966948 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbc1400e750 con 0x7fbc24106540 2024-05-08T05:18:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.230+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbc24103bd0 con 0x7fbc24106540 2024-05-08T05:18:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.231+0000 7fbc21ffb700 1 -- 172.21.15.59:0/2883966948 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fbc1400e940 con 0x7fbc24106540 2024-05-08T05:18:31.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.231+0000 7fbc21ffb700 1 --2- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbc0c03e570 0x7fbc0c040a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.231+0000 7fbc21ffb700 1 -- 172.21.15.59:0/2883966948 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fbc1401e070 con 0x7fbc24106540 2024-05-08T05:18:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.231+0000 7fbc23fff700 1 --2- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbc0c03e570 0x7fbc0c040a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.231+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbc24102c80 con 0x7fbc24106540 2024-05-08T05:18:31.233 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.231+0000 7fbc23fff700 1 --2- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbc0c03e570 0x7fbc0c040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc18006fd0 tx=0x7fbc18006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:31.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.237+0000 7fbc21ffb700 1 -- 172.21.15.59:0/2883966948 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbc14058070 con 0x7fbc24106540 2024-05-08T05:18:31.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:30 smithi059 ceph-mon[30304]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:31.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:31.457+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}) -- 0x7fbc241046b0 con 0x7fbc0c03e570 2024-05-08T05:18:32.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:32 smithi059 ceph-mon[30304]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi143", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:18:33.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:33.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:33.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:33.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:33.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:33.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi096", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:18:33.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:18:33.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:18:33.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-08T05:18:33.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:18:33.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:33 smithi059 ceph-mon[30304]: Deploying cephadm binary to smithi143 2024-05-08T05:18:34.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:34 smithi059 ceph-mon[30304]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:18:34.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:34 smithi059 ceph-mon[30304]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:18:35.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:35 smithi059 ceph-mon[30304]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:36.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.004+0000 7fbc21ffb700 1 -- 172.21.15.59:0/2883966948 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7fbc241046b0 con 0x7fbc0c03e570 2024-05-08T05:18:36.006 INFO:teuthology.orchestra.run.smithi059.stdout:Added host 'smithi143' with addr '172.21.15.143' 2024-05-08T05:18:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.007+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbc0c03e570 msgr2=0x7fbc0c040a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.007+0000 7fbc2b02d700 1 --2- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbc0c03e570 0x7fbc0c040a30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fbc18006fd0 tx=0x7fbc18006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.007+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 msgr2=0x7fbc241031d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.007+0000 7fbc2b02d700 1 --2- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 0x7fbc241031d0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fbc14005b40 tx=0x7fbc14005050 comp rx=0 tx=0).stop 2024-05-08T05:18:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.007+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 shutdown_connections 2024-05-08T05:18:36.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.007+0000 7fbc2b02d700 1 --2- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbc0c03e570 0x7fbc0c040a30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.007+0000 7fbc2b02d700 1 --2- 172.21.15.59:0/2883966948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbc24106540 0x7fbc241031d0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.007+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 >> 172.21.15.59:0/2883966948 conn(0x7fbc240793d0 msgr2=0x7fbc241005a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.008+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 shutdown_connections 2024-05-08T05:18:36.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:36.008+0000 7fbc2b02d700 1 -- 172.21.15.59:0/2883966948 wait complete. 2024-05-08T05:18:36.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:36.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:36.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:36.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:36.654 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:18:36.689 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch host ls --format=json 2024-05-08T05:18:36.942 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:18:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:37 smithi059 ceph-mon[30304]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:37 smithi059 ceph-mon[30304]: Added host smithi143 2024-05-08T05:18:37.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:37 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:37.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:37 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.402+0000 7f3413cc9700 1 -- 172.21.15.59:0/2510245596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 msgr2=0x7f340c102420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.402+0000 7f3413cc9700 1 --2- 172.21.15.59:0/2510245596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 0x7f340c102420 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3408009910 tx=0x7f3408009c20 comp rx=0 tx=0).stop 2024-05-08T05:18:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.402+0000 7f3413cc9700 1 -- 172.21.15.59:0/2510245596 shutdown_connections 2024-05-08T05:18:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.402+0000 7f3413cc9700 1 --2- 172.21.15.59:0/2510245596 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 0x7f340c102420 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.402+0000 7f3413cc9700 1 -- 172.21.15.59:0/2510245596 >> 172.21.15.59:0/2510245596 conn(0x7f340c0fd8f0 msgr2=0x7f340c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.402+0000 7f3413cc9700 1 -- 172.21.15.59:0/2510245596 shutdown_connections 2024-05-08T05:18:38.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.402+0000 7f3413cc9700 1 -- 172.21.15.59:0/2510245596 wait complete. 2024-05-08T05:18:38.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.403+0000 7f3413cc9700 1 Processor -- start 2024-05-08T05:18:38.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.403+0000 7f3413cc9700 1 -- start start 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.403+0000 7f3413cc9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 0x7f340c19d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.403+0000 7f3413cc9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f340c10c2c0 con 0x7f340c102040 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.403+0000 7f3411a65700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 0x7f340c19d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.403+0000 7f3411a65700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 0x7f340c19d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33440/0 (socket says 172.21.15.59:33440) 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.403+0000 7f3411a65700 1 -- 172.21.15.59:0/3602647699 learned_addr learned my addr 172.21.15.59:0/3602647699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.404+0000 7f3411a65700 1 -- 172.21.15.59:0/3602647699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f340c19dad0 con 0x7f340c102040 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.404+0000 7f3411a65700 1 --2- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 0x7f340c19d590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f34080056d0 tx=0x7f34080057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.404+0000 7f3402ffd700 1 -- 172.21.15.59:0/3602647699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3408028070 con 0x7f340c102040 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.404+0000 7f3402ffd700 1 -- 172.21.15.59:0/3602647699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f340800bba0 con 0x7f340c102040 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.404+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f340c19dd60 con 0x7f340c102040 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.404+0000 7f3402ffd700 1 -- 172.21.15.59:0/3602647699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3408003920 con 0x7f340c102040 2024-05-08T05:18:38.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.404+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f340c197660 con 0x7f340c102040 2024-05-08T05:18:38.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.405+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f340c103520 con 0x7f340c102040 2024-05-08T05:18:38.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.405+0000 7f3402ffd700 1 -- 172.21.15.59:0/3602647699 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f340802b070 con 0x7f340c102040 2024-05-08T05:18:38.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.405+0000 7f3402ffd700 1 --2- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33f803e510 0x7f33f80409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:38.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.405+0000 7f3402ffd700 1 -- 172.21.15.59:0/3602647699 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f3408020070 con 0x7f340c102040 2024-05-08T05:18:38.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.406+0000 7f3411264700 1 --2- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33f803e510 0x7f33f80409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:38.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.406+0000 7f3411264700 1 --2- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33f803e510 0x7f33f80409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f33fc006fd0 tx=0x7f33fc006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:38.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.411+0000 7f3402ffd700 1 -- 172.21.15.59:0/3602647699 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3408013070 con 0x7f340c102040 2024-05-08T05:18:38.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.631+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f340c064b60 con 0x7f33f803e510 2024-05-08T05:18:38.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.633+0000 7f3402ffd700 1 -- 172.21.15.59:0/3602647699 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7f340c064b60 con 0x7f33f803e510 2024-05-08T05:18:38.633 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:18:38.633 INFO:teuthology.orchestra.run.smithi059.stdout:[{"addr": "172.21.15.59", "hostname": "smithi059", "labels": [], "status": ""}, {"addr": "172.21.15.96", "hostname": "smithi096", "labels": [], "status": ""}, {"addr": "172.21.15.143", "hostname": "smithi143", "labels": [], "status": ""}] 2024-05-08T05:18:38.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33f803e510 msgr2=0x7f33f80409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 --2- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33f803e510 0x7f33f80409d0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f33fc006fd0 tx=0x7f33fc006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 msgr2=0x7f340c19d590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 --2- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 0x7f340c19d590 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f34080056d0 tx=0x7f34080057b0 comp rx=0 tx=0).stop 2024-05-08T05:18:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 shutdown_connections 2024-05-08T05:18:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 --2- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33f803e510 0x7f33f80409d0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 --2- 172.21.15.59:0/3602647699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f340c102040 0x7f340c19d590 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 >> 172.21.15.59:0/3602647699 conn(0x7f340c0fd8f0 msgr2=0x7f340c103a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 shutdown_connections 2024-05-08T05:18:38.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:38.635+0000 7f3413cc9700 1 -- 172.21.15.59:0/3602647699 wait complete. 2024-05-08T05:18:39.181 INFO:tasks.cephadm:Setting crush tunables to default 2024-05-08T05:18:39.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd crush tunables default 2024-05-08T05:18:39.432 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:18:39.744 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:39 smithi059 ceph-mon[30304]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:39.744 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:40.732 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:40 smithi059 ceph-mon[30304]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:18:40.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.985+0000 7fac52bb2700 1 -- 172.21.15.59:0/3457726053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 msgr2=0x7fac4c1034f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:40.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.985+0000 7fac52bb2700 1 --2- 172.21.15.59:0/3457726053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 0x7fac4c1034f0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fac40009910 tx=0x7fac40009c20 comp rx=0 tx=0).stop 2024-05-08T05:18:40.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.985+0000 7fac52bb2700 1 -- 172.21.15.59:0/3457726053 shutdown_connections 2024-05-08T05:18:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.985+0000 7fac52bb2700 1 --2- 172.21.15.59:0/3457726053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 0x7fac4c1034f0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.985+0000 7fac52bb2700 1 -- 172.21.15.59:0/3457726053 >> 172.21.15.59:0/3457726053 conn(0x7fac4c0fe9e0 msgr2=0x7fac4c100e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.986+0000 7fac52bb2700 1 -- 172.21.15.59:0/3457726053 shutdown_connections 2024-05-08T05:18:40.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.986+0000 7fac52bb2700 1 -- 172.21.15.59:0/3457726053 wait complete. 2024-05-08T05:18:40.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.986+0000 7fac52bb2700 1 Processor -- start 2024-05-08T05:18:40.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.986+0000 7fac52bb2700 1 -- start start 2024-05-08T05:18:40.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac52bb2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 0x7fac4c19e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:40.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac52bb2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c10d390 con 0x7fac4c103110 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac5094e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 0x7fac4c19e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac5094e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 0x7fac4c19e630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56746/0 (socket says 172.21.15.59:56746) 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac5094e700 1 -- 172.21.15.59:0/2731516902 learned_addr learned my addr 172.21.15.59:0/2731516902 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac5094e700 1 -- 172.21.15.59:0/2731516902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c19eb70 con 0x7fac4c103110 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac5094e700 1 --2- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 0x7fac4c19e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac40006010 tx=0x7fac40005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac49ffb700 1 -- 172.21.15.59:0/2731516902 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac40026070 con 0x7fac4c103110 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4c19ee60 con 0x7fac4c103110 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac49ffb700 1 -- 172.21.15.59:0/2731516902 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4000bba0 con 0x7fac4c103110 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.987+0000 7fac49ffb700 1 -- 172.21.15.59:0/2731516902 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fac40003910 con 0x7fac4c103110 2024-05-08T05:18:40.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.988+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4c1988d0 con 0x7fac4c103110 2024-05-08T05:18:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.988+0000 7fac49ffb700 1 -- 172.21.15.59:0/2731516902 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fac40029070 con 0x7fac4c103110 2024-05-08T05:18:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.989+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac4c1045f0 con 0x7fac4c103110 2024-05-08T05:18:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.989+0000 7fac49ffb700 1 --2- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fac3c03e510 0x7fac3c0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.989+0000 7fac49ffb700 1 -- 172.21.15.59:0/2731516902 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fac4001e070 con 0x7fac4c103110 2024-05-08T05:18:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.989+0000 7fac4bfff700 1 --2- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fac3c03e510 0x7fac3c0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:40.990 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.989+0000 7fac4bfff700 1 --2- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fac3c03e510 0x7fac3c0409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac38006fd0 tx=0x7fac38006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:40.994 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:40.994+0000 7fac49ffb700 1 -- 172.21.15.59:0/2731516902 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fac40013070 con 0x7fac4c103110 2024-05-08T05:18:41.216 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.216+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fac4c0008d0 con 0x7fac4c103110 2024-05-08T05:18:41.428 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.428+0000 7fac49ffb700 1 -- 172.21.15.59:0/2731516902 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fac40030090 con 0x7fac4c103110 2024-05-08T05:18:41.431 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fac3c03e510 msgr2=0x7fac3c0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:41.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 --2- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fac3c03e510 0x7fac3c0409d0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fac38006fd0 tx=0x7fac38006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:41.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 msgr2=0x7fac4c19e630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:41.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 --2- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 0x7fac4c19e630 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fac40006010 tx=0x7fac40005780 comp rx=0 tx=0).stop 2024-05-08T05:18:41.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 shutdown_connections 2024-05-08T05:18:41.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 --2- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fac3c03e510 0x7fac3c0409d0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:41.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 --2- 172.21.15.59:0/2731516902 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac4c103110 0x7fac4c19e630 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:41.432 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 >> 172.21.15.59:0/2731516902 conn(0x7fac4c0fe9e0 msgr2=0x7fac4c104ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:41.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 shutdown_connections 2024-05-08T05:18:41.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:18:41.431+0000 7fac52bb2700 1 -- 172.21.15.59:0/2731516902 wait complete. 2024-05-08T05:18:41.433 INFO:teuthology.orchestra.run.smithi059.stderr:adjusted tunables profile to default 2024-05-08T05:18:41.761 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:41 smithi059 ceph-mon[30304]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:41.762 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:41 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2731516902' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-05-08T05:18:42.109 INFO:tasks.cephadm:Adding mon.a on smithi059 2024-05-08T05:18:42.109 INFO:tasks.cephadm:Adding mon.b on smithi096 2024-05-08T05:18:42.109 INFO:tasks.cephadm:Adding mon.c on smithi143 2024-05-08T05:18:42.109 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch apply mon '3;smithi059:172.21.15.59=a;smithi096:172.21.15.96=b;smithi143:172.21.15.143=c' 2024-05-08T05:18:42.355 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:18:42.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:42 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2731516902' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-05-08T05:18:42.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:42 smithi059 ceph-mon[30304]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:18:42.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:42.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:42.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:42.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:42.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi143", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:18:42.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:18:42.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:18:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.659+0000 7f44d8e60700 1 -- 172.21.15.143:0/1546000213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 msgr2=0x7f44d4105270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.659+0000 7f44d8e60700 1 --2- 172.21.15.143:0/1546000213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 0x7f44d4105270 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f44c8009910 tx=0x7f44c8009c20 comp rx=0 tx=0).stop 2024-05-08T05:18:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.659+0000 7f44d8e60700 1 -- 172.21.15.143:0/1546000213 shutdown_connections 2024-05-08T05:18:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.659+0000 7f44d8e60700 1 --2- 172.21.15.143:0/1546000213 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 0x7f44d4105270 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:43.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.659+0000 7f44d8e60700 1 -- 172.21.15.143:0/1546000213 >> 172.21.15.143:0/1546000213 conn(0x7f44d40fd010 msgr2=0x7f44d40ff430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.659+0000 7f44d8e60700 1 -- 172.21.15.143:0/1546000213 shutdown_connections 2024-05-08T05:18:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.659+0000 7f44d8e60700 1 -- 172.21.15.143:0/1546000213 wait complete. 2024-05-08T05:18:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.660+0000 7f44d8e60700 1 Processor -- start 2024-05-08T05:18:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.660+0000 7f44d8e60700 1 -- start start 2024-05-08T05:18:43.662 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.660+0000 7f44d8e60700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 0x7f44d419e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:43.663 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.660+0000 7f44d8e60700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f44d410d580 con 0x7f44d4104e90 2024-05-08T05:18:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.661+0000 7f44d259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 0x7f44d419e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.661+0000 7f44d259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 0x7f44d419e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.143:59404/0 (socket says 172.21.15.143:59404) 2024-05-08T05:18:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.661+0000 7f44d259c700 1 -- 172.21.15.143:0/3272385114 learned_addr learned my addr 172.21.15.143:0/3272385114 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:18:43.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.661+0000 7f44d259c700 1 -- 172.21.15.143:0/3272385114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44d419ebf0 con 0x7f44d4104e90 2024-05-08T05:18:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.661+0000 7f44d259c700 1 --2- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 0x7f44d419e6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f44c80056d0 tx=0x7f44c80057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.661+0000 7f44c37fe700 1 -- 172.21.15.143:0/3272385114 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c8026070 con 0x7f44d4104e90 2024-05-08T05:18:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.661+0000 7f44c37fe700 1 -- 172.21.15.143:0/3272385114 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44c800bc00 con 0x7f44d4104e90 2024-05-08T05:18:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.661+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44d419ee80 con 0x7f44d4104e90 2024-05-08T05:18:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.661+0000 7f44c37fe700 1 -- 172.21.15.143:0/3272385114 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44c8003920 con 0x7f44d4104e90 2024-05-08T05:18:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.662+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44d4198870 con 0x7f44d4104e90 2024-05-08T05:18:43.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.662+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44d4104710 con 0x7f44d4104e90 2024-05-08T05:18:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.663+0000 7f44c37fe700 1 -- 172.21.15.143:0/3272385114 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f44c8029070 con 0x7f44d4104e90 2024-05-08T05:18:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.663+0000 7f44c37fe700 1 --2- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f44bc03e510 0x7f44bc0409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.663+0000 7f44c37fe700 1 -- 172.21.15.143:0/3272385114 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f44c801e070 con 0x7f44d4104e90 2024-05-08T05:18:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.663+0000 7f44d1d9b700 1 --2- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f44bc03e510 0x7f44bc0409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:43.667 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.664+0000 7f44d1d9b700 1 --2- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f44bc03e510 0x7f44bc0409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f44c4006fd0 tx=0x7f44c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:43.669 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.669+0000 7f44c37fe700 1 -- 172.21.15.143:0/3272385114 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f44c8013070 con 0x7f44d4104e90 2024-05-08T05:18:43.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:43 smithi059 ceph-mon[30304]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-08T05:18:43.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:43 smithi059 ceph-mon[30304]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:18:43.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:43 smithi059 ceph-mon[30304]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:43.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:43 smithi059 ceph-mon[30304]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:18:43.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:43 smithi059 ceph-mon[30304]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:18:43.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:43 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:43.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:43 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:43.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:43 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:43.882 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.881+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi096:172.21.15.96=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}) -- 0x7f44d4199da0 con 0x7f44bc03e510 2024-05-08T05:18:43.889 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.888+0000 7f44c37fe700 1 -- 172.21.15.143:0/3272385114 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f44d4199da0 con 0x7f44bc03e510 2024-05-08T05:18:43.889 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mon update... 2024-05-08T05:18:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f44bc03e510 msgr2=0x7f44bc0409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 --2- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f44bc03e510 0x7f44bc0409d0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f44c4006fd0 tx=0x7f44c4006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 msgr2=0x7f44d419e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 --2- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 0x7f44d419e6b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f44c80056d0 tx=0x7f44c80057b0 comp rx=0 tx=0).stop 2024-05-08T05:18:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 shutdown_connections 2024-05-08T05:18:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 --2- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f44bc03e510 0x7f44bc0409d0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 --2- 172.21.15.143:0/3272385114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44d4104e90 0x7f44d419e6b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:43.891 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 >> 172.21.15.143:0/3272385114 conn(0x7f44d40fd010 msgr2=0x7f44d410aa40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 shutdown_connections 2024-05-08T05:18:43.892 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:43.890+0000 7f44d8e60700 1 -- 172.21.15.143:0/3272385114 wait complete. 2024-05-08T05:18:44.433 DEBUG:teuthology.orchestra.run.smithi096:mon.b> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.b.service 2024-05-08T05:18:44.436 DEBUG:teuthology.orchestra.run.smithi143:mon.c> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.c.service 2024-05-08T05:18:44.438 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:18:44.438 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:18:45.191 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:18:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi059:172.21.15.59=a;smithi096:172.21.15.96=b;smithi143:172.21.15.143=c", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: Saving service mon spec with placement smithi059:172.21.15.59=a;smithi096:172.21.15.96=b;smithi143:172.21.15.143=c;count:3 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:18:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:44 smithi059 ceph-mon[30304]: Deploying daemon mon.c on smithi143 2024-05-08T05:18:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:46 smithi059 ceph-mon[30304]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:48.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.055+0000 7f036129b700 1 -- 172.21.15.143:0/1583877914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 msgr2=0x7f035c1044e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:48.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.055+0000 7f036129b700 1 --2- 172.21.15.143:0/1583877914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 0x7f035c1044e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f035400b0a0 tx=0x7f035400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:18:48.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.055+0000 7f036129b700 1 -- 172.21.15.143:0/1583877914 shutdown_connections 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.055+0000 7f036129b700 1 --2- 172.21.15.143:0/1583877914 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 0x7f035c1044e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.055+0000 7f036129b700 1 -- 172.21.15.143:0/1583877914 >> 172.21.15.143:0/1583877914 conn(0x7f035c0fd8f0 msgr2=0x7f035c0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.055+0000 7f036129b700 1 -- 172.21.15.143:0/1583877914 shutdown_connections 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.055+0000 7f036129b700 1 -- 172.21.15.143:0/1583877914 wait complete. 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.056+0000 7f036129b700 1 Processor -- start 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.056+0000 7f036129b700 1 -- start start 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.056+0000 7f036129b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 0x7f035c19e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.056+0000 7f036129b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f035c10d400 con 0x7f035c104100 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f035affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 0x7f035c19e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f035affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 0x7f035c19e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.143:57396/0 (socket says 172.21.15.143:57396) 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f035affd700 1 -- 172.21.15.143:0/2033865913 learned_addr learned my addr 172.21.15.143:0/2033865913 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f035affd700 1 -- 172.21.15.143:0/2033865913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f035c19eb20 con 0x7f035c104100 2024-05-08T05:18:48.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f035affd700 1 --2- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 0x7f035c19e5e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0354015040 tx=0x7f0354009790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f0343fff700 1 -- 172.21.15.143:0/2033865913 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0354021020 con 0x7f035c104100 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f035c19edb0 con 0x7f035c104100 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f0343fff700 1 -- 172.21.15.143:0/2033865913 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0354007970 con 0x7f035c104100 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f0343fff700 1 -- 172.21.15.143:0/2033865913 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0354017040 con 0x7f035c104100 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.057+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f035c1986b0 con 0x7f035c104100 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.058+0000 7f0343fff700 1 -- 172.21.15.143:0/2033865913 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f0354024020 con 0x7f035c104100 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.058+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f035c06bb10 con 0x7f035c104100 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.059+0000 7f0343fff700 1 --2- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f034403e510 0x7f03440409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.059+0000 7f0343fff700 1 -- 172.21.15.143:0/2033865913 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f035400f050 con 0x7f035c104100 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.059+0000 7f035a7fc700 1 --2- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f034403e510 0x7f03440409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.059+0000 7f035a7fc700 1 --2- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f034403e510 0x7f03440409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0350006fd0 tx=0x7f0350006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:48.095 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.064+0000 7f0343fff700 1 -- 172.21.15.143:0/2033865913 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f035402b020 con 0x7f035c104100 2024-05-08T05:18:48.400 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.399+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f035c199be0 con 0x7f035c104100 2024-05-08T05:18:48.401 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.400+0000 7f0343fff700 1 -- 172.21.15.143:0/2033865913 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f035401c560 con 0x7f035c104100 2024-05-08T05:18:48.401 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:18:48.401 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":1,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","modified":"2024-05-08T05:16:57.521541Z","created":"2024-05-08T05:16:57.521541Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-05-08T05:18:48.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.402+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f034403e510 msgr2=0x7f03440409d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:48.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.402+0000 7f036129b700 1 --2- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f034403e510 0x7f03440409d0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f0350006fd0 tx=0x7f0350006e40 comp rx=0 tx=0).stop 2024-05-08T05:18:48.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.402+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 msgr2=0x7f035c19e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:48.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.402+0000 7f036129b700 1 --2- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 0x7f035c19e5e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0354015040 tx=0x7f0354009790 comp rx=0 tx=0).stop 2024-05-08T05:18:48.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.402+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 shutdown_connections 2024-05-08T05:18:48.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.402+0000 7f036129b700 1 --2- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f034403e510 0x7f03440409d0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:48.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.402+0000 7f036129b700 1 --2- 172.21.15.143:0/2033865913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035c104100 0x7f035c19e5e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:48.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.402+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 >> 172.21.15.143:0/2033865913 conn(0x7f035c0fd8f0 msgr2=0x7f035c104eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:48.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.403+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 shutdown_connections 2024-05-08T05:18:48.404 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:48.403+0000 7f036129b700 1 -- 172.21.15.143:0/2033865913 wait complete. 2024-05-08T05:18:48.404 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 1 2024-05-08T05:18:49.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:48 smithi059 ceph-mon[30304]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:49.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:48 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/2033865913' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:18:49.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:49 smithi143 systemd[1]: Started Ceph mon.c for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:18:50.093 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:18:50.093 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:18:50.338 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:18:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.792+0000 7fe37a406700 1 -- 172.21.15.143:0/686311497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0035c0 msgr2=0x7fe35c005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.792+0000 7fe37a406700 1 --2- 172.21.15.143:0/686311497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0035c0 0x7fe35c005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe374102eb0 tx=0x7fe36800feb0 comp rx=0 tx=0).stop 2024-05-08T05:18:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.792+0000 7fe37a406700 1 -- 172.21.15.143:0/686311497 shutdown_connections 2024-05-08T05:18:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.792+0000 7fe37a406700 1 --2- 172.21.15.143:0/686311497 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0035c0 0x7fe35c005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.792+0000 7fe37a406700 1 -- 172.21.15.143:0/686311497 >> 172.21.15.143:0/686311497 conn(0x7fe3740fea70 msgr2=0x7fe374100e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.792+0000 7fe37a406700 1 -- 172.21.15.143:0/686311497 shutdown_connections 2024-05-08T05:18:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.792+0000 7fe37a406700 1 -- 172.21.15.143:0/686311497 wait complete. 2024-05-08T05:18:54.794 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.793+0000 7fe37a406700 1 Processor -- start 2024-05-08T05:18:54.795 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.793+0000 7fe37a406700 1 -- start start 2024-05-08T05:18:54.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe37a406700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0035c0 0x7fe3741a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:54.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe37a406700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 0x7fe37419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:54.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe37a406700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe37410aed0 con 0x7fe35c0035c0 2024-05-08T05:18:54.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe37a406700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe37410ad50 con 0x7fe3741a0e70 2024-05-08T05:18:54.796 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe3737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 0x7fe37419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe3737fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 0x7fe37419aaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36512/0 (socket says 172.21.15.143:36512) 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe3737fe700 1 -- 172.21.15.143:0/2803818950 learned_addr learned my addr 172.21.15.143:0/2803818950 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe373fff700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0035c0 0x7fe3741a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe3737fe700 1 -- 172.21.15.143:0/2803818950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 msgr2=0x7fe37419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe3737fe700 1 -- 172.21.15.143:0/2803818950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 msgr2=0x7fe37419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe3737fe700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 0x7fe37419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe3737fe700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 0x7fe37419aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe373fff700 1 -- 172.21.15.143:0/2803818950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 msgr2=0x7fe37419aaf0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe373fff700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 0x7fe37419aaf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe373fff700 1 -- 172.21.15.143:0/2803818950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3741a14b0 con 0x7fe35c0035c0 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.794+0000 7fe373fff700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0035c0 0x7fe3741a0930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe368010010 tx=0x7fe3680050d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.795+0000 7fe3717fa700 1 -- 172.21.15.143:0/2803818950 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe368004aa0 con 0x7fe35c0035c0 2024-05-08T05:18:54.797 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.795+0000 7fe3717fa700 1 -- 172.21.15.143:0/2803818950 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe368004030 con 0x7fe35c0035c0 2024-05-08T05:18:54.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.795+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe37419b030 con 0x7fe35c0035c0 2024-05-08T05:18:54.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.795+0000 7fe3717fa700 1 -- 172.21.15.143:0/2803818950 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7fe368011bd0 con 0x7fe35c0035c0 2024-05-08T05:18:54.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.795+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe37419b5c0 con 0x7fe35c0035c0 2024-05-08T05:18:54.798 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.796+0000 7fe3717fa700 1 -- 172.21.15.143:0/2803818950 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7fe368023070 con 0x7fe35c0035c0 2024-05-08T05:18:54.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.796+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3741046c0 con 0x7fe35c0035c0 2024-05-08T05:18:54.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.796+0000 7fe3717fa700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe36003e300 0x7fe3600407c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:18:54.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.796+0000 7fe3717fa700 1 -- 172.21.15.143:0/2803818950 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fe368020050 con 0x7fe35c0035c0 2024-05-08T05:18:54.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.796+0000 7fe3737fe700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe36003e300 0x7fe3600407c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:18:54.799 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.797+0000 7fe3737fe700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe36003e300 0x7fe3600407c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe364009180 tx=0x7fe364008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:18:54.803 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:54.802+0000 7fe3717fa700 1 -- 172.21.15.143:0/2803818950 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe368028020 con 0x7fe35c0035c0 2024-05-08T05:18:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:18:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: mon.a calling monitor election 2024-05-08T05:18:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: mon.c calling monitor election 2024-05-08T05:18:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: monmap epoch 2 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: last_changed 2024-05-08T05:18:49.531446+0000 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: created 2024-05-08T05:16:57.521541+0000 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: min_mon_release 19 (squid) 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: election_strategy: 1 2024-05-08T05:18:54.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: fsmap 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: mgrmap e13: x(active, since 50s) 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: overall HEALTH_OK 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:18:54.996 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:18:55.035 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: mon.a calling monitor election 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: mon.c calling monitor election 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: monmap epoch 2 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:18:55.036 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: last_changed 2024-05-08T05:18:49.531446+0000 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: created 2024-05-08T05:16:57.521541+0000 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: min_mon_release 19 (squid) 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: election_strategy: 1 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: fsmap 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: mgrmap e13: x(active, since 50s) 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: overall HEALTH_OK 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:18:55.037 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:18:55.093 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.093+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7fe37419bfe0 con 0x7fe35c0035c0 2024-05-08T05:18:55.094 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.093+0000 7fe3717fa700 1 -- 172.21.15.143:0/2803818950 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7fe3680103d0 con 0x7fe35c0035c0 2024-05-08T05:18:55.094 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:18:55.094 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":2,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","modified":"2024-05-08T05:18:49.531446Z","created":"2024-05-08T05:16:57.521541Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-05-08T05:18:55.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.095+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe36003e300 msgr2=0x7fe3600407c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:55.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.095+0000 7fe37a406700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe36003e300 0x7fe3600407c0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe364009180 tx=0x7fe364008040 comp rx=0 tx=0).stop 2024-05-08T05:18:55.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.095+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0035c0 msgr2=0x7fe3741a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:18:55.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.095+0000 7fe37a406700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0035c0 0x7fe3741a0930 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7fe368010010 tx=0x7fe3680050d0 comp rx=0 tx=0).stop 2024-05-08T05:18:55.096 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.096+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 shutdown_connections 2024-05-08T05:18:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.096+0000 7fe37a406700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe36003e300 0x7fe3600407c0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.096+0000 7fe37a406700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe35c0035c0 0x7fe3741a0930 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.096+0000 7fe37a406700 1 --2- 172.21.15.143:0/2803818950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3741a0e70 0x7fe37419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:18:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.096+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 >> 172.21.15.143:0/2803818950 conn(0x7fe3740fea70 msgr2=0x7fe3740ffd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:18:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.096+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 shutdown_connections 2024-05-08T05:18:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:18:55.096+0000 7fe37a406700 1 -- 172.21.15.143:0/2803818950 wait complete. 2024-05-08T05:18:55.097 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 2 2024-05-08T05:18:55.654 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:55 smithi143 ceph-mon[34337]: Deploying daemon mon.b on smithi096 2024-05-08T05:18:55.654 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:55 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/2803818950' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:18:55.655 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:56.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:55 smithi059 ceph-mon[30304]: Deploying daemon mon.b on smithi096 2024-05-08T05:18:56.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:55 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/2803818950' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:18:56.007 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:18:56.657 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-05-08T05:18:56.658 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph mon dump -f json 2024-05-08T05:18:56.895 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:18:56.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:18:56 smithi143 ceph-mon[34337]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:57.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:18:56 smithi059 ceph-mon[30304]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:18:58.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:18:58 smithi096 systemd[1]: Started Ceph mon.b for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: mon.a calling monitor election 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: mon.c calling monitor election 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: mon.b calling monitor election 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: monmap epoch 3 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: last_changed 2024-05-08T05:18:58.343953+0000 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: created 2024-05-08T05:16:57.521541+0000 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: min_mon_release 19 (squid) 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: election_strategy: 1 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-08T05:19:03.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-08T05:19:03.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: fsmap 2024-05-08T05:19:03.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:19:03.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: mgrmap e13: x(active, since 59s) 2024-05-08T05:19:03.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: overall HEALTH_OK 2024-05-08T05:19:03.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:03.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:19:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: mon.a calling monitor election 2024-05-08T05:19:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: mon.c calling monitor election 2024-05-08T05:19:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:19:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: mon.b calling monitor election 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: monmap epoch 3 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: last_changed 2024-05-08T05:18:58.343953+0000 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: created 2024-05-08T05:16:57.521541+0000 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: min_mon_release 19 (squid) 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: election_strategy: 1 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-08T05:19:03.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-08T05:19:03.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: fsmap 2024-05-08T05:19:03.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:19:03.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: mgrmap e13: x(active, since 59s) 2024-05-08T05:19:03.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: overall HEALTH_OK 2024-05-08T05:19:03.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:03.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:03.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: mon.a calling monitor election 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: mon.c calling monitor election 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: mon.b calling monitor election 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: monmap epoch 3 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: last_changed 2024-05-08T05:18:58.343953+0000 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: created 2024-05-08T05:16:57.521541+0000 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: min_mon_release 19 (squid) 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: election_strategy: 1 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: 0: [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] mon.a 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: 1: [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] mon.c 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: 2: [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] mon.b 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: fsmap 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: osdmap e4: 0 total, 0 up, 0 in 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: mgrmap e13: x(active, since 59s) 2024-05-08T05:19:03.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: overall HEALTH_OK 2024-05-08T05:19:03.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:03.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.349+0000 7f781c6cc700 1 -- 172.21.15.143:0/30502254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78000062a0 msgr2=0x7f7800008710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:04.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.349+0000 7f781c6cc700 1 --2- 172.21.15.143:0/30502254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78000062a0 0x7f7800008710 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7814102f60 tx=0x7f781000fe90 comp rx=0 tx=0).stop 2024-05-08T05:19:04.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.349+0000 7f781c6cc700 1 -- 172.21.15.143:0/30502254 shutdown_connections 2024-05-08T05:19:04.354 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.349+0000 7f781c6cc700 1 --2- 172.21.15.143:0/30502254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78000062a0 0x7f7800008710 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:04.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.349+0000 7f781c6cc700 1 -- 172.21.15.143:0/30502254 >> 172.21.15.143:0/30502254 conn(0x7f78140fd990 msgr2=0x7f78140ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:19:04.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.349+0000 7f781c6cc700 1 -- 172.21.15.143:0/30502254 shutdown_connections 2024-05-08T05:19:04.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.349+0000 7f781c6cc700 1 -- 172.21.15.143:0/30502254 wait complete. 2024-05-08T05:19:04.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781c6cc700 1 Processor -- start 2024-05-08T05:19:04.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781c6cc700 1 -- start start 2024-05-08T05:19:04.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781c6cc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 0x7f781419ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:04.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781c6cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781419b360 0x7f781419f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:04.355 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781c6cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f781419fbe0 0x7f78141a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:04.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781c6cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7814109dd0 con 0x7f781419fbe0 2024-05-08T05:19:04.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781c6cc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7814109c50 con 0x7f781419b360 2024-05-08T05:19:04.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781c6cc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7814109f50 con 0x7f78000062a0 2024-05-08T05:19:04.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781a468700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 0x7f781419ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:04.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781a468700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 0x7f781419ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:36520/0 (socket says 172.21.15.143:36520) 2024-05-08T05:19:04.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781ac69700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f781419fbe0 0x7f78141a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:04.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f7819c67700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781419b360 0x7f781419f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:04.356 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.350+0000 7f781a468700 1 -- 172.21.15.143:0/3566748810 learned_addr learned my addr 172.21.15.143:0/3566748810 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:19:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781a468700 1 -- 172.21.15.143:0/3566748810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 msgr2=0x7f781419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-05-08T05:19:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f7819c67700 1 -- 172.21.15.143:0/3566748810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781419b360 msgr2=0x7f781419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 13 2024-05-08T05:19:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781a468700 1 -- 172.21.15.143:0/3566748810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 msgr2=0x7f781419ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T05:19:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f7819c67700 1 -- 172.21.15.143:0/3566748810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781419b360 msgr2=0x7f781419f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-05-08T05:19:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781a468700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 0x7f781419ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:19:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f7819c67700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781419b360 0x7f781419f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-05-08T05:19:04.357 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f7819c67700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781419b360 0x7f781419f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:19:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781a468700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 0x7f781419ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:19:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781ac69700 1 -- 172.21.15.143:0/3566748810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 msgr2=0x7f781419ae20 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:19:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781ac69700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 0x7f781419ae20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781ac69700 1 -- 172.21.15.143:0/3566748810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781419b360 msgr2=0x7f781419f6a0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:19:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781ac69700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781419b360 0x7f781419f6a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781ac69700 1 -- 172.21.15.143:0/3566748810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78141a7950 con 0x7f781419fbe0 2024-05-08T05:19:04.358 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781ac69700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f781419fbe0 0x7f78141a0090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f780c00bd50 tx=0x7f780c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f780b7fe700 1 -- 172.21.15.143:0/3566748810 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c019070 con 0x7f781419fbe0 2024-05-08T05:19:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f780b7fe700 1 -- 172.21.15.143:0/3566748810 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f780c00cd10 con 0x7f781419fbe0 2024-05-08T05:19:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78141a7b80 con 0x7f781419fbe0 2024-05-08T05:19:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f780b7fe700 1 -- 172.21.15.143:0/3566748810 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f780c012700 con 0x7f781419fbe0 2024-05-08T05:19:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.351+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78141a8070 con 0x7f781419fbe0 2024-05-08T05:19:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.352+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7814064b00 con 0x7f781419fbe0 2024-05-08T05:19:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.352+0000 7f780b7fe700 1 -- 172.21.15.143:0/3566748810 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f780c01c070 con 0x7f781419fbe0 2024-05-08T05:19:04.359 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.352+0000 7f780b7fe700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77f803e9a0 0x7f77f8040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:04.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.352+0000 7f780b7fe700 1 -- 172.21.15.143:0/3566748810 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f780c0520a0 con 0x7f781419fbe0 2024-05-08T05:19:04.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.353+0000 7f781a468700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77f803e9a0 0x7f77f8040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:04.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.353+0000 7f781a468700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77f803e9a0 0x7f77f8040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7810005b40 tx=0x7f7810005ab0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:04.360 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.358+0000 7f780b7fe700 1 -- 172.21.15.143:0/3566748810 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f780c01c360 con 0x7f781419fbe0 2024-05-08T05:19:04.607 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.608 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.608 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:04.608 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.608 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:04.660 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.659+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f781402e6d0 con 0x7f781419fbe0 2024-05-08T05:19:04.661 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.660+0000 7f780b7fe700 1 -- 172.21.15.143:0/3566748810 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1281 (secure 0 0 0) 0x7f780c01bd50 con 0x7f781419fbe0 2024-05-08T05:19:04.662 INFO:teuthology.orchestra.run.smithi143.stdout: 2024-05-08T05:19:04.662 INFO:teuthology.orchestra.run.smithi143.stdout:{"epoch":3,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","modified":"2024-05-08T05:18:58.343953Z","created":"2024-05-08T05:16:57.521541Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:3300","nonce":0},{"type":"v1","addr":"172.21.15.59:6789","nonce":0}]},"addr":"172.21.15.59:6789/0","public_addr":"172.21.15.59:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:3300","nonce":0},{"type":"v1","addr":"172.21.15.143:6789","nonce":0}]},"addr":"172.21.15.143:6789/0","public_addr":"172.21.15.143:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:3300","nonce":0},{"type":"v1","addr":"172.21.15.96:6789","nonce":0}]},"addr":"172.21.15.96:6789/0","public_addr":"172.21.15.96:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-05-08T05:19:04.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77f803e9a0 msgr2=0x7f77f8040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:04.664 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77f803e9a0 0x7f77f8040e60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7810005b40 tx=0x7f7810005ab0 comp rx=0 tx=0).stop 2024-05-08T05:19:04.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f781419fbe0 msgr2=0x7f78141a0090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:04.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f781419fbe0 0x7f78141a0090 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f780c00bd50 tx=0x7f780c00be30 comp rx=0 tx=0).stop 2024-05-08T05:19:04.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 shutdown_connections 2024-05-08T05:19:04.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f781419fbe0 0x7f78141a0090 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:04.665 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78000062a0 0x7f781419ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:04.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77f803e9a0 0x7f77f8040e60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:04.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 --2- 172.21.15.143:0/3566748810 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f781419b360 0x7f781419f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:04.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 >> 172.21.15.143:0/3566748810 conn(0x7f78140fd990 msgr2=0x7f781406b810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:19:04.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 shutdown_connections 2024-05-08T05:19:04.666 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:04.663+0000 7f781c6cc700 1 -- 172.21.15.143:0/3566748810 wait complete. 2024-05-08T05:19:04.666 INFO:teuthology.orchestra.run.smithi143.stderr:dumped monmap epoch 3 2024-05-08T05:19:04.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:04.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:19:05.393 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-05-08T05:19:05.393 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph config generate-minimal-conf 2024-05-08T05:19:05.667 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:19:05.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:05.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/3566748810' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:19:05.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='client.? 172.21.15.143:0/3566748810' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:05.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/3566748810' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-05-08T05:19:05.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.790 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:05.791 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-08T05:19:06.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-08T05:19:06.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-08T05:19:06.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:19:06.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:19:06.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:19:06.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:19:06.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:19:06.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:06.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:19:06.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:19:06.786 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:19:06.787 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:07.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.638+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3317704789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d941040b0 msgr2=0x7f7d94104490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.638+0000 7f7d9b21a700 1 --2- 172.21.15.59:0/3317704789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d941040b0 0x7f7d94104490 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7d88009910 tx=0x7f7d88009c20 comp rx=0 tx=0).stop 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.638+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3317704789 shutdown_connections 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.638+0000 7f7d9b21a700 1 --2- 172.21.15.59:0/3317704789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d941040b0 0x7f7d94104490 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.638+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3317704789 >> 172.21.15.59:0/3317704789 conn(0x7f7d940fd900 msgr2=0x7f7d940ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.638+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3317704789 shutdown_connections 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.638+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3317704789 wait complete. 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d9b21a700 1 Processor -- start 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d9b21a700 1 -- start start 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d9b21a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d941040b0 0x7f7d9419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:07.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d9b21a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d9419ed80 0x7f7d94198a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d9b21a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94198f40 0x7f7d941993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d9b21a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410d430 con 0x7f7d94198f40 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d9b21a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410d2b0 con 0x7f7d941040b0 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d9b21a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7d9410d5b0 con 0x7f7d9419ed80 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d997b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94198f40 0x7f7d941993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d997b7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94198f40 0x7f7d941993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50068/0 (socket says 172.21.15.59:50068) 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d997b7700 1 -- 172.21.15.59:0/3808634757 learned_addr learned my addr 172.21.15.59:0/3808634757 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d997b7700 1 -- 172.21.15.59:0/3808634757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d9419ed80 msgr2=0x7f7d94198a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d997b7700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d9419ed80 0x7f7d94198a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d997b7700 1 -- 172.21.15.59:0/3808634757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d941040b0 msgr2=0x7f7d9419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:07.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d997b7700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d941040b0 0x7f7d9419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.639+0000 7f7d997b7700 1 -- 172.21.15.59:0/3808634757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d941a5710 con 0x7f7d94198f40 2024-05-08T05:19:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.640+0000 7f7d997b7700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94198f40 0x7f7d941993f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00ba50 tx=0x7f7d8c00be10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.640+0000 7f7d91ffb700 1 -- 172.21.15.59:0/3808634757 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c019070 con 0x7f7d94198f40 2024-05-08T05:19:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.640+0000 7f7d91ffb700 1 -- 172.21.15.59:0/3808634757 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d8c014440 con 0x7f7d94198f40 2024-05-08T05:19:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.640+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d941a5940 con 0x7f7d94198f40 2024-05-08T05:19:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.640+0000 7f7d91ffb700 1 -- 172.21.15.59:0/3808634757 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d8c007590 con 0x7f7d94198f40 2024-05-08T05:19:07.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.640+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d941a5e80 con 0x7f7d94198f40 2024-05-08T05:19:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.641+0000 7f7d91ffb700 1 -- 172.21.15.59:0/3808634757 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f7d8c01c070 con 0x7f7d94198f40 2024-05-08T05:19:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.641+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d94064b00 con 0x7f7d94198f40 2024-05-08T05:19:07.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.641+0000 7f7d91ffb700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7d8403e9a0 0x7f7d84040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.641+0000 7f7d91ffb700 1 -- 172.21.15.59:0/3808634757 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f7d8c051e00 con 0x7f7d94198f40 2024-05-08T05:19:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.641+0000 7f7d98fb6700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7d8403e9a0 0x7f7d84040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:07.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.642+0000 7f7d98fb6700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7d8403e9a0 0x7f7d84040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800b5c0 tx=0x7f7d88005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:07.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.647+0000 7f7d91ffb700 1 -- 172.21.15.59:0/3808634757 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7d8c01c360 con 0x7f7d94198f40 2024-05-08T05:19:07.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:07 smithi143 ceph-mon[34337]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:19:07.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:07 smithi143 ceph-mon[34337]: Reconfiguring daemon mon.a on smithi059 2024-05-08T05:19:07.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:07 smithi143 ceph-mon[34337]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:07.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:07 smithi096 ceph-mon[34253]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:19:07.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:07 smithi096 ceph-mon[34253]: Reconfiguring daemon mon.a on smithi059 2024-05-08T05:19:07.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:07 smithi096 ceph-mon[34253]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:07.824 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:07 smithi059 ceph-mon[30304]: Reconfiguring mon.a (unknown last config time)... 2024-05-08T05:19:07.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:07 smithi059 ceph-mon[30304]: Reconfiguring daemon mon.a on smithi059 2024-05-08T05:19:07.825 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:07 smithi059 ceph-mon[30304]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:07.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.862+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f7d941a61d0 con 0x7f7d94198f40 2024-05-08T05:19:07.863 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.863+0000 7f7d91ffb700 1 -- 172.21.15.59:0/3808634757 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7f7d8c012e30 con 0x7f7d94198f40 2024-05-08T05:19:07.863 INFO:teuthology.orchestra.run.smithi059.stdout:# minimal ceph.conf for 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:19:07.863 INFO:teuthology.orchestra.run.smithi059.stdout:[global] 2024-05-08T05:19:07.864 INFO:teuthology.orchestra.run.smithi059.stdout: fsid = 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:19:07.864 INFO:teuthology.orchestra.run.smithi059.stdout: mon_host = [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] 2024-05-08T05:19:07.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7d8403e9a0 msgr2=0x7f7d84040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:07.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7d8403e9a0 0x7f7d84040e60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f7d8800b5c0 tx=0x7f7d88005da0 comp rx=0 tx=0).stop 2024-05-08T05:19:07.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94198f40 msgr2=0x7f7d941993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:07.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94198f40 0x7f7d941993f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f7d8c00ba50 tx=0x7f7d8c00be10 comp rx=0 tx=0).stop 2024-05-08T05:19:07.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 shutdown_connections 2024-05-08T05:19:07.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7d94198f40 0x7f7d941993f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:07.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7d8403e9a0 0x7f7d84040e60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:07.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7d941040b0 0x7f7d9419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:07.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 --2- 172.21.15.59:0/3808634757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7d9419ed80 0x7f7d94198a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:07.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 >> 172.21.15.59:0/3808634757 conn(0x7f7d940fd900 msgr2=0x7f7d94104f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:19:07.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 shutdown_connections 2024-05-08T05:19:07.867 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:07.865+0000 7f7d9b21a700 1 -- 172.21.15.59:0/3808634757 wait complete. 2024-05-08T05:19:08.730 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:08 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3808634757' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:08.730 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:08.730 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:08.730 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:19:08.730 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:19:08.730 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:08.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:08 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3808634757' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:08.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:08.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:08.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:19:08.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:19:08.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:08.747 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:08 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3808634757' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:08.747 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:08.747 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:08.748 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:19:08.748 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:19:08.748 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:08.775 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-05-08T05:19:08.775 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:19:08.775 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:19:08.805 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:19:08.805 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:19:08.871 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:19:08.872 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:19:08.898 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:19:08.898 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:19:08.963 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:19:08.963 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.conf 2024-05-08T05:19:08.995 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:19:08.995 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:19:09.063 INFO:tasks.cephadm:Adding mgr.x on smithi059 2024-05-08T05:19:09.064 INFO:tasks.cephadm:Adding mgr.y on smithi096 2024-05-08T05:19:09.064 INFO:tasks.cephadm:Adding mgr.z on smithi143 2024-05-08T05:19:09.064 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch apply mgr '3;smithi059=x;smithi096=y;smithi143=z' 2024-05-08T05:19:09.336 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:19:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:09 smithi096 ceph-mon[34253]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:09 smithi096 ceph-mon[34253]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:19:09.636 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:09 smithi096 ceph-mon[34253]: Reconfiguring daemon mon.b on smithi096 2024-05-08T05:19:09.700 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:09 smithi143 ceph-mon[34337]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:09.700 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:09 smithi143 ceph-mon[34337]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:19:09.700 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:09 smithi143 ceph-mon[34337]: Reconfiguring daemon mon.b on smithi096 2024-05-08T05:19:09.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:09 smithi059 ceph-mon[30304]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:09.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:09 smithi059 ceph-mon[30304]: Reconfiguring mon.b (monmap changed)... 2024-05-08T05:19:09.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:09 smithi059 ceph-mon[30304]: Reconfiguring daemon mon.b on smithi096 2024-05-08T05:19:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:10 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:10 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:10 smithi143 ceph-mon[34337]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:19:10.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:10 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:19:10.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:10 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:19:10.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:10 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:10.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:10 smithi143 ceph-mon[34337]: Reconfiguring daemon mon.c on smithi143 2024-05-08T05:19:10.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:10 smithi143 ceph-mon[34337]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:10 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:10 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:10 smithi096 ceph-mon[34253]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:19:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:10 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:19:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:10 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:19:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:10 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:10 smithi096 ceph-mon[34253]: Reconfiguring daemon mon.c on smithi143 2024-05-08T05:19:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:10 smithi096 ceph-mon[34253]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:11.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:10 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:11.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:10 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:11.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:10 smithi059 ceph-mon[30304]: Reconfiguring mon.c (monmap changed)... 2024-05-08T05:19:11.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:10 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-05-08T05:19:11.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:10 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-05-08T05:19:11.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:10 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:11.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:10 smithi059 ceph-mon[30304]: Reconfiguring daemon mon.c on smithi143 2024-05-08T05:19:11.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:10 smithi059 ceph-mon[30304]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:11.282 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.279+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/1314540419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d28103a90 msgr2=0x7f2d28105e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.279+0000 7f2d2e8f2700 1 --2- 172.21.15.143:0/1314540419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d28103a90 0x7f2d28105e80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f2d18009910 tx=0x7f2d18009c20 comp rx=0 tx=0).stop 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.279+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/1314540419 shutdown_connections 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.279+0000 7f2d2e8f2700 1 --2- 172.21.15.143:0/1314540419 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d28103a90 0x7f2d28105e80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.279+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/1314540419 >> 172.21.15.143:0/1314540419 conn(0x7f2d280fd4a0 msgr2=0x7f2d280ff8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.279+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/1314540419 shutdown_connections 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.279+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/1314540419 wait complete. 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.280+0000 7f2d2e8f2700 1 Processor -- start 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.280+0000 7f2d2e8f2700 1 -- start start 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.280+0000 7f2d2e8f2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d28103a90 0x7f2d281986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.280+0000 7f2d2e8f2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d28198c20 0x7f2d2819dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:11.283 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d2e8f2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281990a0 0x7f2d281a0220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d2e8f2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28108ee0 con 0x7f2d28198c20 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d2e8f2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28108d60 con 0x7f2d281990a0 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d2e8f2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2d28109060 con 0x7f2d28103a90 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d27fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d28103a90 0x7f2d281986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d27fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d28103a90 0x7f2d281986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:37778/0 (socket says 172.21.15.143:37778) 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d27fff700 1 -- 172.21.15.143:0/3275407430 learned_addr learned my addr 172.21.15.143:0/3275407430 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d277fe700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d28198c20 0x7f2d2819dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d27fff700 1 -- 172.21.15.143:0/3275407430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281990a0 msgr2=0x7f2d281a0220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d27fff700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281990a0 0x7f2d281a0220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d27fff700 1 -- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d28198c20 msgr2=0x7f2d2819dc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d27fff700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d28198c20 0x7f2d2819dc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d27fff700 1 -- 172.21.15.143:0/3275407430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d281a0af0 con 0x7f2d28103a90 2024-05-08T05:19:11.284 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d27fff700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d28103a90 0x7f2d281986e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005e00 tx=0x7f2d18004fa0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:11.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d257fa700 1 -- 172.21.15.143:0/3275407430 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d18027070 con 0x7f2d28103a90 2024-05-08T05:19:11.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d281a0de0 con 0x7f2d28103a90 2024-05-08T05:19:11.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d257fa700 1 -- 172.21.15.143:0/3275407430 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d1800be10 con 0x7f2d28103a90 2024-05-08T05:19:11.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.281+0000 7f2d257fa700 1 -- 172.21.15.143:0/3275407430 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1800f460 con 0x7f2d28103a90 2024-05-08T05:19:11.285 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.282+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d281a1360 con 0x7f2d28103a90 2024-05-08T05:19:11.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.282+0000 7f2d257fa700 1 -- 172.21.15.143:0/3275407430 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f2d1802a070 con 0x7f2d28103a90 2024-05-08T05:19:11.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.283+0000 7f2d257fa700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2d1003e9a0 0x7f2d10040e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:11.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.283+0000 7f2d257fa700 1 -- 172.21.15.143:0/3275407430 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f2d180539a0 con 0x7f2d28103a90 2024-05-08T05:19:11.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.283+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d28064b00 con 0x7f2d28103a90 2024-05-08T05:19:11.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.283+0000 7f2d277fe700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2d1003e9a0 0x7f2d10040e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:11.286 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.283+0000 7f2d277fe700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2d1003e9a0 0x7f2d10040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009ad0 tx=0x7f2d1c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:11.289 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.288+0000 7f2d257fa700 1 -- 172.21.15.143:0/3275407430 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2d18013070 con 0x7f2d28103a90 2024-05-08T05:19:11.508 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.508+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi096=y;smithi143=z", "target": ["mon-mgr", ""]}) -- 0x7f2d28063970 con 0x7f2d1003e9a0 2024-05-08T05:19:11.515 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.514+0000 7f2d257fa700 1 -- 172.21.15.143:0/3275407430 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f2d28063970 con 0x7f2d1003e9a0 2024-05-08T05:19:11.515 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mgr update... 2024-05-08T05:19:11.517 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.516+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2d1003e9a0 msgr2=0x7f2d10040e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.516+0000 7f2d2e8f2700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2d1003e9a0 0x7f2d10040e60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c009ad0 tx=0x7f2d1c009210 comp rx=0 tx=0).stop 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.516+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d28103a90 msgr2=0x7f2d281986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.516+0000 7f2d2e8f2700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d28103a90 0x7f2d281986e0 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f2d18005e00 tx=0x7f2d18004fa0 comp rx=0 tx=0).stop 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.517+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 shutdown_connections 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.517+0000 7f2d2e8f2700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2d1003e9a0 0x7f2d10040e60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.517+0000 7f2d2e8f2700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2d281990a0 0x7f2d281a0220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.517+0000 7f2d2e8f2700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2d28103a90 0x7f2d281986e0 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.517+0000 7f2d2e8f2700 1 --2- 172.21.15.143:0/3275407430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2d28198c20 0x7f2d2819dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.517+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 >> 172.21.15.143:0/3275407430 conn(0x7f2d280fd4a0 msgr2=0x7f2d280fe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.517+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 shutdown_connections 2024-05-08T05:19:11.518 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:19:11.517+0000 7f2d2e8f2700 1 -- 172.21.15.143:0/3275407430 wait complete. 2024-05-08T05:19:12.353 DEBUG:teuthology.orchestra.run.smithi096:mgr.y> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.y.service 2024-05-08T05:19:12.356 DEBUG:teuthology.orchestra.run.smithi143:mgr.z> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.z.service 2024-05-08T05:19:12.358 INFO:tasks.cephadm:Deploying OSDs... 2024-05-08T05:19:12.359 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:19:12.359 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:19:12.376 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:19:12.376 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_1 2024-05-08T05:19:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:19:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:12.433 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 666 Links: 1 2024-05-08T05:19:12.433 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:12.433 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:12.433 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-08 05:18:15.946898649 +0000 2024-05-08T05:19:12.433 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-08 05:12:38.361385029 +0000 2024-05-08T05:19:12.433 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-08 05:12:38.361385029 +0000 2024-05-08T05:19:12.433 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-08 05:12:38.361385029 +0000 2024-05-08T05:19:12.434 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:19:12.498 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-08T05:19:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-08T05:19:12.499 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000215983 s, 2.4 MB/s 2024-05-08T05:19:12.500 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:19:12.557 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_2 2024-05-08T05:19:12.613 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:19:12.614 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:12.614 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-05-08T05:19:12.614 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:12.614 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:12.614 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-08 05:18:15.984898173 +0000 2024-05-08T05:19:12.614 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-08 05:12:38.718380485 +0000 2024-05-08T05:19:12.614 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-08 05:12:38.718380485 +0000 2024-05-08T05:19:12.614 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-08 05:12:38.718380485 +0000 2024-05-08T05:19:12.614 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:19:12.678 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-08T05:19:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-08T05:19:12.679 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000215267 s, 2.4 MB/s 2024-05-08T05:19:12.680 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:19:12.737 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_3 2024-05-08T05:19:12.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi096=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:19:12.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: Saving service mgr spec with placement smithi059=x;smithi096=y;smithi143=z;count:3 2024-05-08T05:19:12.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:12.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:12 smithi143 ceph-mon[34337]: Deploying daemon mgr.z on smithi143 2024-05-08T05:19:12.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi096=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:19:12.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: Saving service mgr spec with placement smithi059=x;smithi096=y;smithi143=z;count:3 2024-05-08T05:19:12.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:12.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:12.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:12.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:12.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:19:12.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:19:12.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:19:12.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:12.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:12 smithi096 ceph-mon[34253]: Deploying daemon mgr.z on smithi143 2024-05-08T05:19:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:19:12.793 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:12.793 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-05-08T05:19:12.793 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:12.793 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:12.793 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-08 05:18:16.021897709 +0000 2024-05-08T05:19:12.794 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-08 05:12:39.074375954 +0000 2024-05-08T05:19:12.794 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-08 05:12:39.074375954 +0000 2024-05-08T05:19:12.794 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-08 05:12:39.074375954 +0000 2024-05-08T05:19:12.794 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:19:12.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='client.24100 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi059=x;smithi096=y;smithi143=z", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:19:12.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: Saving service mgr spec with placement smithi059=x;smithi096=y;smithi143=z;count:3 2024-05-08T05:19:12.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:12.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:12 smithi059 ceph-mon[30304]: Deploying daemon mgr.z on smithi143 2024-05-08T05:19:12.858 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-08T05:19:12.858 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-08T05:19:12.858 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000221149 s, 2.3 MB/s 2024-05-08T05:19:12.859 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:19:12.916 DEBUG:teuthology.orchestra.run.smithi059:> stat /dev/vg_nvme/lv_4 2024-05-08T05:19:12.972 INFO:teuthology.orchestra.run.smithi059.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:19:12.972 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:12.972 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 5h/5d Inode: 725 Links: 1 2024-05-08T05:19:12.972 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:12.972 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:12.972 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-08 05:18:16.055897283 +0000 2024-05-08T05:19:12.972 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-08 05:12:39.424371532 +0000 2024-05-08T05:19:12.972 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-08 05:12:39.424371532 +0000 2024-05-08T05:19:12.973 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-08 05:12:39.424371532 +0000 2024-05-08T05:19:12.973 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:19:13.036 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records in 2024-05-08T05:19:13.037 INFO:teuthology.orchestra.run.smithi059.stderr:1+0 records out 2024-05-08T05:19:13.037 INFO:teuthology.orchestra.run.smithi059.stderr:512 bytes copied, 0.000209088 s, 2.4 MB/s 2024-05-08T05:19:13.038 DEBUG:teuthology.orchestra.run.smithi059:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:19:13.095 DEBUG:teuthology.orchestra.run.smithi096:> set -ex 2024-05-08T05:19:13.095 DEBUG:teuthology.orchestra.run.smithi096:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:19:13.114 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:19:13.114 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_1 2024-05-08T05:19:13.172 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:19:13.172 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:13.172 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 660 Links: 1 2024-05-08T05:19:13.172 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:13.173 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:13.173 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-08 05:18:57.784384320 +0000 2024-05-08T05:19:13.173 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-08 05:13:45.793945808 +0000 2024-05-08T05:19:13.173 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-08 05:13:45.793945808 +0000 2024-05-08T05:19:13.173 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-08 05:13:45.793945808 +0000 2024-05-08T05:19:13.173 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:19:13.239 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-08T05:19:13.239 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-08T05:19:13.239 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000373578 s, 1.4 MB/s 2024-05-08T05:19:13.240 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:19:13.299 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_2 2024-05-08T05:19:13.356 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:19:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:13.357 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 678 Links: 1 2024-05-08T05:19:13.357 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:13.357 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:13.357 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-08 05:18:57.785384296 +0000 2024-05-08T05:19:13.357 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-08 05:13:46.131937917 +0000 2024-05-08T05:19:13.357 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-08 05:13:46.131937917 +0000 2024-05-08T05:19:13.357 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-08 05:13:46.131937917 +0000 2024-05-08T05:19:13.357 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:19:13.421 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-08T05:19:13.422 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-08T05:19:13.422 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000202463 s, 2.5 MB/s 2024-05-08T05:19:13.423 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:19:13.481 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_3 2024-05-08T05:19:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:19:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:13.537 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 698 Links: 1 2024-05-08T05:19:13.537 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:13.537 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:13.537 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-08 05:18:57.785384296 +0000 2024-05-08T05:19:13.537 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-08 05:13:46.496929396 +0000 2024-05-08T05:19:13.537 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-08 05:13:46.496929396 +0000 2024-05-08T05:19:13.537 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-08 05:13:46.496929396 +0000 2024-05-08T05:19:13.538 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:19:13.600 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-08T05:19:13.600 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-08T05:19:13.600 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.000225945 s, 2.3 MB/s 2024-05-08T05:19:13.601 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:19:13.658 DEBUG:teuthology.orchestra.run.smithi096:> stat /dev/vg_nvme/lv_4 2024-05-08T05:19:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:19:13.715 INFO:teuthology.orchestra.run.smithi096.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:13.715 INFO:teuthology.orchestra.run.smithi096.stdout:Device: 5h/5d Inode: 717 Links: 1 2024-05-08T05:19:13.715 INFO:teuthology.orchestra.run.smithi096.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:13.715 INFO:teuthology.orchestra.run.smithi096.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:13.716 INFO:teuthology.orchestra.run.smithi096.stdout:Access: 2024-05-08 05:18:57.785384296 +0000 2024-05-08T05:19:13.716 INFO:teuthology.orchestra.run.smithi096.stdout:Modify: 2024-05-08 05:13:46.834921505 +0000 2024-05-08T05:19:13.716 INFO:teuthology.orchestra.run.smithi096.stdout:Change: 2024-05-08 05:13:46.834921505 +0000 2024-05-08T05:19:13.716 INFO:teuthology.orchestra.run.smithi096.stdout: Birth: 2024-05-08 05:13:46.834921505 +0000 2024-05-08T05:19:13.716 DEBUG:teuthology.orchestra.run.smithi096:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:19:13.777 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records in 2024-05-08T05:19:13.778 INFO:teuthology.orchestra.run.smithi096.stderr:1+0 records out 2024-05-08T05:19:13.778 INFO:teuthology.orchestra.run.smithi096.stderr:512 bytes copied, 0.00019231 s, 2.7 MB/s 2024-05-08T05:19:13.779 DEBUG:teuthology.orchestra.run.smithi096:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:19:13.835 DEBUG:teuthology.orchestra.run.smithi143:> set -ex 2024-05-08T05:19:13.835 DEBUG:teuthology.orchestra.run.smithi143:> dd if=/scratch_devs of=/dev/stdout 2024-05-08T05:19:13.851 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-05-08T05:19:13.851 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_1 2024-05-08T05:19:13.906 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-05-08T05:19:13.906 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:13.906 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-05-08T05:19:13.907 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:13.907 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:13.907 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-05-08 05:18:48.607539870 +0000 2024-05-08T05:19:13.907 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-05-08 05:13:04.154433808 +0000 2024-05-08T05:19:13.907 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-05-08 05:13:04.154433808 +0000 2024-05-08T05:19:13.907 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-05-08 05:13:04.154433808 +0000 2024-05-08T05:19:13.907 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-05-08T05:19:14.072 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-05-08T05:19:14.072 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-05-08T05:19:14.073 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000253503 s, 2.0 MB/s 2024-05-08T05:19:14.073 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-05-08T05:19:14.090 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_2 2024-05-08T05:19:14.150 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-05-08T05:19:14.150 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:14.150 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 682 Links: 1 2024-05-08T05:19:14.150 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:14.151 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:14.151 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-05-08 05:19:14.026880838 +0000 2024-05-08T05:19:14.151 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-05-08 05:13:04.493424913 +0000 2024-05-08T05:19:14.151 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-05-08 05:13:04.493424913 +0000 2024-05-08T05:19:14.151 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-05-08 05:13:04.493424913 +0000 2024-05-08T05:19:14.151 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-05-08T05:19:14.251 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-05-08T05:19:14.251 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-05-08T05:19:14.251 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000205157 s, 2.5 MB/s 2024-05-08T05:19:14.252 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-05-08T05:19:14.267 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_3 2024-05-08T05:19:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-05-08T05:19:14.323 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:14.323 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-05-08T05:19:14.323 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:14.324 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:14.324 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-05-08 05:19:14.207876146 +0000 2024-05-08T05:19:14.324 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-05-08 05:13:04.849415573 +0000 2024-05-08T05:19:14.324 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-05-08 05:13:04.849415573 +0000 2024-05-08T05:19:14.324 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-05-08 05:13:04.849415573 +0000 2024-05-08T05:19:14.324 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-05-08T05:19:14.389 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-05-08T05:19:14.389 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-05-08T05:19:14.389 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.00021904 s, 2.3 MB/s 2024-05-08T05:19:14.390 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-05-08T05:19:14.446 DEBUG:teuthology.orchestra.run.smithi143:> stat /dev/vg_nvme/lv_4 2024-05-08T05:19:14.501 INFO:teuthology.orchestra.run.smithi143.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-05-08T05:19:14.501 INFO:teuthology.orchestra.run.smithi143.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-05-08T05:19:14.501 INFO:teuthology.orchestra.run.smithi143.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-05-08T05:19:14.501 INFO:teuthology.orchestra.run.smithi143.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:19:14.502 INFO:teuthology.orchestra.run.smithi143.stdout:Context: system_u:object_r:device_t:s0 2024-05-08T05:19:14.502 INFO:teuthology.orchestra.run.smithi143.stdout:Access: 2024-05-08 05:19:14.207876146 +0000 2024-05-08T05:19:14.502 INFO:teuthology.orchestra.run.smithi143.stdout:Modify: 2024-05-08 05:13:05.180406887 +0000 2024-05-08T05:19:14.502 INFO:teuthology.orchestra.run.smithi143.stdout:Change: 2024-05-08 05:13:05.180406887 +0000 2024-05-08T05:19:14.502 INFO:teuthology.orchestra.run.smithi143.stdout: Birth: 2024-05-08 05:13:05.180406887 +0000 2024-05-08T05:19:14.502 DEBUG:teuthology.orchestra.run.smithi143:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-05-08T05:19:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records in 2024-05-08T05:19:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:1+0 records out 2024-05-08T05:19:14.564 INFO:teuthology.orchestra.run.smithi143.stderr:512 bytes copied, 0.000198961 s, 2.6 MB/s 2024-05-08T05:19:14.565 DEBUG:teuthology.orchestra.run.smithi143:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-05-08T05:19:14.621 INFO:tasks.cephadm:Deploying osd.0 on smithi059 with /dev/vg_nvme/lv_4... 2024-05-08T05:19:14.622 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:19:14.755 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:14 smithi143 podman[36104]: 2024-05-08 05:19:14.730721675 +0000 UTC m=+0.346035029 container start 8ac990e3137bbebc3c120bb12d0e3de062022b2310a16267e92e9b1e827822ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:19:14.878 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:19:15.074 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:14 smithi143 ceph-mon[34337]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:15.074 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:15.074 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:15.074 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:15.075 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:19:15.075 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:19:15.075 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:19:15.075 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:15.075 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:14 smithi143 ceph-mgr[36197]: -- 172.21.15.143:0/1349444815 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ff212da5a0 con 0x55ff212aec00 2024-05-08T05:19:15.075 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:14 smithi143 bash[36104]: 8ac990e3137bbebc3c120bb12d0e3de062022b2310a16267e92e9b1e827822ab 2024-05-08T05:19:15.075 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:14 smithi143 systemd[1]: Started Ceph mgr.z for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:19:15.075 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:14 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:14.886+0000 7fba718fb200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:19:15.075 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:15.073+0000 7fba718fb200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:19:15.219 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:14 smithi096 ceph-mon[34253]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:15.220 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:15.220 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:15.220 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:15.220 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:19:15.220 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:19:15.220 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:19:15.220 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:15.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:14 smithi059 ceph-mon[30304]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:15.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:15.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:15.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:15.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:19:15.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-05-08T05:19:15.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:19:15.226 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:15.483 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:15.175+0000 7fba718fb200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:19:15.483 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:15.482+0000 7fba718fb200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:19:15.744 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:15.568+0000 7fba718fb200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:19:16.073 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:15 smithi059 ceph-mon[30304]: Deploying daemon mgr.y on smithi096 2024-05-08T05:19:16.187 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:15 smithi143 ceph-mon[34337]: Deploying daemon mgr.y on smithi096 2024-05-08T05:19:16.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:15 smithi096 ceph-mon[34253]: Deploying daemon mgr.y on smithi096 2024-05-08T05:19:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:19:16.615 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:19:16.615 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-08T05:19:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-08T05:19:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112776 s, 930 MB/s 2024-05-08T05:19:16.616 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 2024-05-08T05:19:16.616 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-08T05:19:16.950 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:16 smithi096 systemd[1]: Starting Ceph mgr.y for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:19:17.146 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:16 smithi143 ceph-mon[34337]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:17.146 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:16 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:16.799+0000 7fba718fb200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:19:17.146 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:16 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:16.971+0000 7fba718fb200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:19:17.189 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_4 2024-05-08T05:19:17.214 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:16 smithi059 ceph-mon[30304]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:17.254 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:16 smithi096 ceph-mon[34253]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:17.254 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:16 smithi096 podman[35634]: 2024-05-08 05:19:16.852567531 +0000 UTC m=+0.017624393 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:19:17.255 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:16 smithi096 podman[35634]: 2024-05-08 05:19:16.960719752 +0000 UTC m=+0.125776637 container create 02b28c2448fd5d95bd91ab9a3c3d66f6694e147d7ca03673bc7842615414cd9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True) 2024-05-08T05:19:17.255 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:17 smithi096 podman[35634]: 2024-05-08 05:19:17.171264049 +0000 UTC m=+0.336320889 container init 02b28c2448fd5d95bd91ab9a3c3d66f6694e147d7ca03673bc7842615414cd9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5) 2024-05-08T05:19:17.255 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:17 smithi096 podman[35634]: 2024-05-08 05:19:17.173743355 +0000 UTC m=+0.338800206 container start 02b28c2448fd5d95bd91ab9a3c3d66f6694e147d7ca03673bc7842615414cd9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True) 2024-05-08T05:19:17.255 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:17 smithi096 ceph-mgr[35654]: -- 172.21.15.96:0/2102323380 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b11ccf05a0 con 0x55b11ccc5000 2024-05-08T05:19:17.255 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:17 smithi096 bash[35634]: 02b28c2448fd5d95bd91ab9a3c3d66f6694e147d7ca03673bc7842615414cd9c 2024-05-08T05:19:17.437 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:17.148+0000 7fba718fb200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:19:17.437 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:17.436+0000 7fba718fb200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:19:17.443 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:19:17.512 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:17 smithi096 systemd[1]: Started Ceph mgr.y for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:19:17.512 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:17.323+0000 7f35ca758200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:19:17.512 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:17.511+0000 7f35ca758200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:19:17.743 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:17.532+0000 7fba718fb200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:19:17.779 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:17.619+0000 7f35ca758200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:19:18.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:17.748+0000 7fba718fb200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:19:18.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:17.946+0000 7fba718fb200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:19:18.251 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:17.947+0000 7f35ca758200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:19:18.251 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:18 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:18.045+0000 7f35ca758200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:19:18.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.509 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:18.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:18.598 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.599 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.599 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.599 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:18.599 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.977+0000 7f8258ace700 1 -- 172.21.15.59:0/18362291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82541092b0 msgr2=0x7f825410b6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.977+0000 7f8258ace700 1 --2- 172.21.15.59:0/18362291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82541092b0 0x7f825410b6a0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f824c00b0a0 tx=0x7f824c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:19:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.977+0000 7f8258ace700 1 -- 172.21.15.59:0/18362291 shutdown_connections 2024-05-08T05:19:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.977+0000 7f8258ace700 1 --2- 172.21.15.59:0/18362291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82541092b0 0x7f825410b6a0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.977+0000 7f8258ace700 1 --2- 172.21.15.59:0/18362291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8254104050 0x7f8254106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.977+0000 7f8258ace700 1 --2- 172.21.15.59:0/18362291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8254106980 0x7f8254108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.977+0000 7f8258ace700 1 -- 172.21.15.59:0/18362291 >> 172.21.15.59:0/18362291 conn(0x7f82540fd9a0 msgr2=0x7f82540ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:19:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.977+0000 7f8258ace700 1 -- 172.21.15.59:0/18362291 shutdown_connections 2024-05-08T05:19:18.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.977+0000 7f8258ace700 1 -- 172.21.15.59:0/18362291 wait complete. 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8258ace700 1 Processor -- start 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8258ace700 1 -- start start 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8258ace700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8254104050 0x7f82541995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8258ace700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8254106980 0x7f8254199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8258ace700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82541092b0 0x7f82541a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8258ace700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f825410ef70 con 0x7f8254104050 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8258ace700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f825410edf0 con 0x7f82541092b0 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8258ace700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f825410f0f0 con 0x7f8254106980 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f825259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8254104050 0x7f82541995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:18.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f825259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8254104050 0x7f82541995b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39620/0 (socket says 172.21.15.59:39620) 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f825259c700 1 -- 172.21.15.59:0/2735168375 learned_addr learned my addr 172.21.15.59:0/2735168375 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8251d9b700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8254106980 0x7f8254199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8252d9d700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82541092b0 0x7f82541a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f825259c700 1 -- 172.21.15.59:0/2735168375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8254106980 msgr2=0x7f8254199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f825259c700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8254106980 0x7f8254199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f825259c700 1 -- 172.21.15.59:0/2735168375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82541092b0 msgr2=0x7f82541a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f825259c700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82541092b0 0x7f82541a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f825259c700 1 -- 172.21.15.59:0/2735168375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82541a1210 con 0x7f8254104050 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8251d9b700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8254106980 0x7f8254199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.978+0000 7f8252d9d700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82541092b0 0x7f82541a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.979+0000 7f825259c700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8254104050 0x7f82541995b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f824400e9f0 tx=0x7f824400ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.979+0000 7f82437fe700 1 -- 172.21.15.59:0/2735168375 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f824400cbd0 con 0x7f8254104050 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.979+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82541a1500 con 0x7f8254104050 2024-05-08T05:19:18.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.979+0000 7f82437fe700 1 -- 172.21.15.59:0/2735168375 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f824400ee90 con 0x7f8254104050 2024-05-08T05:19:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.979+0000 7f82437fe700 1 -- 172.21.15.59:0/2735168375 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8244005590 con 0x7f8254104050 2024-05-08T05:19:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.979+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82541a1a40 con 0x7f8254104050 2024-05-08T05:19:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.980+0000 7f82437fe700 1 -- 172.21.15.59:0/2735168375 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 13) ==== 50278+0+0 (secure 0 0 0) 0x7f8244021440 con 0x7f8254104050 2024-05-08T05:19:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.980+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f825406a190 con 0x7f8254104050 2024-05-08T05:19:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.980+0000 7f82437fe700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f823c03e750 0x7f823c040c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.980+0000 7f82437fe700 1 -- 172.21.15.59:0/2735168375 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f8244052b70 con 0x7f8254104050 2024-05-08T05:19:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.980+0000 7f8251d9b700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f823c03e750 0x7f823c040c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:18.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.981+0000 7f8251d9b700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f823c03e750 0x7f823c040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8248011490 tx=0x7f8248009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:18.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:18.985+0000 7f82437fe700 1 -- 172.21.15.59:0/2735168375 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f824401bd90 con 0x7f8254104050 2024-05-08T05:19:18.993 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:18 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:18.597+0000 7fba718fb200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:19:18.993 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:18 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:18.827+0000 7fba718fb200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:19:19.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:19.225+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f82540638e0 con 0x7f823c03e750 2024-05-08T05:19:19.529 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:19.273+0000 7f35ca758200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:19.443+0000 7f35ca758200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:19:19.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:19:19.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:19.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:19:19.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:19:19.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:19:19.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:19.718 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:19.249+0000 7fba718fb200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:19:19.718 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:19.346+0000 7fba718fb200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:19:19.718 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:19.717+0000 7fba718fb200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:19.719 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:19:19.720 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:19:19.720 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:19.897 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:19.612+0000 7f35ca758200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:19:19.897 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:19.896+0000 7f35ca758200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:19:20.188 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:19.987+0000 7f35ca758200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:19:20.188 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:20.187+0000 7f35ca758200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:19:20.342 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:20.067+0000 7fba718fb200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:19:20.343 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:20.158+0000 7fba718fb200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:19:20.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:20 smithi059 ceph-mon[30304]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:19:20.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:20 smithi059 ceph-mon[30304]: Reconfiguring daemon mgr.x on smithi059 2024-05-08T05:19:20.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:20 smithi059 ceph-mon[30304]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:19:20.529 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:20.380+0000 7f35ca758200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:19:20.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:20 smithi096 ceph-mon[34253]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:19:20.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:20 smithi096 ceph-mon[34253]: Reconfiguring daemon mgr.x on smithi059 2024-05-08T05:19:20.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:20 smithi096 ceph-mon[34253]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:19:20.629 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:20 smithi143 ceph-mon[34337]: Reconfiguring mgr.x (unknown last config time)... 2024-05-08T05:19:20.629 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:20 smithi143 ceph-mon[34337]: Reconfiguring daemon mgr.x on smithi059 2024-05-08T05:19:20.630 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:20 smithi143 ceph-mon[34337]: from='client.14229 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:19:20.630 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:20.399+0000 7fba718fb200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:19:20.630 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:20.533+0000 7fba718fb200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:19:20.630 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:20.628+0000 7fba718fb200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:19:21.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:20.960+0000 7fba718fb200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:19:21.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:21.051+0000 7fba718fb200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:19:21.279 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:20.976+0000 7f35ca758200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:19:21.279 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:21 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:21.193+0000 7f35ca758200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:19:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:21 smithi059 ceph-mon[30304]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:21.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:21 smithi096 ceph-mon[34253]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:21.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:21.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:21.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:21.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:21.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:21.612 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:21.650 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:21 smithi143 ceph-mon[34337]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:21.650 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:21.650 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:21.650 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:21.651 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:21.651 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:21.651 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:21.993 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:21.649+0000 7fba718fb200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:19:21.993 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:19:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:19:21.865+0000 7fba718fb200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:19:22.029 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:21 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:21.610+0000 7f35ca758200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:19:22.030 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:21 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:21.705+0000 7f35ca758200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:19:22.350 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:22.066+0000 7f35ca758200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:19:22.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:22.353+0000 7f82437fe700 1 -- 172.21.15.59:0/2735168375 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 14) ==== 99937+0+0 (secure 0 0 0) 0x7f824401dd00 con 0x7f8254104050 2024-05-08T05:19:22.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:22 smithi143 ceph-mon[34337]: Standby manager daemon z started 2024-05-08T05:19:22.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:22 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:19:22.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:22 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:19:22.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:22 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:19:22.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:22 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:19:22.754 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:22 smithi096 ceph-mon[34253]: Standby manager daemon z started 2024-05-08T05:19:22.754 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:22 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:19:22.754 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:22 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:19:22.754 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:22 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:19:22.755 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:22 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:19:22.755 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:22.421+0000 7f35ca758200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:19:22.755 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:22.512+0000 7f35ca758200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:19:22.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:22 smithi059 ceph-mon[30304]: Standby manager daemon z started 2024-05-08T05:19:22.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:22 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:19:22.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:22 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:19:22.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:22 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:19:22.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:22 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.143:0/3150115183' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:19:23.030 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:22.753+0000 7f35ca758200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:19:23.030 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:22.885+0000 7f35ca758200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:19:23.030 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:22.984+0000 7f35ca758200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:19:23.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:23 smithi143 ceph-mon[34337]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:23.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:23 smithi143 ceph-mon[34337]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-08T05:19:23.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:23 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:19:23.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:23 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2409038724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bd987ee-e978-4d9b-89b6-6acca697cac4"}]: dispatch 2024-05-08T05:19:23.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:23 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2409038724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bd987ee-e978-4d9b-89b6-6acca697cac4"}]': finished 2024-05-08T05:19:23.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:23 smithi143 ceph-mon[34337]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:19:23.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:23 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:23.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:23 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3479525060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:19:23.779 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:23 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:23.307+0000 7f35ca758200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:19:23.780 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:23 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:23.401+0000 7f35ca758200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:19:23.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:23 smithi096 ceph-mon[34253]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:23.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:23 smithi096 ceph-mon[34253]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-08T05:19:23.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:23 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:19:23.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:23 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2409038724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bd987ee-e978-4d9b-89b6-6acca697cac4"}]: dispatch 2024-05-08T05:19:23.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:23 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2409038724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bd987ee-e978-4d9b-89b6-6acca697cac4"}]': finished 2024-05-08T05:19:23.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:23 smithi096 ceph-mon[34253]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:19:23.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:23 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:23.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:23 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3479525060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:19:23.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:23 smithi059 ceph-mon[30304]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:23.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:23 smithi059 ceph-mon[30304]: mgrmap e14: x(active, since 78s), standbys: z 2024-05-08T05:19:23.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:23 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:19:23.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:23 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2409038724' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "2bd987ee-e978-4d9b-89b6-6acca697cac4"}]: dispatch 2024-05-08T05:19:23.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:23 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2409038724' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "2bd987ee-e978-4d9b-89b6-6acca697cac4"}]': finished 2024-05-08T05:19:23.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:23 smithi059 ceph-mon[30304]: osdmap e5: 1 total, 0 up, 1 in 2024-05-08T05:19:23.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:23 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:23.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:23 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3479525060' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:19:24.279 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:23 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:23.996+0000 7f35ca758200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:19:24.280 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:19:24 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:19:24.211+0000 7f35ca758200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:19:24.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:24.361+0000 7f82437fe700 1 -- 172.21.15.59:0/2735168375 <== mon.0 v2:172.21.15.59:3300/0 8 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f8244016330 con 0x7f8254104050 2024-05-08T05:19:24.705 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:24 smithi096 ceph-mon[34253]: Standby manager daemon y started 2024-05-08T05:19:24.706 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:24 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:19:24.706 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:24 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:19:24.706 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:24 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:19:24.706 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:24 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:19:24.738 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:24 smithi059 ceph-mon[30304]: Standby manager daemon y started 2024-05-08T05:19:24.738 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:24 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:19:24.738 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:24 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:19:24.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:24 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:19:24.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:24 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:19:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:24 smithi143 ceph-mon[34337]: Standby manager daemon y started 2024-05-08T05:19:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:24 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:19:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:24 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:19:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:24 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:19:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:24 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.96:0/745172647' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:19:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:25 smithi143 ceph-mon[34337]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:25 smithi143 ceph-mon[34337]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-08T05:19:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:25 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:19:25.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:25 smithi096 ceph-mon[34253]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:25.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:25 smithi096 ceph-mon[34253]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-08T05:19:25.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:25 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:19:25.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:25 smithi059 ceph-mon[30304]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:25.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:25 smithi059 ceph-mon[30304]: mgrmap e15: x(active, since 80s), standbys: z, y 2024-05-08T05:19:25.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:25 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:19:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:27 smithi143 ceph-mon[34337]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:27.658 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:27 smithi059 ceph-mon[30304]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:27.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:27 smithi096 ceph-mon[34253]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:29.539 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:29 smithi059 ceph-mon[30304]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:29.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:29 smithi143 ceph-mon[34337]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:29.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:29 smithi096 ceph-mon[34253]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:30.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:19:30.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:30.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:19:30.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:30.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-05-08T05:19:30.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:31.689 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:31 smithi059 ceph-mon[30304]: Deploying daemon osd.0 on smithi059 2024-05-08T05:19:31.689 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:31 smithi059 ceph-mon[30304]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:31.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:31 smithi143 ceph-mon[34337]: Deploying daemon osd.0 on smithi059 2024-05-08T05:19:31.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:31 smithi143 ceph-mon[34337]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:31.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:31 smithi096 ceph-mon[34253]: Deploying daemon osd.0 on smithi059 2024-05-08T05:19:31.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:31 smithi096 ceph-mon[34253]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:33.630 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:33 smithi059 ceph-mon[30304]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:33.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:33 smithi143 ceph-mon[34337]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:33.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:33 smithi096 ceph-mon[34253]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:35.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:35 smithi143 ceph-mon[34337]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:35.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:35 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:35.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:35 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:35.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:35 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:35.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:35 smithi096 ceph-mon[34253]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:35.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:35 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:35.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:35 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:35.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:35 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:35.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:35 smithi059 ceph-mon[30304]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:35.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:35 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:35.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:35 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:35.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:35 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:37.595 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:37 smithi059 ceph-mon[30304]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:37.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:37 smithi143 ceph-mon[34337]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:37.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:37 smithi096 ceph-mon[34253]: pgmap v41: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:38.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.579+0000 7f82437fe700 1 -- 172.21.15.59:0/2735168375 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f82540638e0 con 0x7f823c03e750 2024-05-08T05:19:38.582 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 0 on host 'smithi059' 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f823c03e750 msgr2=0x7f823c040c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f823c03e750 0x7f823c040c10 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8248011490 tx=0x7f8248009260 comp rx=0 tx=0).stop 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8254104050 msgr2=0x7f82541995b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8254104050 0x7f82541995b0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f824400e9f0 tx=0x7f824400ed00 comp rx=0 tx=0).stop 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 shutdown_connections 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f823c03e750 0x7f823c040c10 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82541092b0 0x7f82541a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8254104050 0x7f82541995b0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 --2- 172.21.15.59:0/2735168375 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8254106980 0x7f8254199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 >> 172.21.15.59:0/2735168375 conn(0x7f82540fd9a0 msgr2=0x7f8254104910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 shutdown_connections 2024-05-08T05:19:38.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:19:38.582+0000 7f8258ace700 1 -- 172.21.15.59:0/2735168375 wait complete. 2024-05-08T05:19:39.186 DEBUG:teuthology.orchestra.run.smithi059:osd.0> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.0.service 2024-05-08T05:19:39.228 INFO:tasks.cephadm:Deploying osd.1 on smithi096 with /dev/vg_nvme/lv_4... 2024-05-08T05:19:39.229 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:19:39.475 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:19:39.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:39 smithi059 ceph-mon[30304]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:39.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.513 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:39 smithi059 ceph-mon[30304]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:19:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:39 smithi096 ceph-mon[34253]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:39 smithi096 ceph-mon[34253]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:19:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:39 smithi143 ceph-mon[34337]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:39.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:39 smithi143 ceph-mon[34337]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-05-08T05:19:40.880 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:40 smithi096 ceph-mon[34253]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:19:40.880 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:40 smithi096 ceph-mon[34253]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:19:40.880 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:40 smithi096 ceph-mon[34253]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:19:40.880 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:40 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:40.880 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:40 smithi096 ceph-mon[34253]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:40.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:40 smithi143 ceph-mon[34337]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:19:40.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:40 smithi143 ceph-mon[34337]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:19:40.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:40 smithi143 ceph-mon[34337]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:19:40.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:40 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:40.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:40 smithi143 ceph-mon[34337]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:41.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:40 smithi059 ceph-mon[30304]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-05-08T05:19:41.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:40 smithi059 ceph-mon[30304]: osdmap e6: 1 total, 0 up, 1 in 2024-05-08T05:19:41.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:40 smithi059 ceph-mon[30304]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:19:41.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:40 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:41.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:40 smithi059 ceph-mon[30304]: pgmap v44: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-05-08T05:19:41.088 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:19:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0[45258]: 2024-05-08T05:19:40.663+0000 7f5772282700 -1 osd.0 0 waiting for initial osdmap 2024-05-08T05:19:41.088 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:19:40 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0[45258]: 2024-05-08T05:19:40.666+0000 7f576e075700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:19:41.469 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:19:41.469 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:19:41.470 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-08T05:19:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-08T05:19:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.011421 s, 918 MB/s 2024-05-08T05:19:41.470 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-08T05:19:41.819 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:41 smithi096 ceph-mon[34253]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:19:41.819 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:41 smithi096 ceph-mon[34253]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:19:41.819 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:41 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:41.819 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:41 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:41.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:41 smithi059 ceph-mon[30304]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:19:41.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:41 smithi059 ceph-mon[30304]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:19:41.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:41 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:41.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:41 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:41.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:41 smithi143 ceph-mon[34337]: from='osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:19:41.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:41 smithi143 ceph-mon[34337]: osdmap e7: 1 total, 0 up, 1 in 2024-05-08T05:19:41.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:41 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:41.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:41 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:42.088 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_4 2024-05-08T05:19:42.331 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:19:42.755 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:19:42.755 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:19:42.755 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] boot 2024-05-08T05:19:42.755 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:19:42.755 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:42.755 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:42.755 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi059 2024-05-08T05:19:42.755 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:42.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:42.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:19:42.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi059 to 17093M 2024-05-08T05:19:42.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:42.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:42.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:42.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:42.756 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:42.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:19:42.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:19:42.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] boot 2024-05-08T05:19:42.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:19:42.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi059 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi059 to 17093M 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:42.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:43.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:19:43.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: osd.0 [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] boot 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: osdmap e8: 1 total, 1 up, 1 in 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi059 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi059 to 17093M 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:43.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:43.088 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:19:43.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.830+0000 7fccaf180700 1 -- 172.21.15.96:0/63061697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca81050d0 msgr2=0x7fcca810b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:43.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.830+0000 7fccaf180700 1 --2- 172.21.15.96:0/63061697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca81050d0 0x7fcca810b980 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fcca000b0d0 tx=0x7fcca000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:19:43.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.830+0000 7fccaf180700 1 -- 172.21.15.96:0/63061697 shutdown_connections 2024-05-08T05:19:43.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.830+0000 7fccaf180700 1 --2- 172.21.15.96:0/63061697 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca81050d0 0x7fcca810b980 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:43.833 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.830+0000 7fccaf180700 1 --2- 172.21.15.96:0/63061697 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca8103530 0x7fcca8103950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.830+0000 7fccaf180700 1 --2- 172.21.15.96:0/63061697 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcca8104730 0x7fcca8104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.830+0000 7fccaf180700 1 -- 172.21.15.96:0/63061697 >> 172.21.15.96:0/63061697 conn(0x7fcca80feaf0 msgr2=0x7fcca8100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.830+0000 7fccaf180700 1 -- 172.21.15.96:0/63061697 shutdown_connections 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.830+0000 7fccaf180700 1 -- 172.21.15.96:0/63061697 wait complete. 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccaf180700 1 Processor -- start 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccaf180700 1 -- start start 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccaf180700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca8103530 0x7fcca8199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccaf180700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca8104730 0x7fcca8199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccaf180700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcca81050d0 0x7fcca81a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccaf180700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810efd0 con 0x7fcca8103530 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccaf180700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810ee50 con 0x7fcca8104730 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccaf180700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcca810f150 con 0x7fcca81050d0 2024-05-08T05:19:43.834 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fcca7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca8104730 0x7fcca8199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fcca7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca8104730 0x7fcca8199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:58270/0 (socket says 172.21.15.96:58270) 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fcca7fff700 1 -- 172.21.15.96:0/266814623 learned_addr learned my addr 172.21.15.96:0/266814623 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccad71d700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcca81050d0 0x7fcca81a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccacf1c700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca8103530 0x7fcca8199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fcca7fff700 1 -- 172.21.15.96:0/266814623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcca81050d0 msgr2=0x7fcca81a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fcca7fff700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcca81050d0 0x7fcca81a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fcca7fff700 1 -- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca8103530 msgr2=0x7fcca8199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fcca7fff700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca8103530 0x7fcca8199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fcca7fff700 1 -- 172.21.15.96:0/266814623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcca81a1160 con 0x7fcca8104730 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccad71d700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcca81050d0 0x7fcca81a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.831+0000 7fccacf1c700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca8103530 0x7fcca8199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.832+0000 7fcca7fff700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca8104730 0x7fcca8199a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400d980 tx=0x7fcc9400dd40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.832+0000 7fcca5ffb700 1 -- 172.21.15.96:0/266814623 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94017070 con 0x7fcca8104730 2024-05-08T05:19:43.835 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.832+0000 7fcca5ffb700 1 -- 172.21.15.96:0/266814623 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc94004830 con 0x7fcca8104730 2024-05-08T05:19:43.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.832+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcca81a1450 con 0x7fcca8104730 2024-05-08T05:19:43.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.832+0000 7fcca5ffb700 1 -- 172.21.15.96:0/266814623 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc94005440 con 0x7fcca8104730 2024-05-08T05:19:43.836 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.832+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcca81a1940 con 0x7fcca8104730 2024-05-08T05:19:43.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.833+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcca806a190 con 0x7fcca8104730 2024-05-08T05:19:43.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.834+0000 7fcca5ffb700 1 -- 172.21.15.96:0/266814623 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fcc940055e0 con 0x7fcca8104730 2024-05-08T05:19:43.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.835+0000 7fcca5ffb700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc980b27f0 0x7fcc980b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:19:43.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.835+0000 7fccacf1c700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc980b27f0 0x7fcc980b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:19:43.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.835+0000 7fcca5ffb700 1 -- 172.21.15.96:0/266814623 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fcc940e06f0 con 0x7fcca8104730 2024-05-08T05:19:43.837 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.836+0000 7fccacf1c700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc980b27f0 0x7fcc980b4cb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c005fd0 tx=0x7fcc9c005e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:19:43.841 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:43.840+0000 7fcca5ffb700 1 -- 172.21.15.96:0/266814623 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc94020020 con 0x7fcca8104730 2024-05-08T05:19:43.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:43 smithi143 ceph-mon[34337]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:19:43.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:43 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:43.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:43 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:43.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:43 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:44.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:43 smithi096 ceph-mon[34253]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:19:44.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:43 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:44.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:43 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:44.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:43 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:44.053 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:19:44.052+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7fcca80638e0 con 0x7fcc980b27f0 2024-05-08T05:19:44.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:43 smithi059 ceph-mon[30304]: osdmap e9: 1 total, 1 up, 1 in 2024-05-08T05:19:44.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:43 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:44.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:43 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:19:44.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:43 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:19:44.915 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:44 smithi096 ceph-mon[34253]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:44.915 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:44 smithi096 ceph-mon[34253]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:19:44.915 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:19:44.915 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:19:44.915 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:44.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:44 smithi143 ceph-mon[34337]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:44.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:44 smithi143 ceph-mon[34337]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:19:44.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:19:44.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:19:44.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:45.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:44 smithi059 ceph-mon[30304]: pgmap v49: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:45.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:44 smithi059 ceph-mon[30304]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:19:45.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:19:45.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:19:45.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:47.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:46 smithi143 ceph-mon[34337]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:47.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:46 smithi143 ceph-mon[34337]: from='client.? 172.21.15.96:0/2177417107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "855aa3c7-8818-419d-8a7d-078e3bd10535"}]: dispatch 2024-05-08T05:19:47.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:46 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "855aa3c7-8818-419d-8a7d-078e3bd10535"}]: dispatch 2024-05-08T05:19:47.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:46 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "855aa3c7-8818-419d-8a7d-078e3bd10535"}]': finished 2024-05-08T05:19:47.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:46 smithi143 ceph-mon[34337]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:19:47.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:46 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:19:47.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:46 smithi096 ceph-mon[34253]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:47.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:46 smithi096 ceph-mon[34253]: from='client.? 172.21.15.96:0/2177417107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "855aa3c7-8818-419d-8a7d-078e3bd10535"}]: dispatch 2024-05-08T05:19:47.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:46 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "855aa3c7-8818-419d-8a7d-078e3bd10535"}]: dispatch 2024-05-08T05:19:47.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:46 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "855aa3c7-8818-419d-8a7d-078e3bd10535"}]': finished 2024-05-08T05:19:47.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:46 smithi096 ceph-mon[34253]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:19:47.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:46 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:19:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:46 smithi059 ceph-mon[30304]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:46 smithi059 ceph-mon[30304]: from='client.? 172.21.15.96:0/2177417107' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "855aa3c7-8818-419d-8a7d-078e3bd10535"}]: dispatch 2024-05-08T05:19:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:46 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "855aa3c7-8818-419d-8a7d-078e3bd10535"}]: dispatch 2024-05-08T05:19:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:46 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "855aa3c7-8818-419d-8a7d-078e3bd10535"}]': finished 2024-05-08T05:19:47.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:46 smithi059 ceph-mon[30304]: osdmap e10: 2 total, 1 up, 2 in 2024-05-08T05:19:47.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:46 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:19:48.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:47 smithi143 ceph-mon[34337]: from='client.? 172.21.15.96:0/2008720838' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:19:48.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:47 smithi096 ceph-mon[34253]: from='client.? 172.21.15.96:0/2008720838' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:19:48.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:47 smithi059 ceph-mon[30304]: from='client.? 172.21.15.96:0/2008720838' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:19:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:48 smithi143 ceph-mon[34337]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:48 smithi096 ceph-mon[34253]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:49.321 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:48 smithi059 ceph-mon[30304]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:51.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:50 smithi143 ceph-mon[34337]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:51.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:50 smithi096 ceph-mon[34253]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:51.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:50 smithi059 ceph-mon[30304]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:53.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:52 smithi143 ceph-mon[34337]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:53.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:52 smithi096 ceph-mon[34253]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:53.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:52 smithi059 ceph-mon[30304]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:54.237 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:19:54.238 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:54.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:19:54.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:54.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-05-08T05:19:54.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:19:55.237 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:54 smithi096 ceph-mon[34253]: Deploying daemon osd.1 on smithi096 2024-05-08T05:19:55.237 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:54 smithi096 ceph-mon[34253]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:55.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:54 smithi143 ceph-mon[34337]: Deploying daemon osd.1 on smithi096 2024-05-08T05:19:55.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:54 smithi143 ceph-mon[34337]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:55.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:54 smithi059 ceph-mon[30304]: Deploying daemon osd.1 on smithi096 2024-05-08T05:19:55.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:54 smithi059 ceph-mon[30304]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:57.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:56 smithi143 ceph-mon[34337]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:57.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:56 smithi096 ceph-mon[34253]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:57.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:56 smithi059 ceph-mon[30304]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:59.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:19:58 smithi143 ceph-mon[34337]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:59.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:19:58 smithi096 ceph-mon[34253]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:19:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:19:58 smithi059 ceph-mon[30304]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:00.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:00 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:00.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:00 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:00.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:00 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:00.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:00 smithi096 ceph-mon[34253]: overall HEALTH_OK 2024-05-08T05:20:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:00 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:00 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:00 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:00 smithi143 ceph-mon[34337]: overall HEALTH_OK 2024-05-08T05:20:00.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:00 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:00.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:00 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:00.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:00 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:00.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:00 smithi059 ceph-mon[30304]: overall HEALTH_OK 2024-05-08T05:20:01.125 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:01 smithi096 ceph-mon[34253]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:01 smithi143 ceph-mon[34337]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:01.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:01 smithi059 ceph-mon[30304]: pgmap v58: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:02.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:02.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:02.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:02.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:02.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:02.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:02.620 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.619+0000 7fcca5ffb700 1 -- 172.21.15.96:0/266814623 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fcca80638e0 con 0x7fcc980b27f0 2024-05-08T05:20:02.620 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 1 on host 'smithi096' 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.621+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc980b27f0 msgr2=0x7fcc980b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.621+0000 7fccaf180700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc980b27f0 0x7fcc980b4cb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fcc9c005fd0 tx=0x7fcc9c005e40 comp rx=0 tx=0).stop 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca8104730 msgr2=0x7fcca8199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca8104730 0x7fcca8199a40 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fcc9400d980 tx=0x7fcc9400dd40 comp rx=0 tx=0).stop 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 shutdown_connections 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcca81050d0 0x7fcca81a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcca8103530 0x7fcca8199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc980b27f0 0x7fcc980b4cb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 --2- 172.21.15.96:0/266814623 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcca8104730 0x7fcca8199a40 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 >> 172.21.15.96:0/266814623 conn(0x7fcca80feaf0 msgr2=0x7fcca81079d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 shutdown_connections 2024-05-08T05:20:02.623 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:02.622+0000 7fccaf180700 1 -- 172.21.15.96:0/266814623 wait complete. 2024-05-08T05:20:02.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:02.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:02.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:02.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:02.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:03.271 DEBUG:teuthology.orchestra.run.smithi096:osd.1> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.1.service 2024-05-08T05:20:03.273 INFO:tasks.cephadm:Deploying osd.2 on smithi143 with /dev/vg_nvme/lv_4... 2024-05-08T05:20:03.273 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-05-08T05:20:03.521 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:20:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:03 smithi096 ceph-mon[34253]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:03.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:03 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:03.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:03 smithi096 ceph-mon[34253]: from='osd.1 [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:20:03.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:03 smithi096 ceph-mon[34253]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:20:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:03 smithi059 ceph-mon[30304]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:03.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:03 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:03.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:03 smithi059 ceph-mon[30304]: from='osd.1 [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:20:03.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:03 smithi059 ceph-mon[30304]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:20:03.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:03 smithi143 ceph-mon[34337]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:03.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:03.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:03.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:03 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:03.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:03 smithi143 ceph-mon[34337]: from='osd.1 [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:20:03.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:03 smithi143 ceph-mon[34337]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-05-08T05:20:04.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:04 smithi059 ceph-mon[30304]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:20:04.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:04 smithi059 ceph-mon[30304]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:20:04.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:04 smithi059 ceph-mon[30304]: from='osd.1 [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:20:04.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:04.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:04 smithi059 ceph-mon[30304]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:20:04.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:04 smithi059 ceph-mon[30304]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:04.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:04 smithi143 ceph-mon[34337]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:20:04.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:04 smithi143 ceph-mon[34337]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:20:04.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:04 smithi143 ceph-mon[34337]: from='osd.1 [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:20:04.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:04.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:04 smithi143 ceph-mon[34337]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:20:04.840 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:04 smithi143 ceph-mon[34337]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:05.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:04 smithi096 ceph-mon[34253]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-05-08T05:20:05.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:04 smithi096 ceph-mon[34253]: osdmap e11: 2 total, 1 up, 2 in 2024-05-08T05:20:05.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:04 smithi096 ceph-mon[34253]: from='osd.1 [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:20:05.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:05.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:04 smithi096 ceph-mon[34253]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:20:05.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:04 smithi096 ceph-mon[34253]: pgmap v61: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:05.031 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:04 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1[38851]: 2024-05-08T05:20:04.583+0000 7f8930c10700 -1 osd.1 0 waiting for initial osdmap 2024-05-08T05:20:05.031 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:04 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1[38851]: 2024-05-08T05:20:04.589+0000 7f892ca03700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:20:05.456 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-05-08T05:20:05.456 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-05-08T05:20:05.456 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-05-08T05:20:05.457 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-05-08T05:20:05.457 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0113715 s, 922 MB/s 2024-05-08T05:20:05.457 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-05-08T05:20:05.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:05 smithi096 ceph-mon[34253]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:20:05.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:05 smithi096 ceph-mon[34253]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:20:05.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:05.729 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:05.992 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:05 smithi143 ceph-mon[34337]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:20:05.992 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:05 smithi143 ceph-mon[34337]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:20:05.992 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:05.992 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:06.023 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_4 2024-05-08T05:20:06.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:05 smithi059 ceph-mon[30304]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:20:06.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:05 smithi059 ceph-mon[30304]: osdmap e12: 2 total, 1 up, 2 in 2024-05-08T05:20:06.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:06.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:06.270 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:20:06.642 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:20:06.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:20:06.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: osd.1 [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] boot 2024-05-08T05:20:06.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:20:06.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:06.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:06.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi096 2024-05-08T05:20:06.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:06.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:06.643 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:06.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi096 to 17088M 2024-05-08T05:20:06.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:06.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:06.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:06.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:20:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:20:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: osd.1 [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] boot 2024-05-08T05:20:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:20:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi096 2024-05-08T05:20:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:07.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:07.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:07.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi096 to 17088M 2024-05-08T05:20:07.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:07.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:07.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:07.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:07.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: osd.1 [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] boot 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: osdmap e13: 2 total, 2 up, 2 in 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: pgmap v64: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi096 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi096 to 17088M 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:07.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:07.613 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:07 smithi143 ceph-mon[34337]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:20:07.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.815+0000 7f77c330c700 1 -- 172.21.15.143:0/2887310792 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 msgr2=0x7f77bc103950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:07.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.815+0000 7f77c330c700 1 --2- 172.21.15.143:0/2887310792 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 0x7f77bc103950 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f77b8009990 tx=0x7f77b8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:20:07.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.815+0000 7f77c330c700 1 -- 172.21.15.143:0/2887310792 shutdown_connections 2024-05-08T05:20:07.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.815+0000 7f77c330c700 1 --2- 172.21.15.143:0/2887310792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77bc1050d0 0x7f77bc10b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:07.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.815+0000 7f77c330c700 1 --2- 172.21.15.143:0/2887310792 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 0x7f77bc103950 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:07.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.815+0000 7f77c330c700 1 --2- 172.21.15.143:0/2887310792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77bc104730 0x7f77bc104b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:07.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.815+0000 7f77c330c700 1 -- 172.21.15.143:0/2887310792 >> 172.21.15.143:0/2887310792 conn(0x7f77bc0feaf0 msgr2=0x7f77bc100f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:20:07.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.815+0000 7f77c330c700 1 -- 172.21.15.143:0/2887310792 shutdown_connections 2024-05-08T05:20:07.816 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.815+0000 7f77c330c700 1 -- 172.21.15.143:0/2887310792 wait complete. 2024-05-08T05:20:07.818 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c330c700 1 Processor -- start 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c330c700 1 -- start start 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c330c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 0x7f77bc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c330c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77bc104730 0x7f77bc199c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c330c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77bc1050d0 0x7f77bc1a0ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c330c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc10f0a0 con 0x7f77bc104730 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c330c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc10ef20 con 0x7f77bc1050d0 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c330c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f77bc10f220 con 0x7f77bc103530 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c10a8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 0x7f77bc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c10a8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 0x7f77bc199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:41106/0 (socket says 172.21.15.143:41106) 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c10a8700 1 -- 172.21.15.143:0/427548329 learned_addr learned my addr 172.21.15.143:0/427548329 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:20:07.819 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c18a9700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77bc1050d0 0x7f77bc1a0ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c08a7700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77bc104730 0x7f77bc199c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c10a8700 1 -- 172.21.15.143:0/427548329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77bc1050d0 msgr2=0x7f77bc1a0ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c10a8700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77bc1050d0 0x7f77bc1a0ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c10a8700 1 -- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77bc104730 msgr2=0x7f77bc199c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c10a8700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77bc104730 0x7f77bc199c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c10a8700 1 -- 172.21.15.143:0/427548329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77bc1a1380 con 0x7f77bc103530 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c18a9700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77bc1050d0 0x7f77bc1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77c10a8700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 0x7f77bc199720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77b80047b0 tx=0x7f77b8003940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.816+0000 7f77b27fc700 1 -- 172.21.15.143:0/427548329 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8027070 con 0x7f77bc103530 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.817+0000 7f77b27fc700 1 -- 172.21.15.143:0/427548329 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77b8022470 con 0x7f77bc103530 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.817+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77bc0756d0 con 0x7f77bc103530 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.817+0000 7f77b27fc700 1 -- 172.21.15.143:0/427548329 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b800f6c0 con 0x7f77bc103530 2024-05-08T05:20:07.820 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.817+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77bc075c30 con 0x7f77bc103530 2024-05-08T05:20:07.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.817+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77bc06a190 con 0x7f77bc103530 2024-05-08T05:20:07.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.819+0000 7f77b27fc700 1 -- 172.21.15.143:0/427548329 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7f77b802a070 con 0x7f77bc103530 2024-05-08T05:20:07.821 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.819+0000 7f77b27fc700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77a80b27f0 0x7f77a80b4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:07.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.819+0000 7f77b27fc700 1 -- 172.21.15.143:0/427548329 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f77b80e1e30 con 0x7f77bc103530 2024-05-08T05:20:07.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.820+0000 7f77c08a7700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77a80b27f0 0x7f77a80b4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:07.822 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.820+0000 7f77c08a7700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77a80b27f0 0x7f77a80b4cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77ac006fd0 tx=0x7f77ac008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:20:07.824 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:07.823+0000 7f77b27fc700 1 -- 172.21.15.143:0/427548329 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f77b8013070 con 0x7f77bc103530 2024-05-08T05:20:08.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:07 smithi096 ceph-mon[34253]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:20:08.036 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:08.035+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f77bc0638e0 con 0x7f77a80b27f0 2024-05-08T05:20:08.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:07 smithi059 ceph-mon[30304]: osdmap e14: 2 total, 2 up, 2 in 2024-05-08T05:20:08.853 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:08 smithi143 ceph-mon[34337]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:08.853 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:08 smithi143 ceph-mon[34337]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:20:08.853 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:20:08.853 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:20:08.854 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:09.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:08 smithi096 ceph-mon[34253]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:09.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:08 smithi096 ceph-mon[34253]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:20:09.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:20:09.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:20:09.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:09.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:08 smithi059 ceph-mon[30304]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:09.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:08 smithi059 ceph-mon[30304]: from='client.24163 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:20:09.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:20:09.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:20:09.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:11.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:10 smithi143 ceph-mon[34337]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:11.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:10 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/1626795556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05037ce-9958-4ac3-9377-9a6bef15510d"}]: dispatch 2024-05-08T05:20:11.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:10 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05037ce-9958-4ac3-9377-9a6bef15510d"}]: dispatch 2024-05-08T05:20:11.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:10 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a05037ce-9958-4ac3-9377-9a6bef15510d"}]': finished 2024-05-08T05:20:11.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:10 smithi143 ceph-mon[34337]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:20:11.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:10 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:11.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:10 smithi096 ceph-mon[34253]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:11.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:10 smithi096 ceph-mon[34253]: from='client.? 172.21.15.143:0/1626795556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05037ce-9958-4ac3-9377-9a6bef15510d"}]: dispatch 2024-05-08T05:20:11.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:10 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05037ce-9958-4ac3-9377-9a6bef15510d"}]: dispatch 2024-05-08T05:20:11.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:10 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a05037ce-9958-4ac3-9377-9a6bef15510d"}]': finished 2024-05-08T05:20:11.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:10 smithi096 ceph-mon[34253]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:20:11.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:10 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:11.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:10 smithi059 ceph-mon[30304]: pgmap v67: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:11.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:10 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/1626795556' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05037ce-9958-4ac3-9377-9a6bef15510d"}]: dispatch 2024-05-08T05:20:11.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:10 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "a05037ce-9958-4ac3-9377-9a6bef15510d"}]: dispatch 2024-05-08T05:20:11.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:10 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "a05037ce-9958-4ac3-9377-9a6bef15510d"}]': finished 2024-05-08T05:20:11.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:10 smithi059 ceph-mon[30304]: osdmap e15: 3 total, 2 up, 3 in 2024-05-08T05:20:11.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:10 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:12.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:11 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/2886812720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:20:12.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:11 smithi096 ceph-mon[34253]: from='client.? 172.21.15.143:0/2886812720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:20:12.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:11 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/2886812720' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:20:13.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:12 smithi143 ceph-mon[34337]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:13.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:12 smithi096 ceph-mon[34253]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:12 smithi059 ceph-mon[30304]: pgmap v69: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:15.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:14 smithi143 ceph-mon[34337]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:15.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:14 smithi096 ceph-mon[34253]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:15.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:14 smithi059 ceph-mon[30304]: pgmap v70: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:17.182 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:16 smithi143 ceph-mon[34337]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:16 smithi096 ceph-mon[34253]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:16 smithi059 ceph-mon[30304]: pgmap v71: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:18.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:17 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T05:20:18.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:17 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:18.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:17 smithi143 ceph-mon[34337]: Deploying daemon osd.2 on smithi143 2024-05-08T05:20:18.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:17 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T05:20:18.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:17 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:18.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:17 smithi096 ceph-mon[34253]: Deploying daemon osd.2 on smithi143 2024-05-08T05:20:18.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-05-08T05:20:18.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:18.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:17 smithi059 ceph-mon[30304]: Deploying daemon osd.2 on smithi143 2024-05-08T05:20:19.200 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:18 smithi143 ceph-mon[34337]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:19.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:18 smithi096 ceph-mon[34253]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:18 smithi059 ceph-mon[30304]: pgmap v72: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:21.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:20 smithi143 ceph-mon[34337]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:21.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:20 smithi096 ceph-mon[34253]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:20 smithi059 ceph-mon[30304]: pgmap v73: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:23.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:22 smithi143 ceph-mon[34337]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:23.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:23.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:23.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:23.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:22 smithi096 ceph-mon[34253]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:23.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:23.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:23.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:22 smithi059 ceph-mon[30304]: pgmap v74: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:25.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:24 smithi143 ceph-mon[34337]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:25.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:24 smithi096 ceph-mon[34253]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:24 smithi059 ceph-mon[30304]: pgmap v75: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:26.102 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.100+0000 7f77b27fc700 1 -- 172.21.15.143:0/427548329 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f77bc0638e0 con 0x7f77a80b27f0 2024-05-08T05:20:26.102 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 2 on host 'smithi143' 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77a80b27f0 msgr2=0x7f77a80b4cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77a80b27f0 0x7f77a80b4cb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f77ac006fd0 tx=0x7f77ac008040 comp rx=0 tx=0).stop 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 msgr2=0x7f77bc199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 0x7f77bc199720 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f77b80047b0 tx=0x7f77b8003940 comp rx=0 tx=0).stop 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 shutdown_connections 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f77a80b27f0 0x7f77a80b4cb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f77bc1050d0 0x7f77bc1a0ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f77bc103530 0x7f77bc199720 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 --2- 172.21.15.143:0/427548329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f77bc104730 0x7f77bc199c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 >> 172.21.15.143:0/427548329 conn(0x7f77bc0feaf0 msgr2=0x7f77bc107aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 shutdown_connections 2024-05-08T05:20:26.104 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:20:26.103+0000 7f77c330c700 1 -- 172.21.15.143:0/427548329 wait complete. 2024-05-08T05:20:26.714 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:26 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.714 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:26 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.714 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:26 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:26.714 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:26 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:26.714 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:26 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.714 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:26 smithi143 ceph-mon[34337]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:26.714 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:26 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.714 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:26 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.714 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:26 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:26.741 DEBUG:teuthology.orchestra.run.smithi143:osd.2> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.2.service 2024-05-08T05:20:26.743 INFO:tasks.cephadm:Deploying osd.3 on smithi059 with /dev/vg_nvme/lv_3... 2024-05-08T05:20:26.743 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T05:20:26.768 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:26 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.768 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:26 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:26 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:26.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:26 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:26.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:26 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:26 smithi059 ceph-mon[30304]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:26.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:26 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:26 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:26.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:26 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:26.990 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:20:27.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:26 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:27.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:26 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:27.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:26 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:27.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:26 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:27.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:26 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:27.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:26 smithi096 ceph-mon[34253]: pgmap v76: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:27.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:26 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:27.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:26 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:27.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:26 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:27 smithi059 ceph-mon[30304]: from='osd.2 [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:20:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:27 smithi059 ceph-mon[30304]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:20:27.900 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:27 smithi143 ceph-mon[34337]: from='osd.2 [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:20:27.900 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:27 smithi143 ceph-mon[34337]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:20:28.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:27 smithi096 ceph-mon[34253]: from='osd.2 [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:20:28.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:27 smithi096 ceph-mon[34253]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-05-08T05:20:28.937 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:28 smithi143 ceph-mon[34337]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T05:20:28.937 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:28 smithi143 ceph-mon[34337]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T05:20:28.937 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:28 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:28.938 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:28 smithi143 ceph-mon[34337]: from='osd.2 [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:20:28.938 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:28 smithi143 ceph-mon[34337]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:20:28.938 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:28 smithi143 ceph-mon[34337]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:28.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:28 smithi059 ceph-mon[30304]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T05:20:28.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:28 smithi059 ceph-mon[30304]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T05:20:28.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:28 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:28.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:28 smithi059 ceph-mon[30304]: from='osd.2 [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:20:28.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:28 smithi059 ceph-mon[30304]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:20:28.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:28 smithi059 ceph-mon[30304]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:28.960 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T05:20:28.961 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T05:20:28.961 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-08T05:20:28.961 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-08T05:20:28.961 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0112561 s, 932 MB/s 2024-05-08T05:20:28.961 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-08T05:20:29.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:28 smithi096 ceph-mon[34253]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-05-08T05:20:29.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:28 smithi096 ceph-mon[34253]: osdmap e16: 3 total, 2 up, 3 in 2024-05-08T05:20:29.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:28 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:29.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:28 smithi096 ceph-mon[34253]: from='osd.2 [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:20:29.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:28 smithi096 ceph-mon[34253]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:20:29.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:28 smithi096 ceph-mon[34253]: pgmap v78: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:29.243 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2[39488]: 2024-05-08T05:20:29.104+0000 7f7d0ebbf700 -1 osd.2 0 waiting for initial osdmap 2024-05-08T05:20:29.244 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2[39488]: 2024-05-08T05:20:29.108+0000 7f7d069aa700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:20:29.675 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_3 2024-05-08T05:20:29.967 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:20:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:20:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T05:20:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='osd.2 ' entity='osd.2' 2024-05-08T05:20:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='osd.2 ' entity='osd.2' 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:30.073 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: osdmap e17: 3 total, 2 up, 3 in 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='osd.2 ' entity='osd.2' 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:30.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:30.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:30 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:20:30.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:30 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:20:30.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:30 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi143 2024-05-08T05:20:30.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:30 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi143 to 17088M 2024-05-08T05:20:30.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:30.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:30 smithi059 ceph-mon[30304]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:30.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:30 smithi059 ceph-mon[30304]: osd.2 [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] boot 2024-05-08T05:20:30.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:30 smithi059 ceph-mon[30304]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T05:20:30.963 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:30 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:20:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:30 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:20:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:30 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi143 2024-05-08T05:20:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:30 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi143 to 17088M 2024-05-08T05:20:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:30 smithi143 ceph-mon[34337]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:30 smithi143 ceph-mon[34337]: osd.2 [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] boot 2024-05-08T05:20:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:30 smithi143 ceph-mon[34337]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T05:20:30.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:31.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:30 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:20:31.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:30 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:20:31.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:30 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi143 2024-05-08T05:20:31.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:30 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi143 to 17088M 2024-05-08T05:20:31.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:31.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:30 smithi096 ceph-mon[34253]: pgmap v80: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-05-08T05:20:31.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:30 smithi096 ceph-mon[34253]: osd.2 [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] boot 2024-05-08T05:20:31.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:30 smithi096 ceph-mon[34253]: osdmap e18: 3 total, 3 up, 3 in 2024-05-08T05:20:31.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:20:31.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.472+0000 7fc22cd64700 1 -- 172.21.15.59:0/2051522993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228103500 msgr2=0x7fc228103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:31.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.473+0000 7fc22cd64700 1 --2- 172.21.15.59:0/2051522993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228103500 0x7fc228103920 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fc2180099c0 tx=0x7fc218009cd0 comp rx=0 tx=0).stop 2024-05-08T05:20:31.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.473+0000 7fc22cd64700 1 -- 172.21.15.59:0/2051522993 shutdown_connections 2024-05-08T05:20:31.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.473+0000 7fc22cd64700 1 --2- 172.21.15.59:0/2051522993 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2281050a0 0x7fc22810b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:31.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.473+0000 7fc22cd64700 1 --2- 172.21.15.59:0/2051522993 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc228103500 0x7fc228103920 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:31.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.473+0000 7fc22cd64700 1 --2- 172.21.15.59:0/2051522993 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc228104700 0x7fc228104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.473+0000 7fc22cd64700 1 -- 172.21.15.59:0/2051522993 >> 172.21.15.59:0/2051522993 conn(0x7fc2280feaa0 msgr2=0x7fc228100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.473+0000 7fc22cd64700 1 -- 172.21.15.59:0/2051522993 shutdown_connections 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.473+0000 7fc22cd64700 1 -- 172.21.15.59:0/2051522993 wait complete. 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.473+0000 7fc22cd64700 1 Processor -- start 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22cd64700 1 -- start start 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22cd64700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc228103500 0x7fc228199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22cd64700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc228104700 0x7fc228199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22cd64700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281050a0 0x7fc2281a0d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22cd64700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc22810f070 con 0x7fc2281050a0 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22cd64700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc22810eef0 con 0x7fc228103500 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22cd64700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc22810f1f0 con 0x7fc228104700 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc226d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281050a0 0x7fc2281a0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:31.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc228103500 0x7fc228199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22659c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc228103500 0x7fc228199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:58542/0 (socket says 172.21.15.59:58542) 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc226d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281050a0 0x7fc2281a0d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42154/0 (socket says 172.21.15.59:42154) 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22659c700 1 -- 172.21.15.59:0/1251235576 learned_addr learned my addr 172.21.15.59:0/1251235576 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc225d9b700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc228104700 0x7fc228199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc226d9d700 1 -- 172.21.15.59:0/1251235576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc228104700 msgr2=0x7fc228199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc226d9d700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc228104700 0x7fc228199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc226d9d700 1 -- 172.21.15.59:0/1251235576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc228103500 msgr2=0x7fc228199740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc226d9d700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc228103500 0x7fc228199740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc226d9d700 1 -- 172.21.15.59:0/1251235576 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2281a13a0 con 0x7fc2281050a0 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.474+0000 7fc22659c700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc228103500 0x7fc228199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.475+0000 7fc226d9d700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281050a0 0x7fc2281a0d00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc22000be50 tx=0x7fc220009ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.475+0000 7fc2177fe700 1 -- 172.21.15.59:0/1251235576 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc220019070 con 0x7fc2281050a0 2024-05-08T05:20:31.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.475+0000 7fc2177fe700 1 -- 172.21.15.59:0/1251235576 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc22000cb90 con 0x7fc2281050a0 2024-05-08T05:20:31.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.475+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2281a1690 con 0x7fc2281050a0 2024-05-08T05:20:31.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.475+0000 7fc2177fe700 1 -- 172.21.15.59:0/1251235576 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2200125a0 con 0x7fc2281050a0 2024-05-08T05:20:31.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.475+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2281a1ca0 con 0x7fc2281050a0 2024-05-08T05:20:31.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.476+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc22806a190 con 0x7fc2281050a0 2024-05-08T05:20:31.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.477+0000 7fc2177fe700 1 -- 172.21.15.59:0/1251235576 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 15) ==== 149596+0+0 (secure 0 0 0) 0x7fc22001c070 con 0x7fc2281050a0 2024-05-08T05:20:31.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.478+0000 7fc2177fe700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2100b2870 0x7fc2100b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:31.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.478+0000 7fc22659c700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2100b2870 0x7fc2100b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:31.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.478+0000 7fc2177fe700 1 -- 172.21.15.59:0/1251235576 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2851+0+0 (secure 0 0 0) 0x7fc2200e0890 con 0x7fc2281050a0 2024-05-08T05:20:31.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.479+0000 7fc22659c700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2100b2870 0x7fc2100b4d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc2180095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:20:31.482 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.482+0000 7fc2177fe700 1 -- 172.21.15.59:0/1251235576 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc220023070 con 0x7fc2281050a0 2024-05-08T05:20:31.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:31.707+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc2280638e0 con 0x7fc2100b2870 2024-05-08T05:20:32.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:32 smithi059 ceph-mon[30304]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T05:20:32.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:32 smithi059 ceph-mon[30304]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:20:32.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:32 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:20:32.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:32 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:20:32.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:32 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:32.905 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:32 smithi059 ceph-mon[30304]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:32.906 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:32 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:20:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:32 smithi143 ceph-mon[34337]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T05:20:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:32 smithi143 ceph-mon[34337]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:20:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:32 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:20:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:32 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:20:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:32 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:32 smithi143 ceph-mon[34337]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:32 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:20:33.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:32 smithi096 ceph-mon[34253]: osdmap e19: 3 total, 3 up, 3 in 2024-05-08T05:20:33.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:32 smithi096 ceph-mon[34253]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:20:33.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:32 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:20:33.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:32 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:20:33.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:32 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:33.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:32 smithi096 ceph-mon[34253]: pgmap v83: 0 pgs: ; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:33.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:32 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:20:33.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T05:20:33.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 ceph-mon[30304]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T05:20:33.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:20:33.931 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49963]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:20:33.931 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49963]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:33.931 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49963]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:33.931 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49963]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:33.931 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49966]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:20:33.931 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49966]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:33.931 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49966]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:33.931 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49966]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:33.971 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T05:20:33.971 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 ceph-mon[34337]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T05:20:33.971 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:20:33.971 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43538]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:20:33.971 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43538]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:33.971 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43538]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:33.971 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43538]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:33.971 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43541]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:20:33.971 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43541]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:33.971 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43541]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:33.971 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43541]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:34.008 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-05-08T05:20:34.008 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:33 smithi096 ceph-mon[34253]: osdmap e20: 3 total, 3 up, 3 in 2024-05-08T05:20:34.008 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:20:34.008 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:33 smithi096 sudo[42821]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:20:34.008 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:33 smithi096 sudo[42821]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:34.009 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:33 smithi096 sudo[42821]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:34.009 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:33 smithi096 sudo[42821]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:34.009 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:33 smithi096 sudo[42824]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:20:34.009 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:33 smithi096 sudo[42824]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:34.009 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:33 smithi096 sudo[42824]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:34.009 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:20:33 smithi096 sudo[42824]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:34.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43544]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:20:34.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43544]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:34.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43544]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:34.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43544]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:34.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43547]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:20:34.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43547]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:34.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43547]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:34.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:33 smithi143 sudo[43547]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:34.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 sudo[42827]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:20:34.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 sudo[42827]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:34.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 sudo[42827]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:34.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 sudo[42827]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:34.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 sudo[42830]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:20:34.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 sudo[42830]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:34.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 sudo[42830]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:34.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 sudo[42830]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:34.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49971]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-05-08T05:20:34.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49971]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:34.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49971]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:34.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49971]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:34.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49977]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-05-08T05:20:34.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49977]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-05-08T05:20:34.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49977]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-05-08T05:20:34.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:33 smithi059 sudo[49977]: pam_unix(sudo:session): session closed for user root 2024-05-08T05:20:34.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:34.928+0000 7fc2177fe700 1 -- 172.21.15.59:0/1251235576 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc2200a5800 con 0x7fc2281050a0 2024-05-08T05:20:34.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:20:34.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T05:20:34.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:34.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:34.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:34.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:34.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:20:34.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:20:34.930 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/4000919859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40198b3b-3215-4d59-9a89-4c18fa5271b1"}]: dispatch 2024-05-08T05:20:34.931 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/4000919859' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40198b3b-3215-4d59-9a89-4c18fa5271b1"}]': finished 2024-05-08T05:20:34.931 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T05:20:34.931 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:20:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/4000919859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40198b3b-3215-4d59-9a89-4c18fa5271b1"}]: dispatch 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/4000919859' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40198b3b-3215-4d59-9a89-4c18fa5271b1"}]': finished 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T05:20:34.995 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:35.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:20:35.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: osdmap e21: 3 total, 3 up, 3 in 2024-05-08T05:20:35.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:35.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:35.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: pgmap v86: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:35.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/4000919859' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "40198b3b-3215-4d59-9a89-4c18fa5271b1"}]: dispatch 2024-05-08T05:20:35.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/4000919859' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "40198b3b-3215-4d59-9a89-4c18fa5271b1"}]': finished 2024-05-08T05:20:35.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: osdmap e22: 4 total, 3 up, 4 in 2024-05-08T05:20:35.032 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:35 smithi143 ceph-mon[34337]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T05:20:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:35 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1300165572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:20:36.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:35 smithi096 ceph-mon[34253]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T05:20:36.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:35 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1300165572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:20:36.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:35 smithi059 ceph-mon[30304]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-05-08T05:20:36.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:35 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1300165572' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:20:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:36 smithi143 ceph-mon[34337]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:37.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:36 smithi096 ceph-mon[34253]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:37.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:36 smithi059 ceph-mon[30304]: pgmap v88: 1 pgs: 1 unknown; 0 B data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:39.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:38 smithi143 ceph-mon[34337]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:39.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:38 smithi096 ceph-mon[34253]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:39.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:38 smithi059 ceph-mon[30304]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:41.074 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:40 smithi059 ceph-mon[30304]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:40 smithi143 ceph-mon[34337]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:41.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:40 smithi096 ceph-mon[34253]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:43.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:42 smithi059 ceph-mon[30304]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:43.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T05:20:43.065 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:43.066 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:42 smithi059 ceph-mon[30304]: Deploying daemon osd.3 on smithi059 2024-05-08T05:20:43.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:42 smithi143 ceph-mon[34337]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:43.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T05:20:43.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:43.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:42 smithi143 ceph-mon[34337]: Deploying daemon osd.3 on smithi059 2024-05-08T05:20:43.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:42 smithi096 ceph-mon[34253]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-05-08T05:20:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:42 smithi096 ceph-mon[34253]: Deploying daemon osd.3 on smithi059 2024-05-08T05:20:45.218 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:44 smithi059 ceph-mon[30304]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:45.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:44 smithi143 ceph-mon[34337]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:45.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:44 smithi096 ceph-mon[34253]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:47.034 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:46 smithi059 ceph-mon[30304]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:47.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:46 smithi143 ceph-mon[34337]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:47.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:46 smithi096 ceph-mon[34253]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:48 smithi143 ceph-mon[34337]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:49.278 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:48 smithi059 ceph-mon[30304]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:49.278 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:49.278 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:49.278 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:48 smithi096 ceph-mon[34253]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:51.250 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:50 smithi059 ceph-mon[30304]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:51.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:50 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:51.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:50 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:51.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:50 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:51.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:50 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:51.251 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:50 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:51.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:50 smithi096 ceph-mon[34253]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:51.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:50 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:51.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:50 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:51.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:50 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:51.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:50 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:51.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:50 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:50 smithi143 ceph-mon[34337]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:50 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:50 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:50 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:50 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:50 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:51.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.586+0000 7fc2177fe700 1 -- 172.21.15.59:0/1251235576 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc2280638e0 con 0x7fc2100b2870 2024-05-08T05:20:51.588 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 3 on host 'smithi059' 2024-05-08T05:20:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2100b2870 msgr2=0x7fc2100b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2100b2870 0x7fc2100b4d30 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fc218000c00 tx=0x7fc2180095f0 comp rx=0 tx=0).stop 2024-05-08T05:20:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281050a0 msgr2=0x7fc2281a0d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:51.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281050a0 0x7fc2281a0d00 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fc22000be50 tx=0x7fc220009ed0 comp rx=0 tx=0).stop 2024-05-08T05:20:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 shutdown_connections 2024-05-08T05:20:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2281050a0 0x7fc2281a0d00 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2100b2870 0x7fc2100b4d30 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc228103500 0x7fc228199740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 --2- 172.21.15.59:0/1251235576 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc228104700 0x7fc228199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:51.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 >> 172.21.15.59:0/1251235576 conn(0x7fc2280feaa0 msgr2=0x7fc228107a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:20:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.589+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 shutdown_connections 2024-05-08T05:20:51.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:20:51.590+0000 7fc22cd64700 1 -- 172.21.15.59:0/1251235576 wait complete. 2024-05-08T05:20:52.288 DEBUG:teuthology.orchestra.run.smithi059:osd.3> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.3.service 2024-05-08T05:20:52.290 INFO:tasks.cephadm:Deploying osd.4 on smithi096 with /dev/vg_nvme/lv_3... 2024-05-08T05:20:52.291 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T05:20:52.535 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:20:52.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:52 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:52.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:52 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:52.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:52 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:52.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:52 smithi059 ceph-mon[30304]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T05:20:52.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:52 smithi059 ceph-mon[30304]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:52.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:52 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:52.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:52 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:52.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:52 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:52.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:52 smithi096 ceph-mon[34253]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T05:20:52.933 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:52 smithi096 ceph-mon[34253]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:52.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:52 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:52.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:52 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:52.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:52 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:20:52.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:52 smithi143 ceph-mon[34337]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-05-08T05:20:52.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:52 smithi143 ceph-mon[34337]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:53.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:53 smithi059 ceph-mon[30304]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T05:20:53.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:53 smithi059 ceph-mon[30304]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T05:20:53.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:53 smithi059 ceph-mon[30304]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:20:53.685 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:53.913 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:53 smithi096 ceph-mon[34253]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T05:20:53.913 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:53 smithi096 ceph-mon[34253]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T05:20:53.913 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:53 smithi096 ceph-mon[34253]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:20:53.914 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:53.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:53 smithi143 ceph-mon[34337]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-05-08T05:20:53.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:53 smithi143 ceph-mon[34337]: osdmap e23: 4 total, 3 up, 4 in 2024-05-08T05:20:53.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:53 smithi143 ceph-mon[34337]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:20:53.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:54.336 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:20:54 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3[52074]: 2024-05-08T05:20:54.058+0000 7f781626a700 -1 osd.3 0 waiting for initial osdmap 2024-05-08T05:20:54.336 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:20:54 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3[52074]: 2024-05-08T05:20:54.065+0000 7f780e055700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:20:54.717 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T05:20:54.717 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T05:20:54.717 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-08T05:20:54.717 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-08T05:20:54.717 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011532 s, 909 MB/s 2024-05-08T05:20:54.717 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-08T05:20:54.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:54 smithi059 ceph-mon[30304]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:20:54.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:54 smithi059 ceph-mon[30304]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T05:20:54.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:54.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:54.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:54 smithi059 ceph-mon[30304]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:54.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:54 smithi059 ceph-mon[30304]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' 2024-05-08T05:20:54.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:54 smithi059 ceph-mon[30304]: osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] boot 2024-05-08T05:20:54.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:54 smithi059 ceph-mon[30304]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T05:20:54.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:54.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:54 smithi143 ceph-mon[34337]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:20:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:54 smithi143 ceph-mon[34337]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T05:20:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:54 smithi143 ceph-mon[34337]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:54 smithi143 ceph-mon[34337]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' 2024-05-08T05:20:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:54 smithi143 ceph-mon[34337]: osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] boot 2024-05-08T05:20:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:54 smithi143 ceph-mon[34337]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T05:20:54.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:55.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:54 smithi096 ceph-mon[34253]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:20:55.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:54 smithi096 ceph-mon[34253]: osdmap e24: 4 total, 3 up, 4 in 2024-05-08T05:20:55.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:54 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:55.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:54 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:55.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:54 smithi096 ceph-mon[34253]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 84 MiB used, 268 GiB / 268 GiB avail 2024-05-08T05:20:55.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:54 smithi096 ceph-mon[34253]: from='osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737]' entity='osd.3' 2024-05-08T05:20:55.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:54 smithi096 ceph-mon[34253]: osd.3 [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] boot 2024-05-08T05:20:55.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:54 smithi096 ceph-mon[34253]: osdmap e25: 4 total, 4 up, 4 in 2024-05-08T05:20:55.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:54 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:20:55.440 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_3 2024-05-08T05:20:55.675 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:20:55.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:55 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:20:55.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:55 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:20:55.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:55 smithi096 ceph-mon[34253]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T05:20:55.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:55 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:20:55.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:55 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:20:55.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:55 smithi143 ceph-mon[34337]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T05:20:56.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:55 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:20:56.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:55 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:20:56.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:55 smithi059 ceph-mon[30304]: osdmap e26: 4 total, 4 up, 4 in 2024-05-08T05:20:56.937 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi059 2024-05-08T05:20:56.937 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi059 to 8546M 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:20:56.938 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:56 smithi096 ceph-mon[34253]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T05:20:56.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi059 2024-05-08T05:20:56.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:56.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:56.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi059 to 8546M 2024-05-08T05:20:56.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:56.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:56.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:20:56.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:56 smithi143 ceph-mon[34337]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T05:20:56.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi059 2024-05-08T05:20:56.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:56.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:20:56.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi059 to 8546M 2024-05-08T05:20:56.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:56.998 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:20:56.999 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:20:56.999 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: pgmap v102: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:20:56.999 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:56 smithi059 ceph-mon[30304]: osdmap e27: 4 total, 4 up, 4 in 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.155+0000 7f8fc2322700 1 -- 172.21.15.96:0/1002788406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc103600 msgr2=0x7f8fbc103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.155+0000 7f8fc2322700 1 --2- 172.21.15.96:0/1002788406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc103600 0x7f8fbc103a60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f8fb0009920 tx=0x7f8fb0009c30 comp rx=0 tx=0).stop 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.155+0000 7f8fc2322700 1 -- 172.21.15.96:0/1002788406 shutdown_connections 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.155+0000 7f8fc2322700 1 --2- 172.21.15.96:0/1002788406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fbc103fa0 0x7f8fbc10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.155+0000 7f8fc2322700 1 --2- 172.21.15.96:0/1002788406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fbc102400 0x7f8fbc102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.155+0000 7f8fc2322700 1 --2- 172.21.15.96:0/1002788406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc103600 0x7f8fbc103a60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.155+0000 7f8fc2322700 1 -- 172.21.15.96:0/1002788406 >> 172.21.15.96:0/1002788406 conn(0x7f8fbc0fd9a0 msgr2=0x7f8fbc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.155+0000 7f8fc2322700 1 -- 172.21.15.96:0/1002788406 shutdown_connections 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.156+0000 7f8fc2322700 1 -- 172.21.15.96:0/1002788406 wait complete. 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.156+0000 7f8fc2322700 1 Processor -- start 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.156+0000 7f8fc2322700 1 -- start start 2024-05-08T05:20:57.159 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.156+0000 7f8fc2322700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc102400 0x7f8fbc06bcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.156+0000 7f8fc2322700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fbc103600 0x7f8fbc19e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fc2322700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fbc103fa0 0x7f8fbc1a0a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fc2322700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc10de80 con 0x7f8fbc103600 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fc2322700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc10dd00 con 0x7f8fbc102400 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc102400 0x7f8fbc06bcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fc2322700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8fbc10e000 con 0x7f8fbc103fa0 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc102400 0x7f8fbc06bcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:54174/0 (socket says 172.21.15.96:54174) 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbbfff700 1 -- 172.21.15.96:0/590545034 learned_addr learned my addr 172.21.15.96:0/590545034 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbb7fe700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fbc103600 0x7f8fbc19e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fc08bf700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fbc103fa0 0x7f8fbc1a0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbbfff700 1 -- 172.21.15.96:0/590545034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fbc103fa0 msgr2=0x7f8fbc1a0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbbfff700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fbc103fa0 0x7f8fbc1a0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbbfff700 1 -- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fbc103600 msgr2=0x7f8fbc19e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbbfff700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fbc103600 0x7f8fbc19e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbbfff700 1 -- 172.21.15.96:0/590545034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fbc1a1350 con 0x7f8fbc102400 2024-05-08T05:20:57.160 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fbbfff700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc102400 0x7f8fbc06bcf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00e9f0 tx=0x7f8fac00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:20:57.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fb97fa700 1 -- 172.21.15.96:0/590545034 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac004220 con 0x7f8fbc102400 2024-05-08T05:20:57.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fb97fa700 1 -- 172.21.15.96:0/590545034 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fac0043c0 con 0x7f8fbc102400 2024-05-08T05:20:57.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fb97fa700 1 -- 172.21.15.96:0/590545034 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac003ce0 con 0x7f8fbc102400 2024-05-08T05:20:57.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fbc1a1640 con 0x7f8fbc102400 2024-05-08T05:20:57.161 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.157+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fbc1a1bd0 con 0x7f8fbc102400 2024-05-08T05:20:57.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.158+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fbc06a190 con 0x7f8fbc102400 2024-05-08T05:20:57.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.160+0000 7f8fb97fa700 1 -- 172.21.15.96:0/590545034 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8fac00c7c0 con 0x7f8fbc102400 2024-05-08T05:20:57.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.160+0000 7f8fb97fa700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8fa40b2900 0x7f8fa40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:20:57.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.160+0000 7f8fb97fa700 1 -- 172.21.15.96:0/590545034 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 3669+0+0 (secure 0 0 0) 0x7f8fac0e1490 con 0x7f8fbc102400 2024-05-08T05:20:57.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.161+0000 7f8fbb7fe700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8fa40b2900 0x7f8fa40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:20:57.162 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.161+0000 7f8fbb7fe700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8fa40b2900 0x7f8fa40b4dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000fc20 tx=0x7f8fb0019040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:20:57.165 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.164+0000 7f8fb97fa700 1 -- 172.21.15.96:0/590545034 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8fac0aa460 con 0x7f8fbc102400 2024-05-08T05:20:57.375 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:20:57.374+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f8fbc0638e0 con 0x7f8fa40b2900 2024-05-08T05:20:57.987 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:57 smithi096 ceph-mon[34253]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:20:57.987 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:20:57.988 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:20:57.988 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:57.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:57 smithi143 ceph-mon[34337]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:20:57.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:20:57.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:20:57.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:57.999 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:57 smithi059 ceph-mon[30304]: from='client.24203 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:20:57.999 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:20:57.999 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:20:57.999 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:20:58.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:20:58 smithi143 ceph-mon[34337]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-05-08T05:20:59.018 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:20:58 smithi096 ceph-mon[34253]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-05-08T05:20:59.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:20:58 smithi059 ceph-mon[30304]: pgmap v104: 1 pgs: 1 active+clean; 577 KiB data, 142 MiB used, 357 GiB / 358 GiB avail; 133 KiB/s, 0 objects/s recovering 2024-05-08T05:21:01.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:00 smithi143 ceph-mon[34337]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:21:01.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:00 smithi143 ceph-mon[34337]: from='client.? 172.21.15.96:0/3792743784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "328d5640-5ab1-4baf-b141-039dce5211c9"}]: dispatch 2024-05-08T05:21:01.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:00 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "328d5640-5ab1-4baf-b141-039dce5211c9"}]: dispatch 2024-05-08T05:21:01.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:00 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "328d5640-5ab1-4baf-b141-039dce5211c9"}]': finished 2024-05-08T05:21:01.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:00 smithi143 ceph-mon[34337]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T05:21:01.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:00 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:01.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:00 smithi143 ceph-mon[34337]: from='client.? 172.21.15.96:0/4155659491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:21:01.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:00 smithi096 ceph-mon[34253]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:21:01.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:00 smithi096 ceph-mon[34253]: from='client.? 172.21.15.96:0/3792743784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "328d5640-5ab1-4baf-b141-039dce5211c9"}]: dispatch 2024-05-08T05:21:01.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:00 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "328d5640-5ab1-4baf-b141-039dce5211c9"}]: dispatch 2024-05-08T05:21:01.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:00 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "328d5640-5ab1-4baf-b141-039dce5211c9"}]': finished 2024-05-08T05:21:01.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:00 smithi096 ceph-mon[34253]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T05:21:01.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:00 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:01.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:00 smithi096 ceph-mon[34253]: from='client.? 172.21.15.96:0/4155659491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:21:01.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:00 smithi059 ceph-mon[30304]: pgmap v105: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:21:01.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:00 smithi059 ceph-mon[30304]: from='client.? 172.21.15.96:0/3792743784' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "328d5640-5ab1-4baf-b141-039dce5211c9"}]: dispatch 2024-05-08T05:21:01.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:00 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "328d5640-5ab1-4baf-b141-039dce5211c9"}]: dispatch 2024-05-08T05:21:01.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:00 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "328d5640-5ab1-4baf-b141-039dce5211c9"}]': finished 2024-05-08T05:21:01.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:00 smithi059 ceph-mon[30304]: osdmap e28: 5 total, 4 up, 5 in 2024-05-08T05:21:01.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:00 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:01.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:00 smithi059 ceph-mon[30304]: from='client.? 172.21.15.96:0/4155659491' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:21:03.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:02 smithi143 ceph-mon[34337]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-08T05:21:03.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:02 smithi096 ceph-mon[34253]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-08T05:21:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:02 smithi059 ceph-mon[30304]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-08T05:21:05.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:04 smithi143 ceph-mon[34337]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:21:05.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:04 smithi096 ceph-mon[34253]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:21:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:04 smithi059 ceph-mon[30304]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:21:07.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:06 smithi143 ceph-mon[34337]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T05:21:07.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:06 smithi096 ceph-mon[34253]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T05:21:07.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:06 smithi059 ceph-mon[30304]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T05:21:08.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:07 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T05:21:08.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:07 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:08.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:07 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T05:21:08.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:07 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:08.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:07 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-05-08T05:21:08.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:07 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:09.130 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:08 smithi096 ceph-mon[34253]: Deploying daemon osd.4 on smithi096 2024-05-08T05:21:09.130 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:08 smithi096 ceph-mon[34253]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:09.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:08 smithi143 ceph-mon[34337]: Deploying daemon osd.4 on smithi096 2024-05-08T05:21:09.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:08 smithi143 ceph-mon[34337]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:09.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:08 smithi059 ceph-mon[30304]: Deploying daemon osd.4 on smithi096 2024-05-08T05:21:09.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:08 smithi059 ceph-mon[30304]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:11.262 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:10 smithi096 ceph-mon[34253]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:11.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:10 smithi059 ceph-mon[30304]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:10 smithi143 ceph-mon[34337]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:13.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:12 smithi096 ceph-mon[34253]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:13.296 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:12 smithi143 ceph-mon[34337]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:12 smithi059 ceph-mon[30304]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:14.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:14.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:14.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:14.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:14.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:14.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:14.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:14.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:14.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:15.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:15 smithi096 ceph-mon[34253]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:15.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:15 smithi059 ceph-mon[30304]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:15 smithi143 ceph-mon[34337]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:16.694 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.694+0000 7f8fb97fa700 1 -- 172.21.15.96:0/590545034 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8fbc0638e0 con 0x7f8fa40b2900 2024-05-08T05:21:16.695 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 4 on host 'smithi096' 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8fa40b2900 msgr2=0x7f8fa40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8fa40b2900 0x7f8fa40b4dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000fc20 tx=0x7f8fb0019040 comp rx=0 tx=0).stop 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc102400 msgr2=0x7f8fbc06bcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc102400 0x7f8fbc06bcf0 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00e9f0 tx=0x7f8fac00ed00 comp rx=0 tx=0).stop 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 shutdown_connections 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8fbc103fa0 0x7f8fbc1a0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8fa40b2900 0x7f8fa40b4dc0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8fbc102400 0x7f8fbc06bcf0 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 --2- 172.21.15.96:0/590545034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8fbc103600 0x7f8fbc19e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:16.697 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 >> 172.21.15.96:0/590545034 conn(0x7f8fbc0fd9a0 msgr2=0x7f8fbc106ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:21:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 shutdown_connections 2024-05-08T05:21:16.698 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:21:16.697+0000 7f8fc2322700 1 -- 172.21.15.96:0/590545034 wait complete. 2024-05-08T05:21:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:17 smithi096 ceph-mon[34253]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:17 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:17 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:17 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:17 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:17 smithi096 ceph-mon[34253]: from='osd.4 [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:21:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:17 smithi096 ceph-mon[34253]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:21:17.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:17 smithi059 ceph-mon[30304]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:17.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:17 smithi059 ceph-mon[30304]: from='osd.4 [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:21:17.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:17 smithi059 ceph-mon[30304]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:21:17.348 DEBUG:teuthology.orchestra.run.smithi096:osd.4> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.4.service 2024-05-08T05:21:17.351 INFO:tasks.cephadm:Deploying osd.5 on smithi143 with /dev/vg_nvme/lv_3... 2024-05-08T05:21:17.351 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-05-08T05:21:17.378 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:17 smithi143 ceph-mon[34337]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:17.378 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:17 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.378 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:17 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.378 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:17 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.378 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:17 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:17.378 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:17 smithi143 ceph-mon[34337]: from='osd.4 [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:21:17.378 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:17 smithi143 ceph-mon[34337]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-05-08T05:21:17.595 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:21:18.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:18 smithi143 ceph-mon[34337]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T05:21:18.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:18 smithi143 ceph-mon[34337]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T05:21:18.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:18.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:18 smithi143 ceph-mon[34337]: from='osd.4 [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:21:18.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:18 smithi143 ceph-mon[34337]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:21:18.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:18 smithi143 ceph-mon[34337]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:19.012 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:18 smithi096 ceph-mon[34253]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T05:21:19.012 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:18 smithi096 ceph-mon[34253]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T05:21:19.012 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:19.012 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:18 smithi096 ceph-mon[34253]: from='osd.4 [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:21:19.012 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:18 smithi096 ceph-mon[34253]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:21:19.012 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:18 smithi096 ceph-mon[34253]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:19.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:18 smithi059 ceph-mon[30304]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-05-08T05:21:19.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:18 smithi059 ceph-mon[30304]: osdmap e29: 5 total, 4 up, 5 in 2024-05-08T05:21:19.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:19.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:18 smithi059 ceph-mon[30304]: from='osd.4 [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:21:19.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:18 smithi059 ceph-mon[30304]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:21:19.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:18 smithi059 ceph-mon[30304]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:19.280 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:21:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4[45644]: 2024-05-08T05:21:19.175+0000 7f3ed59aa700 -1 osd.4 0 waiting for initial osdmap 2024-05-08T05:21:19.280 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:21:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4[45644]: 2024-05-08T05:21:19.182+0000 7f3eccf94700 -1 osd.4 30 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:21:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-05-08T05:21:19.586 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-05-08T05:21:19.587 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-05-08T05:21:19.587 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-05-08T05:21:19.587 INFO:teuthology.orchestra.run.smithi143.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114569 s, 915 MB/s 2024-05-08T05:21:19.587 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 2024-05-08T05:21:19.587 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-05-08T05:21:19.767 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:19 smithi143 ceph-mon[34337]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:21:19.767 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:19 smithi143 ceph-mon[34337]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T05:21:19.767 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:19.767 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:19.767 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:19 smithi143 ceph-mon[34337]: from='osd.4 ' entity='osd.4' 2024-05-08T05:21:19.982 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:19 smithi096 ceph-mon[34253]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:21:19.982 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:19 smithi096 ceph-mon[34253]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T05:21:19.982 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:19.982 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:19.982 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:19 smithi096 ceph-mon[34253]: from='osd.4 ' entity='osd.4' 2024-05-08T05:21:20.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:19 smithi059 ceph-mon[30304]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:21:20.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:19 smithi059 ceph-mon[30304]: osdmap e30: 5 total, 4 up, 5 in 2024-05-08T05:21:20.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:20.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:20.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:19 smithi059 ceph-mon[30304]: from='osd.4 ' entity='osd.4' 2024-05-08T05:21:20.253 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_3 2024-05-08T05:21:20.538 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: osd.4 [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] boot 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi096 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:20.918 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi096 to 8544M 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:20.919 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:21:21.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:21:21.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:21.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:21.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: osd.4 [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] boot 2024-05-08T05:21:21.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T05:21:21.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:21.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi096 2024-05-08T05:21:21.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi096 to 8544M 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:21.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:21:21.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:21:21.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:21.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 115 MiB used, 357 GiB / 358 GiB avail 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: osd.4 [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] boot 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: osdmap e31: 5 total, 5 up, 5 in 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi096 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi096 to 8544M 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:21.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:21.088 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:22.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.152+0000 7fd1a7a53700 1 -- 172.21.15.143:0/277328108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 msgr2=0x7fd1a0106400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:22.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.152+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/277328108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 0x7fd1a0106400 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd190009990 tx=0x7fd190009ca0 comp rx=0 tx=0).stop 2024-05-08T05:21:22.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 -- 172.21.15.143:0/277328108 shutdown_connections 2024-05-08T05:21:22.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/277328108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a0109270 0x7fd1a010b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:22.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/277328108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 0x7fd1a0106400 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:22.156 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/277328108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1a0106940 0x7fd1a0108d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 -- 172.21.15.143:0/277328108 >> 172.21.15.143:0/277328108 conn(0x7fd1a00fd9c0 msgr2=0x7fd1a00ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 -- 172.21.15.143:0/277328108 shutdown_connections 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 -- 172.21.15.143:0/277328108 wait complete. 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 Processor -- start 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 -- start start 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 0x7fd1a0199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a0106940 0x7fd1a0199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.153+0000 7fd1a7a53700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1a0109270 0x7fd1a01a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a7a53700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a010efc0 con 0x7fd1a0106940 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a7a53700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a010ee40 con 0x7fd1a0109270 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a7a53700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd1a010f140 con 0x7fd1a0104010 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a57ef700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 0x7fd1a0199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:21:22.157 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a57ef700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 0x7fd1a0199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:38782/0 (socket says 172.21.15.143:38782) 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a57ef700 1 -- 172.21.15.143:0/1762234663 learned_addr learned my addr 172.21.15.143:0/1762234663 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a5ff0700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1a0109270 0x7fd1a01a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a4fee700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a0106940 0x7fd1a0199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a57ef700 1 -- 172.21.15.143:0/1762234663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1a0109270 msgr2=0x7fd1a01a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a57ef700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1a0109270 0x7fd1a01a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a57ef700 1 -- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a0106940 msgr2=0x7fd1a0199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a57ef700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a0106940 0x7fd1a0199b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a57ef700 1 -- 172.21.15.143:0/1762234663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1a01a1270 con 0x7fd1a0104010 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a5ff0700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1a0109270 0x7fd1a01a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a57ef700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 0x7fd1a0199610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1900059d0 tx=0x7fd19000f510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1967fc700 1 -- 172.21.15.143:0/1762234663 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190027070 con 0x7fd1a0104010 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1a01a1500 con 0x7fd1a0104010 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1967fc700 1 -- 172.21.15.143:0/1762234663 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd19000fc20 con 0x7fd1a0104010 2024-05-08T05:21:22.158 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1967fc700 1 -- 172.21.15.143:0/1762234663 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd190017650 con 0x7fd1a0104010 2024-05-08T05:21:22.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.154+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1a01a1950 con 0x7fd1a0104010 2024-05-08T05:21:22.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.155+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1a006a190 con 0x7fd1a0104010 2024-05-08T05:21:22.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.156+0000 7fd1967fc700 1 -- 172.21.15.143:0/1762234663 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd190009d10 con 0x7fd1a0104010 2024-05-08T05:21:22.159 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.157+0000 7fd1967fc700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd18c0b2880 0x7fd18c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:21:22.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.157+0000 7fd1967fc700 1 -- 172.21.15.143:0/1762234663 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(32..32 src has 1..32) ==== 4101+0+0 (secure 0 0 0) 0x7fd1900e1eb0 con 0x7fd1a0104010 2024-05-08T05:21:22.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.157+0000 7fd1a4fee700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd18c0b2880 0x7fd18c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:21:22.160 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.158+0000 7fd1a4fee700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd18c0b2880 0x7fd18c0b4d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009bd0 tx=0x7fd19c009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:21:22.162 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.162+0000 7fd1967fc700 1 -- 172.21.15.143:0/1762234663 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd190013070 con 0x7fd1a0104010 2024-05-08T05:21:22.373 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:22.372+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd1a00638e0 con 0x7fd18c0b2880 2024-05-08T05:21:22.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:22 smithi143 ceph-mon[34337]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T05:21:22.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:22 smithi143 ceph-mon[34337]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:22.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:22 smithi143 ceph-mon[34337]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:21:22.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:21:22.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:21:22.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:23.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:22 smithi096 ceph-mon[34253]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T05:21:23.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:22 smithi096 ceph-mon[34253]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:23.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:22 smithi096 ceph-mon[34253]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:21:23.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:21:23.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:21:23.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:23.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:22 smithi059 ceph-mon[30304]: osdmap e32: 5 total, 5 up, 5 in 2024-05-08T05:21:23.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:22 smithi059 ceph-mon[30304]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:23.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:22 smithi059 ceph-mon[30304]: from='client.24223 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:21:23.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:21:23.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:21:23.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:25.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:24 smithi143 ceph-mon[34337]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:25.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:24 smithi096 ceph-mon[34253]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:24 smithi059 ceph-mon[30304]: pgmap v122: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:26.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:25 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/2081895270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592fc3c9-1310-441d-b834-bec5021dba8e"}]: dispatch 2024-05-08T05:21:26.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:25 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592fc3c9-1310-441d-b834-bec5021dba8e"}]: dispatch 2024-05-08T05:21:26.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:25 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "592fc3c9-1310-441d-b834-bec5021dba8e"}]': finished 2024-05-08T05:21:26.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:25 smithi143 ceph-mon[34337]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T05:21:26.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:25 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:26.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:25 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/2537090749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:21:26.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:25 smithi096 ceph-mon[34253]: from='client.? 172.21.15.143:0/2081895270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592fc3c9-1310-441d-b834-bec5021dba8e"}]: dispatch 2024-05-08T05:21:26.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:25 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592fc3c9-1310-441d-b834-bec5021dba8e"}]: dispatch 2024-05-08T05:21:26.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:25 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "592fc3c9-1310-441d-b834-bec5021dba8e"}]': finished 2024-05-08T05:21:26.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:25 smithi096 ceph-mon[34253]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T05:21:26.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:25 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:26.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:25 smithi096 ceph-mon[34253]: from='client.? 172.21.15.143:0/2537090749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:21:26.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:25 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/2081895270' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592fc3c9-1310-441d-b834-bec5021dba8e"}]: dispatch 2024-05-08T05:21:26.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:25 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "592fc3c9-1310-441d-b834-bec5021dba8e"}]: dispatch 2024-05-08T05:21:26.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:25 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "592fc3c9-1310-441d-b834-bec5021dba8e"}]': finished 2024-05-08T05:21:26.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:25 smithi059 ceph-mon[30304]: osdmap e33: 6 total, 5 up, 6 in 2024-05-08T05:21:26.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:25 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:26.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:25 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/2537090749' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:21:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:27 smithi143 ceph-mon[34337]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:27.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:27 smithi096 ceph-mon[34253]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:27.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:27 smithi059 ceph-mon[30304]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:29.482 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:29 smithi143 ceph-mon[34337]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:29 smithi096 ceph-mon[34253]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:29.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:29 smithi059 ceph-mon[30304]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:31.396 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:31 smithi143 ceph-mon[34337]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:31.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:31 smithi096 ceph-mon[34253]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:31 smithi059 ceph-mon[30304]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:33 smithi143 ceph-mon[34337]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T05:21:33.439 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:33 smithi096 ceph-mon[34253]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T05:21:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:33 smithi059 ceph-mon[30304]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-05-08T05:21:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:34.482 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:34 smithi143 ceph-mon[34337]: Deploying daemon osd.5 on smithi143 2024-05-08T05:21:34.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:34 smithi096 ceph-mon[34253]: Deploying daemon osd.5 on smithi143 2024-05-08T05:21:34.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:34 smithi059 ceph-mon[30304]: Deploying daemon osd.5 on smithi143 2024-05-08T05:21:35.482 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:35 smithi143 ceph-mon[34337]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:35.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:35 smithi096 ceph-mon[34253]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:35.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:35 smithi059 ceph-mon[30304]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:37 smithi143 ceph-mon[34337]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:37 smithi096 ceph-mon[34253]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:37 smithi059 ceph-mon[30304]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:39 smithi143 ceph-mon[34337]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:39 smithi096 ceph-mon[34253]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:39 smithi059 ceph-mon[30304]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:41 smithi143 ceph-mon[34337]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:41 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:41 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:41 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:41 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:41.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:41 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:41.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:41 smithi096 ceph-mon[34253]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:41.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:41 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:41.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:41 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:41.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:41 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:41.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:41 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:41.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:41 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:41 smithi059 ceph-mon[30304]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:41 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:41 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:41 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:41 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:41.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:41 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:42.043 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 5 on host 'smithi143' 2024-05-08T05:21:42.044 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.042+0000 7fd1967fc700 1 -- 172.21.15.143:0/1762234663 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd1a00638e0 con 0x7fd18c0b2880 2024-05-08T05:21:42.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.045+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd18c0b2880 msgr2=0x7fd18c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:42.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.045+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd18c0b2880 0x7fd18c0b4d40 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd19c009bd0 tx=0x7fd19c009260 comp rx=0 tx=0).stop 2024-05-08T05:21:42.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.045+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 msgr2=0x7fd1a0199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:42.046 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.045+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 0x7fd1a0199610 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7fd1900059d0 tx=0x7fd19000f510 comp rx=0 tx=0).stop 2024-05-08T05:21:42.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.046+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 shutdown_connections 2024-05-08T05:21:42.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.046+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd18c0b2880 0x7fd18c0b4d40 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:42.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.046+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd1a0109270 0x7fd1a01a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:42.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.046+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd1a0104010 0x7fd1a0199610 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:42.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.046+0000 7fd1a7a53700 1 --2- 172.21.15.143:0/1762234663 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd1a0106940 0x7fd1a0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:42.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.046+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 >> 172.21.15.143:0/1762234663 conn(0x7fd1a00fd9c0 msgr2=0x7fd1a0104a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:21:42.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.046+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 shutdown_connections 2024-05-08T05:21:42.047 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:21:42.046+0000 7fd1a7a53700 1 -- 172.21.15.143:0/1762234663 wait complete. 2024-05-08T05:21:42.747 DEBUG:teuthology.orchestra.run.smithi143:osd.5> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.5.service 2024-05-08T05:21:42.749 INFO:tasks.cephadm:Deploying osd.6 on smithi059 with /dev/vg_nvme/lv_2... 2024-05-08T05:21:42.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T05:21:43.012 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:21:43.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:43 smithi059 ceph-mon[30304]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:43.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:43 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:43.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:43 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:43.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:43 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:43.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:43 smithi059 ceph-mon[30304]: from='osd.5 [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:21:43.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:43 smithi059 ceph-mon[30304]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:21:43.256 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:43 smithi143 ceph-mon[34337]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:43.256 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:43 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:43.256 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:43 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:43.257 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:43 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:43.257 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:43 smithi143 ceph-mon[34337]: from='osd.5 [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:21:43.257 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:43 smithi143 ceph-mon[34337]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:21:43.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:43 smithi096 ceph-mon[34253]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:43 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:43 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:43 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:21:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:43 smithi096 ceph-mon[34253]: from='osd.5 [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:21:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:43 smithi096 ceph-mon[34253]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-05-08T05:21:44.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:44 smithi059 ceph-mon[30304]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T05:21:44.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:44 smithi059 ceph-mon[30304]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T05:21:44.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:44 smithi059 ceph-mon[30304]: from='osd.5 [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:21:44.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:44.145 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:44 smithi059 ceph-mon[30304]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:21:44.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:44 smithi143 ceph-mon[34337]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T05:21:44.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:44 smithi143 ceph-mon[34337]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T05:21:44.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:44 smithi143 ceph-mon[34337]: from='osd.5 [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:21:44.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:44.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:44 smithi143 ceph-mon[34337]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:21:44.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:44 smithi096 ceph-mon[34253]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-05-08T05:21:44.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:44 smithi096 ceph-mon[34253]: osdmap e34: 6 total, 5 up, 6 in 2024-05-08T05:21:44.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:44 smithi096 ceph-mon[34253]: from='osd.5 [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:21:44.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:44.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:44 smithi096 ceph-mon[34253]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:21:44.555 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:21:44 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:21:44.458+0000 7f4223142700 -1 osd.5 0 waiting for initial osdmap 2024-05-08T05:21:44.556 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:21:44 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:21:44.467+0000 7f421af2d700 -1 osd.5 35 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:21:45.042 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T05:21:45.043 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T05:21:45.043 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-08T05:21:45.043 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-08T05:21:45.043 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0118087 s, 888 MB/s 2024-05-08T05:21:45.043 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-08T05:21:45.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:45 smithi096 ceph-mon[34253]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:45.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:45 smithi096 ceph-mon[34253]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:21:45.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:45 smithi096 ceph-mon[34253]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T05:21:45.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:45 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:45.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:45 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:45.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:45 smithi096 ceph-mon[34253]: from='osd.5 ' entity='osd.5' 2024-05-08T05:21:45.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:45 smithi059 ceph-mon[30304]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:45.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:45 smithi059 ceph-mon[30304]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:21:45.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:45 smithi059 ceph-mon[30304]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T05:21:45.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:45 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:45.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:45 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:45.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:45 smithi059 ceph-mon[30304]: from='osd.5 ' entity='osd.5' 2024-05-08T05:21:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:45 smithi143 ceph-mon[34337]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 146 MiB used, 447 GiB / 447 GiB avail 2024-05-08T05:21:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:45 smithi143 ceph-mon[34337]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:21:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:45 smithi143 ceph-mon[34337]: osdmap e35: 6 total, 5 up, 6 in 2024-05-08T05:21:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:45 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:45.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:45 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:45.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:45 smithi143 ceph-mon[34337]: from='osd.5 ' entity='osd.5' 2024-05-08T05:21:45.692 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_2 2024-05-08T05:21:45.951 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:21:46.109 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:46 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:21:46.109 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:46 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:21:46.109 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:46 smithi143 ceph-mon[34337]: osd.5 [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] boot 2024-05-08T05:21:46.109 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:46 smithi143 ceph-mon[34337]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T05:21:46.109 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:46 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:46.109 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:46 smithi143 ceph-mon[34337]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T05:21:46.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:46 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:46 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:46 smithi096 ceph-mon[34253]: osd.5 [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] boot 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:46 smithi096 ceph-mon[34253]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:46 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:46 smithi096 ceph-mon[34253]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:46 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:46 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:46 smithi059 ceph-mon[30304]: osd.5 [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] boot 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:46 smithi059 ceph-mon[30304]: osdmap e36: 6 total, 6 up, 6 in 2024-05-08T05:21:46.280 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:46 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:21:46.281 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:46 smithi059 ceph-mon[30304]: osdmap e37: 6 total, 6 up, 6 in 2024-05-08T05:21:47.374 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:21:47.374 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi143 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi143 to 8544M 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.375 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:47 smithi059 ceph-mon[30304]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T05:21:47.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 -- 172.21.15.59:0/3901067448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4103600 msgr2=0x7f78d4103a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:47.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 --2- 172.21.15.59:0/3901067448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4103600 0x7f78d4103a60 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f78bc0099c0 tx=0x7f78bc009cd0 comp rx=0 tx=0).stop 2024-05-08T05:21:47.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 -- 172.21.15.59:0/3901067448 shutdown_connections 2024-05-08T05:21:47.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 --2- 172.21.15.59:0/3901067448 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78d4103fa0 0x7f78d410a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:47.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 --2- 172.21.15.59:0/3901067448 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d4102400 0x7f78d4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:47.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 --2- 172.21.15.59:0/3901067448 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4103600 0x7f78d4103a60 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 -- 172.21.15.59:0/3901067448 >> 172.21.15.59:0/3901067448 conn(0x7f78d40fd9a0 msgr2=0x7f78d40ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 -- 172.21.15.59:0/3901067448 shutdown_connections 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 -- 172.21.15.59:0/3901067448 wait complete. 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 Processor -- start 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 -- start start 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.392+0000 7f78da97c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4102400 0x7f78d4106390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78da97c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78d4103600 0x7f78d41049e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78da97c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d4103fa0 0x7f78d4104f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78da97c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f78d410de80 con 0x7f78d4102400 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78da97c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f78d410dd00 con 0x7f78d4103600 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78da97c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f78d410e000 con 0x7f78d4103fa0 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4102400 0x7f78d4106390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4102400 0x7f78d4106390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46514/0 (socket says 172.21.15.59:46514) 2024-05-08T05:21:47.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d3fff700 1 -- 172.21.15.59:0/4224880325 learned_addr learned my addr 172.21.15.59:0/4224880325 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78cbfff700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78d4103600 0x7f78d41049e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d8f19700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d4103fa0 0x7f78d4104f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d3fff700 1 -- 172.21.15.59:0/4224880325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d4103fa0 msgr2=0x7f78d4104f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d3fff700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d4103fa0 0x7f78d4104f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d3fff700 1 -- 172.21.15.59:0/4224880325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78d4103600 msgr2=0x7f78d41049e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d3fff700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78d4103600 0x7f78d41049e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d3fff700 1 -- 172.21.15.59:0/4224880325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78d4105460 con 0x7f78d4102400 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d3fff700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4102400 0x7f78d4106390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f78c400c990 tx=0x7f78c400cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d1ffb700 1 -- 172.21.15.59:0/4224880325 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4007b00 con 0x7f78d4102400 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d1ffb700 1 -- 172.21.15.59:0/4224880325 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c4007ca0 con 0x7f78d4102400 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78d4105750 con 0x7f78d4102400 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.393+0000 7f78d1ffb700 1 -- 172.21.15.59:0/4224880325 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4018660 con 0x7f78d4102400 2024-05-08T05:21:47.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.394+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78d41a8170 con 0x7f78d4102400 2024-05-08T05:21:47.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.395+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78d41a8340 con 0x7f78d4102400 2024-05-08T05:21:47.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.396+0000 7f78d1ffb700 1 -- 172.21.15.59:0/4224880325 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f78c400f450 con 0x7f78d4102400 2024-05-08T05:21:47.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.397+0000 7f78d1ffb700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f78c00b2880 0x7f78c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:21:47.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.397+0000 7f78cbfff700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f78c00b2880 0x7f78c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:21:47.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.397+0000 7f78d1ffb700 1 -- 172.21.15.59:0/4224880325 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(38..38 src has 1..38) ==== 4533+0+0 (secure 0 0 0) 0x7f78c40e1350 con 0x7f78d4102400 2024-05-08T05:21:47.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.397+0000 7f78cbfff700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f78c00b2880 0x7f78c00b4d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009990 tx=0x7f78bc000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:21:47.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.401+0000 7f78d1ffb700 1 -- 172.21.15.59:0/4224880325 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f78c4014030 con 0x7f78d4102400 2024-05-08T05:21:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi143 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi143 to 8544M 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:47 smithi143 ceph-mon[34337]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: pgmap v138: 1 pgs: 1 active+clean; 577 KiB data, 176 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi143 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi143 to 8544M 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:21:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:47 smithi096 ceph-mon[34253]: osdmap e38: 6 total, 6 up, 6 in 2024-05-08T05:21:47.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:21:47.627+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f78d41a8510 con 0x7f78c00b2880 2024-05-08T05:21:48.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:21:48.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:21:48.477 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:48.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:21:48.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:21:48.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:48.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:21:48.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:21:48.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:49 smithi059 ceph-mon[30304]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:21:49.298 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:49 smithi059 ceph-mon[30304]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T05:21:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:49 smithi143 ceph-mon[34337]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:21:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:49 smithi143 ceph-mon[34337]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T05:21:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:49 smithi096 ceph-mon[34253]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:21:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:49 smithi096 ceph-mon[34253]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-05-08T05:21:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:51 smithi143 ceph-mon[34337]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T05:21:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:51 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3941223965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bce63f2d-f8c7-4304-bec8-8f1d803e2791"}]: dispatch 2024-05-08T05:21:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:51 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3941223965' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bce63f2d-f8c7-4304-bec8-8f1d803e2791"}]': finished 2024-05-08T05:21:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:51 smithi143 ceph-mon[34337]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T05:21:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:21:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:51 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3576858703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:21:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:51 smithi096 ceph-mon[34253]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T05:21:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:51 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3941223965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bce63f2d-f8c7-4304-bec8-8f1d803e2791"}]: dispatch 2024-05-08T05:21:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:51 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3941223965' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bce63f2d-f8c7-4304-bec8-8f1d803e2791"}]': finished 2024-05-08T05:21:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:51 smithi096 ceph-mon[34253]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T05:21:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:21:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:51 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3576858703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:21:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:51 smithi059 ceph-mon[30304]: pgmap v141: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 97 KiB/s, 0 objects/s recovering 2024-05-08T05:21:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:51 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3941223965' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "bce63f2d-f8c7-4304-bec8-8f1d803e2791"}]: dispatch 2024-05-08T05:21:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:51 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3941223965' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "bce63f2d-f8c7-4304-bec8-8f1d803e2791"}]': finished 2024-05-08T05:21:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:51 smithi059 ceph-mon[30304]: osdmap e39: 7 total, 6 up, 7 in 2024-05-08T05:21:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:21:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:51 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3576858703' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:21:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:53 smithi143 ceph-mon[34337]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-08T05:21:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:53 smithi096 ceph-mon[34253]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-08T05:21:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:53 smithi059 ceph-mon[30304]: pgmap v143: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 89 KiB/s, 0 objects/s recovering 2024-05-08T05:21:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:55 smithi143 ceph-mon[34337]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:21:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:55 smithi096 ceph-mon[34253]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:21:55.582 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:55 smithi059 ceph-mon[30304]: pgmap v144: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-05-08T05:21:57.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:57 smithi059 ceph-mon[30304]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T05:21:57.457 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:57 smithi096 ceph-mon[34253]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T05:21:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:57 smithi143 ceph-mon[34337]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail; 61 KiB/s, 0 objects/s recovering 2024-05-08T05:21:58.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:58 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T05:21:58.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:58 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:58.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:58 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T05:21:58.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:58 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:58.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:58 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-05-08T05:21:58.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:58 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:21:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:59 smithi143 ceph-mon[34337]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:21:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:21:59 smithi143 ceph-mon[34337]: Deploying daemon osd.6 on smithi059 2024-05-08T05:21:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:59 smithi096 ceph-mon[34253]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:21:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:21:59 smithi096 ceph-mon[34253]: Deploying daemon osd.6 on smithi059 2024-05-08T05:21:59.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:59 smithi059 ceph-mon[30304]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:21:59.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:21:59 smithi059 ceph-mon[30304]: Deploying daemon osd.6 on smithi059 2024-05-08T05:22:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:01 smithi143 ceph-mon[34337]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:01 smithi096 ceph-mon[34253]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:01.553 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:01 smithi059 ceph-mon[30304]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:03 smithi143 ceph-mon[34337]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:03.498 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:03 smithi059 ceph-mon[30304]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:03 smithi096 ceph-mon[34253]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:05.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:05.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:05.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:05.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:04 smithi059 ceph-mon[30304]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:05.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:05.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:05.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:05.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:04 smithi143 ceph-mon[34337]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:05.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:05.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:05.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:05.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:04 smithi096 ceph-mon[34253]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:07.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:06 smithi143 ceph-mon[34337]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:07.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:07.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:07.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:07.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:07.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:07.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:06 smithi096 ceph-mon[34253]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:07.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:07.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:07.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:07.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:07.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:07.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:06 smithi059 ceph-mon[30304]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:07.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:07.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:07.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:07.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:07.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:07.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.686+0000 7f78d1ffb700 1 -- 172.21.15.59:0/4224880325 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f78d41a8510 con 0x7f78c00b2880 2024-05-08T05:22:07.688 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 6 on host 'smithi059' 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f78c00b2880 msgr2=0x7f78c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f78c00b2880 0x7f78c00b4d40 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f78bc009990 tx=0x7f78bc000bc0 comp rx=0 tx=0).stop 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4102400 msgr2=0x7f78d4106390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4102400 0x7f78d4106390 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f78c400c990 tx=0x7f78c400cca0 comp rx=0 tx=0).stop 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 shutdown_connections 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f78d4103fa0 0x7f78d4104f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f78d4102400 0x7f78d4106390 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f78c00b2880 0x7f78c00b4d40 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 --2- 172.21.15.59:0/4224880325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f78d4103600 0x7f78d41049e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:07.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 >> 172.21.15.59:0/4224880325 conn(0x7f78d40fd9a0 msgr2=0x7f78d40ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:22:07.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 shutdown_connections 2024-05-08T05:22:07.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:22:07.690+0000 7f78da97c700 1 -- 172.21.15.59:0/4224880325 wait complete. 2024-05-08T05:22:08.434 DEBUG:teuthology.orchestra.run.smithi059:osd.6> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.6.service 2024-05-08T05:22:08.436 INFO:tasks.cephadm:Deploying osd.7 on smithi096 with /dev/vg_nvme/lv_2... 2024-05-08T05:22:08.437 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T05:22:08.680 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:22:08.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:08.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:08.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:08.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:08 smithi096 ceph-mon[34253]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:22:08.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:08 smithi096 ceph-mon[34253]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:08.928 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:08.928 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:08.928 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:08.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:08 smithi059 ceph-mon[30304]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:22:08.929 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:08 smithi059 ceph-mon[30304]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:08.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:08.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:08.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:08.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:08 smithi143 ceph-mon[34337]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-05-08T05:22:08.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:08 smithi143 ceph-mon[34337]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:09.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:09 smithi059 ceph-mon[30304]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T05:22:09.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:09 smithi059 ceph-mon[30304]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T05:22:09.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:09 smithi059 ceph-mon[30304]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:22:09.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:09 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:09.837 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:22:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:22:09.689+0000 7f418a931700 -1 osd.6 0 waiting for initial osdmap 2024-05-08T05:22:09.837 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:22:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:22:09.700+0000 7f4186724700 -1 osd.6 41 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:22:09.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:09 smithi096 ceph-mon[34253]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T05:22:09.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:09 smithi096 ceph-mon[34253]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T05:22:09.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:09 smithi096 ceph-mon[34253]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:22:09.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:09 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:09.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:09 smithi143 ceph-mon[34337]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-05-08T05:22:09.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:09 smithi143 ceph-mon[34337]: osdmap e40: 7 total, 6 up, 7 in 2024-05-08T05:22:09.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:09 smithi143 ceph-mon[34337]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:22:09.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:09 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:10.677 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T05:22:10.678 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T05:22:10.678 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-08T05:22:10.678 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-08T05:22:10.678 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0130701 s, 802 MB/s 2024-05-08T05:22:10.678 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-08T05:22:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:10 smithi143 ceph-mon[34337]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:22:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:10 smithi143 ceph-mon[34337]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T05:22:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:10 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:10 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:10 smithi143 ceph-mon[34337]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:11.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:10 smithi096 ceph-mon[34253]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:22:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:10 smithi096 ceph-mon[34253]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T05:22:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:10 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:10 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:10 smithi096 ceph-mon[34253]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:11.083 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:10 smithi059 ceph-mon[30304]: from='osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:22:11.084 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:10 smithi059 ceph-mon[30304]: osdmap e41: 7 total, 6 up, 7 in 2024-05-08T05:22:11.084 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:10 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.084 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:10 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.084 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:10 smithi059 ceph-mon[30304]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 177 MiB used, 536 GiB / 536 GiB avail 2024-05-08T05:22:11.370 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_2 2024-05-08T05:22:11.613 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] boot 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:11.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:11.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:11 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:22:11.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:22:11.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] boot 2024-05-08T05:22:11.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T05:22:11.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.962 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.963 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:11.963 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:11.963 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:11.963 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.963 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:11.963 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:11.963 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:11 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:22:11.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:22:11.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: osd.6 [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] boot 2024-05-08T05:22:11.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: osdmap e42: 7 total, 7 up, 7 in 2024-05-08T05:22:11.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:22:11.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:11.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:11.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:11.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:11.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:11.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:11.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:11 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:12.911 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:12 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi059 2024-05-08T05:22:12.911 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:12 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi059 to 5697M 2024-05-08T05:22:12.911 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:12 smithi096 ceph-mon[34253]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:12.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:12 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi059 2024-05-08T05:22:12.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:12 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi059 to 5697M 2024-05-08T05:22:12.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:12 smithi143 ceph-mon[34337]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:13.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:12 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi059 2024-05-08T05:22:13.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:12 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi059 to 5697M 2024-05-08T05:22:13.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:12 smithi059 ceph-mon[30304]: pgmap v156: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.135+0000 7ff30886d700 1 -- 172.21.15.96:0/227494121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 msgr2=0x7ff304103920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.135+0000 7ff30886d700 1 --2- 172.21.15.96:0/227494121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 0x7ff304103920 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40099c0 tx=0x7ff2f4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.135+0000 7ff30886d700 1 -- 172.21.15.96:0/227494121 shutdown_connections 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.135+0000 7ff30886d700 1 --2- 172.21.15.96:0/227494121 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff3041050a0 0x7ff30410b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.135+0000 7ff30886d700 1 --2- 172.21.15.96:0/227494121 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 0x7ff304103920 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.135+0000 7ff30886d700 1 --2- 172.21.15.96:0/227494121 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff304104700 0x7ff304104b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.135+0000 7ff30886d700 1 -- 172.21.15.96:0/227494121 >> 172.21.15.96:0/227494121 conn(0x7ff3040feaa0 msgr2=0x7ff304100ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.135+0000 7ff30886d700 1 -- 172.21.15.96:0/227494121 shutdown_connections 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.135+0000 7ff30886d700 1 -- 172.21.15.96:0/227494121 wait complete. 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30886d700 1 Processor -- start 2024-05-08T05:22:13.137 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30886d700 1 -- start start 2024-05-08T05:22:13.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30886d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 0x7ff3041996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:22:13.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30886d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff304104700 0x7ff304199c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:22:13.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30886d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3041050a0 0x7ff3041a0ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:22:13.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30886d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff30410efb0 con 0x7ff304104700 2024-05-08T05:22:13.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30886d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff30410ee30 con 0x7ff304103500 2024-05-08T05:22:13.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30886d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff30410f130 con 0x7ff3041050a0 2024-05-08T05:22:13.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 0x7ff3041996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:22:13.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30259c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 0x7ff3041996e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:39554/0 (socket says 172.21.15.96:39554) 2024-05-08T05:22:13.139 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.136+0000 7ff30259c700 1 -- 172.21.15.96:0/133249097 learned_addr learned my addr 172.21.15.96:0/133249097 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff302d9d700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3041050a0 0x7ff3041a0ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff301d9b700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff304104700 0x7ff304199c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff30259c700 1 -- 172.21.15.96:0/133249097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3041050a0 msgr2=0x7ff3041a0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff30259c700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3041050a0 0x7ff3041a0ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff30259c700 1 -- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff304104700 msgr2=0x7ff304199c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff30259c700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff304104700 0x7ff304199c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff30259c700 1 -- 172.21.15.96:0/133249097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3041a1340 con 0x7ff304103500 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff301d9b700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff304104700 0x7ff304199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff30259c700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 0x7ff3041996e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40038c0 tx=0x7ff2f40047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff2f37fe700 1 -- 172.21.15.96:0/133249097 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f4027070 con 0x7ff304103500 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3041a15d0 con 0x7ff304103500 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff2f37fe700 1 -- 172.21.15.96:0/133249097 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2f40041a0 con 0x7ff304103500 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff2f37fe700 1 -- 172.21.15.96:0/133249097 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2f400f460 con 0x7ff304103500 2024-05-08T05:22:13.140 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.137+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3041a1ab0 con 0x7ff304103500 2024-05-08T05:22:13.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.138+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff30406a190 con 0x7ff304103500 2024-05-08T05:22:13.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.139+0000 7ff2f37fe700 1 -- 172.21.15.96:0/133249097 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff2f402a070 con 0x7ff304103500 2024-05-08T05:22:13.141 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.140+0000 7ff2f37fe700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff2ec0b2880 0x7ff2ec0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:22:13.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.140+0000 7ff2f37fe700 1 -- 172.21.15.96:0/133249097 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(43..43 src has 1..43) ==== 4965+0+0 (secure 0 0 0) 0x7ff2f40e28d0 con 0x7ff304103500 2024-05-08T05:22:13.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.140+0000 7ff301d9b700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff2ec0b2880 0x7ff2ec0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:22:13.142 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.140+0000 7ff301d9b700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff2ec0b2880 0x7ff2ec0b4d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800bcf0 tx=0x7ff2f800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:22:13.144 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.144+0000 7ff2f37fe700 1 -- 172.21.15.96:0/133249097 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2f4031070 con 0x7ff304103500 2024-05-08T05:22:13.353 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:13.352+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7ff3040638e0 con 0x7ff2ec0b2880 2024-05-08T05:22:13.965 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:13 smithi096 ceph-mon[34253]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T05:22:13.965 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:13 smithi096 ceph-mon[34253]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:22:13.965 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:13 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:22:13.965 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:13 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:22:13.965 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:13 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:13.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:13 smithi143 ceph-mon[34337]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T05:22:13.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:13 smithi143 ceph-mon[34337]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:22:13.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:13 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:22:13.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:13 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:22:13.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:13 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:14.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:13 smithi059 ceph-mon[30304]: osdmap e43: 7 total, 7 up, 7 in 2024-05-08T05:22:14.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:13 smithi059 ceph-mon[30304]: from='client.24266 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:22:14.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:13 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:22:14.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:13 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:22:14.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:13 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:14.972 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:14 smithi096 ceph-mon[34253]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:14.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:14 smithi143 ceph-mon[34337]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:15.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:14 smithi059 ceph-mon[30304]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:17.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:16 smithi143 ceph-mon[34337]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:17.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:16 smithi143 ceph-mon[34337]: from='client.? 172.21.15.96:0/1165206600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f10748de-9037-46bd-9db3-1d88f6f93af8"}]: dispatch 2024-05-08T05:22:17.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:16 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f10748de-9037-46bd-9db3-1d88f6f93af8"}]: dispatch 2024-05-08T05:22:17.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:16 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f10748de-9037-46bd-9db3-1d88f6f93af8"}]': finished 2024-05-08T05:22:17.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:16 smithi143 ceph-mon[34337]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T05:22:17.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:16 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:17.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:16 smithi143 ceph-mon[34337]: from='client.? 172.21.15.96:0/3751229108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:22:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:16 smithi096 ceph-mon[34253]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:16 smithi096 ceph-mon[34253]: from='client.? 172.21.15.96:0/1165206600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f10748de-9037-46bd-9db3-1d88f6f93af8"}]: dispatch 2024-05-08T05:22:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:16 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f10748de-9037-46bd-9db3-1d88f6f93af8"}]: dispatch 2024-05-08T05:22:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:16 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f10748de-9037-46bd-9db3-1d88f6f93af8"}]': finished 2024-05-08T05:22:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:16 smithi096 ceph-mon[34253]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T05:22:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:16 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:17.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:16 smithi096 ceph-mon[34253]: from='client.? 172.21.15.96:0/3751229108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:22:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:16 smithi059 ceph-mon[30304]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:16 smithi059 ceph-mon[30304]: from='client.? 172.21.15.96:0/1165206600' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f10748de-9037-46bd-9db3-1d88f6f93af8"}]: dispatch 2024-05-08T05:22:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:16 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f10748de-9037-46bd-9db3-1d88f6f93af8"}]: dispatch 2024-05-08T05:22:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:16 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f10748de-9037-46bd-9db3-1d88f6f93af8"}]': finished 2024-05-08T05:22:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:16 smithi059 ceph-mon[30304]: osdmap e44: 8 total, 7 up, 8 in 2024-05-08T05:22:17.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:16 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:17.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:16 smithi059 ceph-mon[30304]: from='client.? 172.21.15.96:0/3751229108' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:22:19.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:18 smithi096 ceph-mon[34253]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:18 smithi059 ceph-mon[30304]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:18 smithi143 ceph-mon[34337]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:21.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:20 smithi096 ceph-mon[34253]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:20 smithi059 ceph-mon[30304]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:20 smithi143 ceph-mon[34337]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:23.262 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:23 smithi096 ceph-mon[34253]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:23 smithi059 ceph-mon[30304]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:23 smithi143 ceph-mon[34337]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:24.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T05:22:24.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:24.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T05:22:24.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:24.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-05-08T05:22:24.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:25.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:25 smithi096 ceph-mon[34253]: Deploying daemon osd.7 on smithi096 2024-05-08T05:22:25.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:25 smithi096 ceph-mon[34253]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:25 smithi059 ceph-mon[30304]: Deploying daemon osd.7 on smithi096 2024-05-08T05:22:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:25 smithi059 ceph-mon[30304]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:25 smithi143 ceph-mon[34337]: Deploying daemon osd.7 on smithi096 2024-05-08T05:22:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:25 smithi143 ceph-mon[34337]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:27.267 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:27 smithi096 ceph-mon[34253]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:27.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:27 smithi059 ceph-mon[30304]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:27 smithi143 ceph-mon[34337]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:29.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:29 smithi096 ceph-mon[34253]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:29.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:29 smithi059 ceph-mon[30304]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:29.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:29 smithi143 ceph-mon[34337]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:30.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:30.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:30.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:30.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:30.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:30.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:30.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:30.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:30.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:31.437 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:31 smithi096 ceph-mon[34253]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:31.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:31 smithi143 ceph-mon[34337]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:31 smithi059 ceph-mon[30304]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:32.724 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.723+0000 7ff2f37fe700 1 -- 172.21.15.96:0/133249097 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff3040638e0 con 0x7ff2ec0b2880 2024-05-08T05:22:32.724 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 7 on host 'smithi096' 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff2ec0b2880 msgr2=0x7ff2ec0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff2ec0b2880 0x7ff2ec0b4d40 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7ff2f800bcf0 tx=0x7ff2f800b2c0 comp rx=0 tx=0).stop 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 msgr2=0x7ff3041996e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 0x7ff3041996e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7ff2f40038c0 tx=0x7ff2f40047b0 comp rx=0 tx=0).stop 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 shutdown_connections 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff3041050a0 0x7ff3041a0ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff2ec0b2880 0x7ff2ec0b4d40 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff304103500 0x7ff3041996e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 --2- 172.21.15.96:0/133249097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff304104700 0x7ff304199c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 >> 172.21.15.96:0/133249097 conn(0x7ff3040feaa0 msgr2=0x7ff304107b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:22:32.727 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 shutdown_connections 2024-05-08T05:22:32.728 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:22:32.726+0000 7ff30886d700 1 -- 172.21.15.96:0/133249097 wait complete. 2024-05-08T05:22:33.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:33 smithi096 ceph-mon[34253]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:33.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:33.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:33.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:33 smithi096 ceph-mon[34253]: from='osd.7 [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:22:33.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:33 smithi096 ceph-mon[34253]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:22:33.395 DEBUG:teuthology.orchestra.run.smithi096:osd.7> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.7.service 2024-05-08T05:22:33.397 INFO:tasks.cephadm:Deploying osd.8 on smithi143 with /dev/vg_nvme/lv_2... 2024-05-08T05:22:33.398 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-05-08T05:22:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:33 smithi143 ceph-mon[34337]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:33 smithi143 ceph-mon[34337]: from='osd.7 [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:22:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:33 smithi143 ceph-mon[34337]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:22:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:33 smithi059 ceph-mon[30304]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:33.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:33 smithi059 ceph-mon[30304]: from='osd.7 [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:22:33.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:33 smithi059 ceph-mon[30304]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-05-08T05:22:33.647 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:22:34.966 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:22:34 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:22:34.713+0000 7fc6aa4a5700 -1 osd.7 0 waiting for initial osdmap 2024-05-08T05:22:34.966 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:22:34 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:22:34.723+0000 7fc6a6298700 -1 osd.7 46 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:22:34.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:34.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:34.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:34 smithi096 ceph-mon[34253]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T05:22:34.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:34 smithi096 ceph-mon[34253]: from='osd.7 [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:22:34.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:34 smithi096 ceph-mon[34253]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T05:22:34.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:34 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:34.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:34 smithi096 ceph-mon[34253]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:22:34.967 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:34 smithi096 ceph-mon[34253]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:34.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:34 smithi143 ceph-mon[34337]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T05:22:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:34 smithi143 ceph-mon[34337]: from='osd.7 [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:22:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:34 smithi143 ceph-mon[34337]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T05:22:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:34 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:34 smithi143 ceph-mon[34337]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:22:34.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:34 smithi143 ceph-mon[34337]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:35.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:35.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:35.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:34 smithi059 ceph-mon[30304]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-05-08T05:22:35.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:34 smithi059 ceph-mon[30304]: from='osd.7 [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:22:35.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:34 smithi059 ceph-mon[30304]: osdmap e45: 8 total, 7 up, 8 in 2024-05-08T05:22:35.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:34 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:35.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:34 smithi059 ceph-mon[30304]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:22:35.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:34 smithi059 ceph-mon[30304]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 204 MiB used, 626 GiB / 626 GiB avail 2024-05-08T05:22:35.705 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-05-08T05:22:35.705 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-05-08T05:22:35.705 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-05-08T05:22:35.705 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-05-08T05:22:35.705 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0134002 s, 783 MB/s 2024-05-08T05:22:35.705 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-05-08T05:22:35.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:35 smithi143 ceph-mon[34337]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:22:35.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:35 smithi143 ceph-mon[34337]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T05:22:35.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:35 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:35.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:35 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:35.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:35 smithi143 ceph-mon[34337]: osd.7 [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] boot 2024-05-08T05:22:35.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:35 smithi143 ceph-mon[34337]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T05:22:35.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:35 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:35.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:35 smithi059 ceph-mon[30304]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:22:35.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:35 smithi059 ceph-mon[30304]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T05:22:35.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:35 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:35.995 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:35 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:35.996 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:35 smithi059 ceph-mon[30304]: osd.7 [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] boot 2024-05-08T05:22:35.996 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:35 smithi059 ceph-mon[30304]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T05:22:35.996 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:35 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:36.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:35 smithi096 ceph-mon[34253]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:22:36.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:35 smithi096 ceph-mon[34253]: osdmap e46: 8 total, 7 up, 8 in 2024-05-08T05:22:36.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:35 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:36.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:35 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:36.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:35 smithi096 ceph-mon[34253]: osd.7 [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] boot 2024-05-08T05:22:36.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:35 smithi096 ceph-mon[34253]: osdmap e47: 8 total, 8 up, 8 in 2024-05-08T05:22:36.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:35 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:22:36.354 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_2 2024-05-08T05:22:36.596 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:22:36.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:36 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:22:36.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:36 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:22:36.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:36 smithi143 ceph-mon[34337]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:36.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:36 smithi143 ceph-mon[34337]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T05:22:37.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:36 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:22:37.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:36 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:22:37.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:36 smithi096 ceph-mon[34253]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:37.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:36 smithi096 ceph-mon[34253]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T05:22:37.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:36 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:22:37.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:36 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:22:37.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:36 smithi059 ceph-mon[30304]: pgmap v173: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:37.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:36 smithi059 ceph-mon[30304]: osdmap e48: 8 total, 8 up, 8 in 2024-05-08T05:22:38.252 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 -- 172.21.15.143:0/1757613748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498105190 msgr2=0x7f649810ba40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 --2- 172.21.15.143:0/1757613748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498105190 0x7f649810ba40 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f649000b0a0 tx=0x7f649000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 -- 172.21.15.143:0/1757613748 shutdown_connections 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 --2- 172.21.15.143:0/1757613748 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498105190 0x7f649810ba40 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 --2- 172.21.15.143:0/1757613748 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6498101810 0x7f6498101c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 --2- 172.21.15.143:0/1757613748 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64981047f0 0x7f6498104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 -- 172.21.15.143:0/1757613748 >> 172.21.15.143:0/1757613748 conn(0x7f64980fd170 msgr2=0x7f64980ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 -- 172.21.15.143:0/1757613748 shutdown_connections 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 -- 172.21.15.143:0/1757613748 wait complete. 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.250+0000 7f649ce96700 1 Processor -- start 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649ce96700 1 -- start start 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649ce96700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498101810 0x7f64981996a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:22:38.253 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649ce96700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64981047f0 0x7f6498199be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649ce96700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6498105190 0x7f64981a0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649ce96700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f649810f0b0 con 0x7f64981047f0 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649ce96700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f649810ef30 con 0x7f6498105190 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649ce96700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f649810f230 con 0x7f6498101810 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498101810 0x7f64981996a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498101810 0x7f64981996a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:53540/0 (socket says 172.21.15.143:53540) 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649659c700 1 -- 172.21.15.143:0/1661047319 learned_addr learned my addr 172.21.15.143:0/1661047319 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f648dd9b700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64981047f0 0x7f6498199be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f6496d9d700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6498105190 0x7f64981a0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649659c700 1 -- 172.21.15.143:0/1661047319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6498105190 msgr2=0x7f64981a0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649659c700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6498105190 0x7f64981a0c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649659c700 1 -- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64981047f0 msgr2=0x7f6498199be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:38.254 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649659c700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64981047f0 0x7f6498199be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:38.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649659c700 1 -- 172.21.15.143:0/1661047319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64981a1300 con 0x7f6498101810 2024-05-08T05:22:38.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f6496d9d700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6498105190 0x7f64981a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:22:38.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f649659c700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498101810 0x7f64981996a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f648800b530 tx=0x7f648800b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:22:38.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.251+0000 7f648ffff700 1 -- 172.21.15.143:0/1661047319 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488015020 con 0x7f6498101810 2024-05-08T05:22:38.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.252+0000 7f648ffff700 1 -- 172.21.15.143:0/1661047319 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6488011ce0 con 0x7f6498101810 2024-05-08T05:22:38.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.252+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64981a15f0 con 0x7f6498101810 2024-05-08T05:22:38.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.252+0000 7f648ffff700 1 -- 172.21.15.143:0/1661047319 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6488010d60 con 0x7f6498101810 2024-05-08T05:22:38.255 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.252+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64981a1b60 con 0x7f6498101810 2024-05-08T05:22:38.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.253+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f649806a190 con 0x7f6498101810 2024-05-08T05:22:38.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.254+0000 7f648ffff700 1 -- 172.21.15.143:0/1661047319 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6488018020 con 0x7f6498101810 2024-05-08T05:22:38.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.254+0000 7f648ffff700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f64840b2880 0x7f64840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:22:38.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.255+0000 7f648ffff700 1 -- 172.21.15.143:0/1661047319 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(49..49 src has 1..49) ==== 5397+0+0 (secure 0 0 0) 0x7f64880e13e0 con 0x7f6498101810 2024-05-08T05:22:38.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.255+0000 7f648dd9b700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f64840b2880 0x7f64840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:22:38.256 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.255+0000 7f648dd9b700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f64840b2880 0x7f64840b4d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6480005950 tx=0x7f648000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:22:38.259 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.258+0000 7f648ffff700 1 -- 172.21.15.143:0/1661047319 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f648801f020 con 0x7f6498101810 2024-05-08T05:22:38.467 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:38.466+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f64980638e0 con 0x7f64840b2880 2024-05-08T05:22:38.467 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi096 2024-05-08T05:22:38.467 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.467 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.467 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:38.467 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:38.467 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi096 to 5696M 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.468 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:38 smithi143 ceph-mon[34337]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi096 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi096 to 5696M 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:38 smithi096 ceph-mon[34253]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T05:22:38.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi096 2024-05-08T05:22:38.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:38.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:38.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi096 to 5696M 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:38.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:38 smithi059 ceph-mon[30304]: osdmap e49: 8 total, 8 up, 8 in 2024-05-08T05:22:39.414 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:39 smithi143 ceph-mon[34337]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:39.414 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:39 smithi143 ceph-mon[34337]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:22:39.414 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:22:39.414 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:22:39.414 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:39 smithi096 ceph-mon[34253]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:39 smithi096 ceph-mon[34253]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:22:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:22:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:22:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:39 smithi059 ceph-mon[30304]: pgmap v176: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:39 smithi059 ceph-mon[30304]: from='client.24280 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:22:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:22:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:22:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:41 smithi096 ceph-mon[34253]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:41.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:41 smithi096 ceph-mon[34253]: from='client.? 172.21.15.143:0/2962253594' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e84c2e9-f309-4779-8e07-0b13fc05d3c1"}]: dispatch 2024-05-08T05:22:41.561 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:41 smithi143 ceph-mon[34337]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:41.561 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:41 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/2962253594' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e84c2e9-f309-4779-8e07-0b13fc05d3c1"}]: dispatch 2024-05-08T05:22:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:41 smithi059 ceph-mon[30304]: pgmap v177: 1 pgs: 1 unknown; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:41 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/2962253594' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e84c2e9-f309-4779-8e07-0b13fc05d3c1"}]: dispatch 2024-05-08T05:22:42.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:42 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e84c2e9-f309-4779-8e07-0b13fc05d3c1"}]: dispatch 2024-05-08T05:22:42.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:42 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e84c2e9-f309-4779-8e07-0b13fc05d3c1"}]': finished 2024-05-08T05:22:42.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:42 smithi096 ceph-mon[34253]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T05:22:42.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:22:42.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:42 smithi096 ceph-mon[34253]: from='client.? 172.21.15.143:0/2517968923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:22:42.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:42 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e84c2e9-f309-4779-8e07-0b13fc05d3c1"}]: dispatch 2024-05-08T05:22:42.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:42 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e84c2e9-f309-4779-8e07-0b13fc05d3c1"}]': finished 2024-05-08T05:22:42.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:42 smithi059 ceph-mon[30304]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T05:22:42.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:22:42.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:42 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/2517968923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:22:42.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:42 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "6e84c2e9-f309-4779-8e07-0b13fc05d3c1"}]: dispatch 2024-05-08T05:22:42.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:42 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "6e84c2e9-f309-4779-8e07-0b13fc05d3c1"}]': finished 2024-05-08T05:22:42.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:42 smithi143 ceph-mon[34337]: osdmap e50: 9 total, 8 up, 9 in 2024-05-08T05:22:42.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:22:42.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:42 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/2517968923' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:22:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:43 smithi096 ceph-mon[34253]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-08T05:22:43.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:43 smithi059 ceph-mon[30304]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-08T05:22:43.644 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:43 smithi143 ceph-mon[34337]: pgmap v179: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 192 KiB/s, 0 objects/s recovering 2024-05-08T05:22:45.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:45 smithi096 ceph-mon[34253]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-05-08T05:22:45.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:45 smithi143 ceph-mon[34337]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-05-08T05:22:45.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:45 smithi059 ceph-mon[30304]: pgmap v180: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 155 KiB/s, 0 objects/s recovering 2024-05-08T05:22:47.448 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:47 smithi143 ceph-mon[34337]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-05-08T05:22:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:47 smithi096 ceph-mon[34253]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-05-08T05:22:47.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:47 smithi059 ceph-mon[30304]: pgmap v181: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 136 KiB/s, 0 objects/s recovering 2024-05-08T05:22:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:49 smithi143 ceph-mon[34337]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:22:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:49 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T05:22:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:49 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:49 smithi096 ceph-mon[34253]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:22:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:49 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T05:22:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:49 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:49.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:49 smithi059 ceph-mon[30304]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:22:49.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:49 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-05-08T05:22:49.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:49 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:50.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:50 smithi143 ceph-mon[34337]: Deploying daemon osd.8 on smithi143 2024-05-08T05:22:50.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:50 smithi096 ceph-mon[34253]: Deploying daemon osd.8 on smithi143 2024-05-08T05:22:50.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:50 smithi059 ceph-mon[30304]: Deploying daemon osd.8 on smithi143 2024-05-08T05:22:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:51 smithi096 ceph-mon[34253]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:22:51.576 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:51 smithi143 ceph-mon[34337]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:22:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:51 smithi059 ceph-mon[30304]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-05-08T05:22:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:53 smithi143 ceph-mon[34337]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:53 smithi096 ceph-mon[34253]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:53 smithi059 ceph-mon[30304]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:55 smithi143 ceph-mon[34337]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:55.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:55.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:55 smithi059 ceph-mon[30304]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:55.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:55 smithi096 ceph-mon[34253]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:55.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:55.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:55.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:57 smithi059 ceph-mon[30304]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:57.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:57.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:57.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:57 smithi143 ceph-mon[34337]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:57.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:57.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:57.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:57.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:57.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:57.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:57 smithi096 ceph-mon[34253]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:57.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:57.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:57.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:22:57.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:22:57.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:58.187 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 8 on host 'smithi143' 2024-05-08T05:22:58.188 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.186+0000 7f648ffff700 1 -- 172.21.15.143:0/1661047319 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f64980638e0 con 0x7f64840b2880 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f64840b2880 msgr2=0x7f64840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f64840b2880 0x7f64840b4d40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6480005950 tx=0x7f648000b410 comp rx=0 tx=0).stop 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498101810 msgr2=0x7f64981996a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498101810 0x7f64981996a0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f648800b530 tx=0x7f648800b840 comp rx=0 tx=0).stop 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 shutdown_connections 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f64840b2880 0x7f64840b4d40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6498105190 0x7f64981a0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6498101810 0x7f64981996a0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 --2- 172.21.15.143:0/1661047319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f64981047f0 0x7f6498199be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 >> 172.21.15.143:0/1661047319 conn(0x7f64980fd170 msgr2=0x7f6498107af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:22:58.190 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 shutdown_connections 2024-05-08T05:22:58.191 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:22:58.189+0000 7f649ce96700 1 -- 172.21.15.143:0/1661047319 wait complete. 2024-05-08T05:22:58.826 DEBUG:teuthology.orchestra.run.smithi143:osd.8> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.8.service 2024-05-08T05:22:58.828 INFO:tasks.cephadm:Deploying osd.9 on smithi059 with /dev/vg_nvme/lv_1... 2024-05-08T05:22:58.829 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T05:22:59.097 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:22:59.449 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:59 smithi143 ceph-mon[34337]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:59.450 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:59 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:59.450 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:59 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:59.450 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:59 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:59.450 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:59 smithi143 ceph-mon[34337]: from='osd.8 [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:22:59.450 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:22:59 smithi143 ceph-mon[34337]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:22:59.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:59 smithi059 ceph-mon[30304]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:59.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:59 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:59.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:59 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:59.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:59 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:59.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:59 smithi059 ceph-mon[30304]: from='osd.8 [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:22:59.463 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:22:59 smithi059 ceph-mon[30304]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:22:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:59 smithi096 ceph-mon[34253]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:22:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:59 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:59.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:59 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:22:59.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:59 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:22:59.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:59 smithi096 ceph-mon[34253]: from='osd.8 [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:22:59.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:22:59 smithi096 ceph-mon[34253]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-05-08T05:23:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:00 smithi059 ceph-mon[30304]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T05:23:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:00 smithi059 ceph-mon[30304]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T05:23:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:00 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:00 smithi059 ceph-mon[30304]: from='osd.8 [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:23:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:00 smithi059 ceph-mon[30304]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:23:00.355 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:00 smithi143 ceph-mon[34337]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T05:23:00.355 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:00 smithi143 ceph-mon[34337]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T05:23:00.355 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:00 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:00.355 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:00 smithi143 ceph-mon[34337]: from='osd.8 [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:23:00.355 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:00 smithi143 ceph-mon[34337]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:23:00.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:00 smithi096 ceph-mon[34253]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-05-08T05:23:00.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:00 smithi096 ceph-mon[34253]: osdmap e51: 9 total, 8 up, 9 in 2024-05-08T05:23:00.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:00 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:00.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:00 smithi096 ceph-mon[34253]: from='osd.8 [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:23:00.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:00 smithi096 ceph-mon[34253]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:23:00.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:23:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:23:00.645+0000 7f7a4529b700 -1 osd.8 0 waiting for initial osdmap 2024-05-08T05:23:00.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:23:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:23:00.655+0000 7f7a3d086700 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:23:01.328 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T05:23:01.328 INFO:teuthology.orchestra.run.smithi059.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T05:23:01.329 INFO:teuthology.orchestra.run.smithi059.stderr: stderr: 10+0 records in 2024-05-08T05:23:01.329 INFO:teuthology.orchestra.run.smithi059.stderr:10+0 records out 2024-05-08T05:23:01.329 INFO:teuthology.orchestra.run.smithi059.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0129327 s, 811 MB/s 2024-05-08T05:23:01.329 INFO:teuthology.orchestra.run.smithi059.stderr:--> Zapping successful for: 2024-05-08T05:23:01.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:01 smithi059 ceph-mon[30304]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:23:01.355 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:01 smithi059 ceph-mon[30304]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:23:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:01 smithi059 ceph-mon[30304]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T05:23:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:01 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:01 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:01 smithi059 ceph-mon[30304]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T05:23:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:01 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:01.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:01 smithi059 ceph-mon[30304]: from='osd.8 ' entity='osd.8' 2024-05-08T05:23:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:01 smithi143 ceph-mon[34337]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:23:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:01 smithi143 ceph-mon[34337]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:23:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:01 smithi143 ceph-mon[34337]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T05:23:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:01 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:01 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:01 smithi143 ceph-mon[34337]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T05:23:01.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:01 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:01.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:01 smithi143 ceph-mon[34337]: from='osd.8 ' entity='osd.8' 2024-05-08T05:23:01.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:01 smithi096 ceph-mon[34253]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:23:01.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:01 smithi096 ceph-mon[34253]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:23:01.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:01 smithi096 ceph-mon[34253]: osdmap e52: 9 total, 8 up, 9 in 2024-05-08T05:23:01.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:01 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:01.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:01 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:01.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:01 smithi096 ceph-mon[34253]: osdmap e53: 9 total, 8 up, 9 in 2024-05-08T05:23:01.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:01 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:01.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:01 smithi096 ceph-mon[34253]: from='osd.8 ' entity='osd.8' 2024-05-08T05:23:01.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi059:vg_nvme/lv_1 2024-05-08T05:23:02.239 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:23:02.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:23:02.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:23:02.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:02.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: osd.8 [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] boot 2024-05-08T05:23:02.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T05:23:02.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:02.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:02.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:02.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:02.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:02.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:02.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:23:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:23:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: osd.8 [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] boot 2024-05-08T05:23:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T05:23:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:02.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:02.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:02.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:02.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:02.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: osd.8 [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] boot 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: osdmap e54: 9 total, 9 up, 9 in 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:02.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:02.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:02.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:02.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:03 smithi059 ceph-mon[30304]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:23:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:03 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi143 2024-05-08T05:23:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:03 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi143 to 5696M 2024-05-08T05:23:03.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:03 smithi059 ceph-mon[30304]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T05:23:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:03 smithi096 ceph-mon[34253]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:23:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:03 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi143 2024-05-08T05:23:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:03 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi143 to 5696M 2024-05-08T05:23:03.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:03 smithi096 ceph-mon[34253]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T05:23:03.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 -- 172.21.15.59:0/1114388573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c1050c0 msgr2=0x7f614c10b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:03.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 --2- 172.21.15.59:0/1114388573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c1050c0 0x7f614c10b970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f614400b0d0 tx=0x7f614400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:23:03.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 -- 172.21.15.59:0/1114388573 shutdown_connections 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 --2- 172.21.15.59:0/1114388573 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c1050c0 0x7f614c10b970 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 --2- 172.21.15.59:0/1114388573 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f614c103520 0x7f614c103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 --2- 172.21.15.59:0/1114388573 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f614c104720 0x7f614c104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 -- 172.21.15.59:0/1114388573 >> 172.21.15.59:0/1114388573 conn(0x7f614c0feac0 msgr2=0x7f614c100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 -- 172.21.15.59:0/1114388573 shutdown_connections 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 -- 172.21.15.59:0/1114388573 wait complete. 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.842+0000 7f6152ff7700 1 Processor -- start 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6152ff7700 1 -- start start 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6152ff7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c103520 0x7f614c199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6152ff7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f614c104720 0x7f614c199b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6152ff7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f614c1050c0 0x7f614c1a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6152ff7700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f614c10f040 con 0x7f614c103520 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6152ff7700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f614c10eec0 con 0x7f614c104720 2024-05-08T05:23:03.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6152ff7700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f614c10f1c0 con 0x7f614c1050c0 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6150d93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c103520 0x7f614c199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6150d93700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c103520 0x7f614c199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52140/0 (socket says 172.21.15.59:52140) 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6150d93700 1 -- 172.21.15.59:0/372918699 learned_addr learned my addr 172.21.15.59:0/372918699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6151594700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f614c1050c0 0x7f614c1a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f614bfff700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f614c104720 0x7f614c199b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6150d93700 1 -- 172.21.15.59:0/372918699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f614c1050c0 msgr2=0x7f614c1a0bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6150d93700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f614c1050c0 0x7f614c1a0bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6150d93700 1 -- 172.21.15.59:0/372918699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f614c104720 msgr2=0x7f614c199b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6150d93700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f614c104720 0x7f614c199b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6150d93700 1 -- 172.21.15.59:0/372918699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f614c1a1290 con 0x7f614c103520 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.843+0000 7f6150d93700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c103520 0x7f614c199630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f614000b530 tx=0x7f614000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.844+0000 7f6149ffb700 1 -- 172.21.15.59:0/372918699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140015020 con 0x7f614c103520 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.844+0000 7f6149ffb700 1 -- 172.21.15.59:0/372918699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6140011ce0 con 0x7f614c103520 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.844+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f614c1a1580 con 0x7f614c103520 2024-05-08T05:23:03.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.844+0000 7f6149ffb700 1 -- 172.21.15.59:0/372918699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6140010d60 con 0x7f614c103520 2024-05-08T05:23:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.844+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f614c1a1b40 con 0x7f614c103520 2024-05-08T05:23:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.845+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f614c06a190 con 0x7f614c103520 2024-05-08T05:23:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.846+0000 7f6149ffb700 1 -- 172.21.15.59:0/372918699 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6140018020 con 0x7f614c103520 2024-05-08T05:23:03.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.847+0000 7f6149ffb700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f613c0b2880 0x7f613c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.847+0000 7f614bfff700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f613c0b2880 0x7f613c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.847+0000 7f6149ffb700 1 -- 172.21.15.59:0/372918699 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(56..56 src has 1..56) ==== 5829+0+0 (secure 0 0 0) 0x7f61400e1030 con 0x7f614c103520 2024-05-08T05:23:03.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.847+0000 7f614bfff700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f613c0b2880 0x7f613c0b4d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6138005ea0 tx=0x7f6138005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:23:03.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:03.852+0000 7f6149ffb700 1 -- 172.21.15.59:0/372918699 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f614001f020 con 0x7f614c103520 2024-05-08T05:23:03.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:03 smithi143 ceph-mon[34337]: pgmap v193: 1 pgs: 1 remapped+peering; 577 KiB data, 231 MiB used, 715 GiB / 715 GiB avail 2024-05-08T05:23:03.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:03 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi143 2024-05-08T05:23:03.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:03 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi143 to 5696M 2024-05-08T05:23:03.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:03 smithi143 ceph-mon[34337]: osdmap e55: 9 total, 9 up, 9 in 2024-05-08T05:23:04.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:04.074+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f614c0638e0 con 0x7f613c0b2880 2024-05-08T05:23:04.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:04 smithi059 ceph-mon[30304]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T05:23:04.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:23:04.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:23:04.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:04.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:04 smithi096 ceph-mon[34253]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T05:23:04.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:23:04.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:23:04.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:04.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:04 smithi143 ceph-mon[34337]: osdmap e56: 9 total, 9 up, 9 in 2024-05-08T05:23:04.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:23:04.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:23:04.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:05.761 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:05 smithi059 ceph-mon[30304]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:05.761 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:05 smithi059 ceph-mon[30304]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:23:05.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:05 smithi096 ceph-mon[34253]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:05.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:05 smithi096 ceph-mon[34253]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:23:05.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:05 smithi143 ceph-mon[34337]: pgmap v196: 1 pgs: 1 remapped+peering; 577 KiB data, 262 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:05.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:05 smithi143 ceph-mon[34337]: from='client.14412 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi059:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:23:07.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:07 smithi096 ceph-mon[34253]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-05-08T05:23:07.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:07 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1745818071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6035037-1d0b-49dd-ae6c-2708beba9c2b"}]: dispatch 2024-05-08T05:23:07.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:07 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1745818071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6035037-1d0b-49dd-ae6c-2708beba9c2b"}]': finished 2024-05-08T05:23:07.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:07 smithi096 ceph-mon[34253]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T05:23:07.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:07 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:07.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:07 smithi059 ceph-mon[30304]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-05-08T05:23:07.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:07 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1745818071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6035037-1d0b-49dd-ae6c-2708beba9c2b"}]: dispatch 2024-05-08T05:23:07.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:07 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1745818071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6035037-1d0b-49dd-ae6c-2708beba9c2b"}]': finished 2024-05-08T05:23:07.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:07 smithi059 ceph-mon[30304]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T05:23:07.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:07 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:07 smithi143 ceph-mon[34337]: pgmap v197: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 106 KiB/s, 0 objects/s recovering 2024-05-08T05:23:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:07 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1745818071' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "d6035037-1d0b-49dd-ae6c-2708beba9c2b"}]: dispatch 2024-05-08T05:23:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:07 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1745818071' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "d6035037-1d0b-49dd-ae6c-2708beba9c2b"}]': finished 2024-05-08T05:23:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:07 smithi143 ceph-mon[34337]: osdmap e57: 10 total, 9 up, 10 in 2024-05-08T05:23:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:07 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:08.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:08 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3728677944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:23:08.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:08 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3728677944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:23:08.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:08 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3728677944' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:23:09.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:09 smithi096 ceph-mon[34253]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:23:09.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:09 smithi059 ceph-mon[30304]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:23:09.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:09 smithi143 ceph-mon[34337]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-05-08T05:23:11.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:11 smithi096 ceph-mon[34253]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T05:23:11.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:11 smithi059 ceph-mon[30304]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T05:23:11.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:11 smithi143 ceph-mon[34337]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 77 KiB/s, 0 objects/s recovering 2024-05-08T05:23:13.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:13 smithi096 ceph-mon[34253]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T05:23:13.795 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:13 smithi143 ceph-mon[34337]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T05:23:13.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:13 smithi059 ceph-mon[30304]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 68 KiB/s, 0 objects/s recovering 2024-05-08T05:23:15.679 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:15 smithi059 ceph-mon[30304]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:23:15.679 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:15 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T05:23:15.679 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:15 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:15.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:15 smithi096 ceph-mon[34253]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:23:15.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:15 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T05:23:15.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:15 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:15.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:15 smithi143 ceph-mon[34337]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:23:15.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:15 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-05-08T05:23:15.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:15 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:16.787 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:16 smithi059 ceph-mon[30304]: Deploying daemon osd.9 on smithi059 2024-05-08T05:23:16.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:16 smithi143 ceph-mon[34337]: Deploying daemon osd.9 on smithi059 2024-05-08T05:23:17.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:16 smithi096 ceph-mon[34253]: Deploying daemon osd.9 on smithi059 2024-05-08T05:23:17.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:17 smithi059 ceph-mon[30304]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:17.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:17 smithi143 ceph-mon[34337]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:18.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:17 smithi096 ceph-mon[34253]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:19.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:19 smithi059 ceph-mon[30304]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:19.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:19 smithi143 ceph-mon[34337]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:20.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:19 smithi096 ceph-mon[34253]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:21.813 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:21 smithi059 ceph-mon[30304]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:21.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:21.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:21.814 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:21.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:21 smithi143 ceph-mon[34337]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:21.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:21.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:21.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:22.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:21 smithi096 ceph-mon[34253]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:22.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:22.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:22.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:22.832 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:22 smithi059 ceph-mon[30304]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:22.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:22 smithi143 ceph-mon[34337]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:23.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:22 smithi096 ceph-mon[34253]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:24.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.738+0000 7f6149ffb700 1 -- 172.21.15.59:0/372918699 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f614c0638e0 con 0x7f613c0b2880 2024-05-08T05:23:24.739 INFO:teuthology.orchestra.run.smithi059.stdout:Created osd(s) 9 on host 'smithi059' 2024-05-08T05:23:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f613c0b2880 msgr2=0x7f613c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f613c0b2880 0x7f613c0b4d40 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f6138005ea0 tx=0x7f6138005e30 comp rx=0 tx=0).stop 2024-05-08T05:23:24.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c103520 msgr2=0x7f614c199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c103520 0x7f614c199630 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f614000b530 tx=0x7f614000b840 comp rx=0 tx=0).stop 2024-05-08T05:23:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 shutdown_connections 2024-05-08T05:23:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f614c1050c0 0x7f614c1a0bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f614c103520 0x7f614c199630 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f613c0b2880 0x7f613c0b4d40 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 --2- 172.21.15.59:0/372918699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f614c104720 0x7f614c199b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 >> 172.21.15.59:0/372918699 conn(0x7f614c0feac0 msgr2=0x7f614c1079c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:23:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 shutdown_connections 2024-05-08T05:23:24.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:23:24.741+0000 7f6152ff7700 1 -- 172.21.15.59:0/372918699 wait complete. 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T05:23:25.061 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T05:23:25.070 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-05-08T05:23:25.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:25.405 DEBUG:teuthology.orchestra.run.smithi059:osd.9> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.9.service 2024-05-08T05:23:25.407 INFO:tasks.cephadm:Deploying osd.10 on smithi096 with /dev/vg_nvme/lv_1... 2024-05-08T05:23:25.407 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T05:23:25.654 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:23:26.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:26 smithi096 ceph-mon[34253]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T05:23:26.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:26 smithi096 ceph-mon[34253]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T05:23:26.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:26 smithi096 ceph-mon[34253]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:23:26.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:26 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:26.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:26 smithi096 ceph-mon[34253]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:26.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:26 smithi059 ceph-mon[30304]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T05:23:26.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:26 smithi059 ceph-mon[30304]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T05:23:26.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:26 smithi059 ceph-mon[30304]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:23:26.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:26 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:26.960 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:26 smithi059 ceph-mon[30304]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:26.960 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:23:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:23:26.709+0000 7fae325f7700 -1 osd.9 0 waiting for initial osdmap 2024-05-08T05:23:26.960 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:23:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:23:26.729+0000 7fae2e3ea700 -1 osd.9 59 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:23:26.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:26 smithi143 ceph-mon[34337]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-05-08T05:23:26.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:26 smithi143 ceph-mon[34337]: osdmap e58: 10 total, 9 up, 10 in 2024-05-08T05:23:26.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:26 smithi143 ceph-mon[34337]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]: dispatch 2024-05-08T05:23:26.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:26 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:26.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:26 smithi143 ceph-mon[34337]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:27.804 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:27 smithi096 ceph-mon[34253]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:23:27.804 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:27 smithi096 ceph-mon[34253]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T05:23:27.804 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:27 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:27.804 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:27 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:27.804 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T05:23:27.804 INFO:teuthology.orchestra.run.smithi096.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T05:23:27.804 INFO:teuthology.orchestra.run.smithi096.stderr: stderr: 10+0 records in 2024-05-08T05:23:27.804 INFO:teuthology.orchestra.run.smithi096.stderr:10+0 records out 2024-05-08T05:23:27.804 INFO:teuthology.orchestra.run.smithi096.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114785 s, 914 MB/s 2024-05-08T05:23:27.805 INFO:teuthology.orchestra.run.smithi096.stderr:--> Zapping successful for: 2024-05-08T05:23:27.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:27 smithi143 ceph-mon[34337]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:23:27.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:27 smithi143 ceph-mon[34337]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T05:23:27.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:27 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:27.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:27 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:28.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:27 smithi059 ceph-mon[30304]: from='osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi059", "root=default"]}]': finished 2024-05-08T05:23:28.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:27 smithi059 ceph-mon[30304]: osdmap e59: 10 total, 9 up, 10 in 2024-05-08T05:23:28.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:27 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:28.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:27 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:28.471 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi096:vg_nvme/lv_1 2024-05-08T05:23:28.713 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:23:28.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:28 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:23:28.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:28 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:23:28.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:28 smithi096 ceph-mon[34253]: osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] boot 2024-05-08T05:23:28.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:28 smithi096 ceph-mon[34253]: osdmap e60: 10 total, 10 up, 10 in 2024-05-08T05:23:28.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:28 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:28.987 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:28 smithi096 ceph-mon[34253]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:28.987 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:28 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:28 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:23:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:28 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:23:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:28 smithi143 ceph-mon[34337]: osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] boot 2024-05-08T05:23:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:28 smithi143 ceph-mon[34337]: osdmap e60: 10 total, 10 up, 10 in 2024-05-08T05:23:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:28 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:28.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:28 smithi143 ceph-mon[34337]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:28.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:28 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:29.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:28 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:23:29.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:28 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:23:29.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:28 smithi059 ceph-mon[30304]: osd.9 [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] boot 2024-05-08T05:23:29.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:28 smithi059 ceph-mon[30304]: osdmap e60: 10 total, 10 up, 10 in 2024-05-08T05:23:29.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:28 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:23:29.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:28 smithi059 ceph-mon[30304]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 263 MiB used, 804 GiB / 805 GiB avail 2024-05-08T05:23:29.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:28 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:29.985 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi059 2024-05-08T05:23:29.985 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:29.985 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:29.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:29.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:29.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:29.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi059 to 4273M 2024-05-08T05:23:29.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:29.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:29.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:29.986 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:29 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi059 2024-05-08T05:23:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi059 to 4273M 2024-05-08T05:23:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:29.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:29 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:30.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi059 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi059 to 4273M 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:30.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:29 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 -- 172.21.15.96:0/3833245499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81068b0 msgr2=0x7f09f8108ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 --2- 172.21.15.96:0/3833245499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81068b0 0x7f09f8108ca0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09e8009990 tx=0x7f09e8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:23:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 -- 172.21.15.96:0/3833245499 shutdown_connections 2024-05-08T05:23:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 --2- 172.21.15.96:0/3833245499 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f81091e0 0x7f09f810b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 --2- 172.21.15.96:0/3833245499 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f8103f80 0x7f09f8106370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 --2- 172.21.15.96:0/3833245499 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81068b0 0x7f09f8108ca0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 -- 172.21.15.96:0/3833245499 >> 172.21.15.96:0/3833245499 conn(0x7f09f80fd990 msgr2=0x7f09f80ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:23:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 -- 172.21.15.96:0/3833245499 shutdown_connections 2024-05-08T05:23:30.543 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 -- 172.21.15.96:0/3833245499 wait complete. 2024-05-08T05:23:30.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.543+0000 7f09fde7c700 1 Processor -- start 2024-05-08T05:23:30.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09fde7c700 1 -- start start 2024-05-08T05:23:30.545 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09fde7c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f8103f80 0x7f09f81038e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09fde7c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f81068b0 0x7f09f8101f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09fde7c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81091e0 0x7f09f8102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09fde7c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810efb0 con 0x7f09f8103f80 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09fde7c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810ee30 con 0x7f09f81091e0 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09fde7c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f09f810f130 con 0x7f09f81068b0 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81091e0 0x7f09f8102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f7fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81091e0 0x7f09f8102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.96:51234/0 (socket says 172.21.15.96:51234) 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f7fff700 1 -- 172.21.15.96:0/2175180558 learned_addr learned my addr 172.21.15.96:0/2175180558 (peer_addr_for_me v2:172.21.15.96:0/0) 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f6ffd700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f81068b0 0x7f09f8101f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f77fe700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f8103f80 0x7f09f81038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f7fff700 1 -- 172.21.15.96:0/2175180558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f81068b0 msgr2=0x7f09f8101f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f7fff700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f81068b0 0x7f09f8101f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f7fff700 1 -- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f8103f80 msgr2=0x7f09f81038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:30.546 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f7fff700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f8103f80 0x7f09f81038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f7fff700 1 -- 172.21.15.96:0/2175180558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09f8102d20 con 0x7f09f81091e0 2024-05-08T05:23:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.544+0000 7f09f7fff700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81091e0 0x7f09f8102470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09f00094f0 tx=0x7f09f000d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:23:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.545+0000 7f09f4ff9700 1 -- 172.21.15.96:0/2175180558 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f000ddd0 con 0x7f09f81091e0 2024-05-08T05:23:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.545+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09f81a37a0 con 0x7f09f81091e0 2024-05-08T05:23:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.545+0000 7f09f4ff9700 1 -- 172.21.15.96:0/2175180558 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09f0009e20 con 0x7f09f81091e0 2024-05-08T05:23:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.545+0000 7f09f4ff9700 1 -- 172.21.15.96:0/2175180558 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0015400 con 0x7f09f81091e0 2024-05-08T05:23:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.545+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09f81a3cb0 con 0x7f09f81091e0 2024-05-08T05:23:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.546+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f09f806a190 con 0x7f09f81091e0 2024-05-08T05:23:30.547 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.548+0000 7f09f4ff9700 1 -- 172.21.15.96:0/2175180558 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f09f0004dd0 con 0x7f09f81091e0 2024-05-08T05:23:30.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.548+0000 7f09f4ff9700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f09e40b2880 0x7f09e40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:30.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.548+0000 7f09f4ff9700 1 -- 172.21.15.96:0/2175180558 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(61..61 src has 1..61) ==== 6261+0+0 (secure 0 0 0) 0x7f09f00e1d20 con 0x7f09f81091e0 2024-05-08T05:23:30.548 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.549+0000 7f09f77fe700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f09e40b2880 0x7f09e40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:30.549 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.549+0000 7f09f77fe700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f09e40b2880 0x7f09e40b4d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f09e0005950 tx=0x7f09e00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:23:30.552 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.552+0000 7f09f4ff9700 1 -- 172.21.15.96:0/2175180558 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f09f0010070 con 0x7f09f81091e0 2024-05-08T05:23:30.763 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:30.763+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f09f80638e0 con 0x7f09e40b2880 2024-05-08T05:23:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:30 smithi143 ceph-mon[34337]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T05:23:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:30 smithi143 ceph-mon[34337]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:31.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:30 smithi096 ceph-mon[34253]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T05:23:31.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:30 smithi096 ceph-mon[34253]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:31.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:30 smithi059 ceph-mon[30304]: osdmap e61: 10 total, 10 up, 10 in 2024-05-08T05:23:31.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:30 smithi059 ceph-mon[30304]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:31.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:31 smithi143 ceph-mon[34337]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:23:31.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:31 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:23:31.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:31 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:23:31.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:31 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:32.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:31 smithi096 ceph-mon[34253]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:23:32.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:31 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:23:32.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:31 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:23:32.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:31 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:32.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:31 smithi059 ceph-mon[30304]: from='client.24332 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi096:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:23:32.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:31 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:23:32.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:31 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:23:32.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:31 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:32 smithi143 ceph-mon[34337]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:33.022 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:32 smithi096 ceph-mon[34253]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:33.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:32 smithi059 ceph-mon[30304]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:34.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:33 smithi096 ceph-mon[34253]: from='client.? 172.21.15.96:0/1308067226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18a53f85-31e5-4969-b393-9dfc48d9aaa3"}]: dispatch 2024-05-08T05:23:34.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:33 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18a53f85-31e5-4969-b393-9dfc48d9aaa3"}]: dispatch 2024-05-08T05:23:34.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:33 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18a53f85-31e5-4969-b393-9dfc48d9aaa3"}]': finished 2024-05-08T05:23:34.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:33 smithi096 ceph-mon[34253]: osdmap e62: 11 total, 10 up, 11 in 2024-05-08T05:23:34.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:34.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:33 smithi059 ceph-mon[30304]: from='client.? 172.21.15.96:0/1308067226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18a53f85-31e5-4969-b393-9dfc48d9aaa3"}]: dispatch 2024-05-08T05:23:34.096 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:33 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18a53f85-31e5-4969-b393-9dfc48d9aaa3"}]: dispatch 2024-05-08T05:23:34.096 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:33 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18a53f85-31e5-4969-b393-9dfc48d9aaa3"}]': finished 2024-05-08T05:23:34.096 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:33 smithi059 ceph-mon[30304]: osdmap e62: 11 total, 10 up, 11 in 2024-05-08T05:23:34.096 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:34.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:33 smithi143 ceph-mon[34337]: from='client.? 172.21.15.96:0/1308067226' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18a53f85-31e5-4969-b393-9dfc48d9aaa3"}]: dispatch 2024-05-08T05:23:34.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:33 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "18a53f85-31e5-4969-b393-9dfc48d9aaa3"}]: dispatch 2024-05-08T05:23:34.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:33 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "18a53f85-31e5-4969-b393-9dfc48d9aaa3"}]': finished 2024-05-08T05:23:34.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:33 smithi143 ceph-mon[34337]: osdmap e62: 11 total, 10 up, 11 in 2024-05-08T05:23:34.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:35.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:34 smithi096 ceph-mon[34253]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:35.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:34 smithi096 ceph-mon[34253]: from='client.? 172.21.15.96:0/1413697225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:23:35.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:34 smithi059 ceph-mon[30304]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:35.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:34 smithi059 ceph-mon[30304]: from='client.? 172.21.15.96:0/1413697225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:23:35.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:34 smithi143 ceph-mon[34337]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:35.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:34 smithi143 ceph-mon[34337]: from='client.? 172.21.15.96:0/1413697225' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:23:37.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:37 smithi096 ceph-mon[34253]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:37.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:37 smithi059 ceph-mon[30304]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:37 smithi143 ceph-mon[34337]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:39.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:39 smithi096 ceph-mon[34253]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:39.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:39 smithi059 ceph-mon[30304]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:39 smithi143 ceph-mon[34337]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:41.119 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:41 smithi096 ceph-mon[34253]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:41 smithi059 ceph-mon[30304]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:41 smithi143 ceph-mon[34337]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:42.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T05:23:42.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:42 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:42.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:42 smithi096 ceph-mon[34253]: Deploying daemon osd.10 on smithi096 2024-05-08T05:23:42.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T05:23:42.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:42 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:42.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:42 smithi059 ceph-mon[30304]: Deploying daemon osd.10 on smithi096 2024-05-08T05:23:42.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-05-08T05:23:42.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:42 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:42.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:42 smithi143 ceph-mon[34337]: Deploying daemon osd.10 on smithi096 2024-05-08T05:23:43.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:43 smithi096 ceph-mon[34253]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:43 smithi059 ceph-mon[30304]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:43.394 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:43 smithi143 ceph-mon[34337]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:45.272 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:45 smithi096 ceph-mon[34253]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:45 smithi059 ceph-mon[30304]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:45 smithi143 ceph-mon[34337]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:47.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:47 smithi096 ceph-mon[34253]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:47 smithi059 ceph-mon[30304]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:47 smithi143 ceph-mon[34337]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:48.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:48.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:48.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:48.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:48.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:48.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:48.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:48.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:48.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:49.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:49 smithi059 ceph-mon[30304]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:49.405 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:49 smithi096 ceph-mon[34253]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:49 smithi143 ceph-mon[34337]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:51.036 INFO:teuthology.orchestra.run.smithi096.stdout:Created osd(s) 10 on host 'smithi096' 2024-05-08T05:23:51.036 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.034+0000 7f09f4ff9700 1 -- 172.21.15.96:0/2175180558 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f09f80638e0 con 0x7f09e40b2880 2024-05-08T05:23:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f09e40b2880 msgr2=0x7f09e40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f09e40b2880 0x7f09e40b4d40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f09e0005950 tx=0x7f09e00058e0 comp rx=0 tx=0).stop 2024-05-08T05:23:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81091e0 msgr2=0x7f09f8102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:51.039 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81091e0 0x7f09f8102470 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f09f00094f0 tx=0x7f09f000d3b0 comp rx=0 tx=0).stop 2024-05-08T05:23:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 shutdown_connections 2024-05-08T05:23:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f09e40b2880 0x7f09e40b4d40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f09f81091e0 0x7f09f8102470 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f09f8103f80 0x7f09f81038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 --2- 172.21.15.96:0/2175180558 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f09f81068b0 0x7f09f8101f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.037+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 >> 172.21.15.96:0/2175180558 conn(0x7f09f80fd990 msgr2=0x7f09f8104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:23:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.038+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 shutdown_connections 2024-05-08T05:23:51.040 INFO:teuthology.orchestra.run.smithi096.stderr:2024-05-08T05:23:51.038+0000 7f09fde7c700 1 -- 172.21.15.96:0/2175180558 wait complete. 2024-05-08T05:23:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='osd.10 [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:23:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:23:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='osd.10 [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:23:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:23:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='osd.10 [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:23:51.681 DEBUG:teuthology.orchestra.run.smithi096:osd.10> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.10.service 2024-05-08T05:23:51.683 INFO:tasks.cephadm:Deploying osd.11 on smithi143 with /dev/vg_nvme/lv_1... 2024-05-08T05:23:51.684 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-05-08T05:23:51.929 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:23:52.389 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:52 smithi143 ceph-mon[34337]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T05:23:52.389 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:52 smithi143 ceph-mon[34337]: from='osd.10 [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:23:52.389 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:52 smithi143 ceph-mon[34337]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T05:23:52.389 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:52 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:52.389 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:52 smithi143 ceph-mon[34337]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:23:52.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:52 smithi096 ceph-mon[34253]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T05:23:52.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:52 smithi096 ceph-mon[34253]: from='osd.10 [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:23:52.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:52 smithi096 ceph-mon[34253]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T05:23:52.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:52 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:52.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:52 smithi096 ceph-mon[34253]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:23:52.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:23:52 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:23:52.261+0000 7f296532a700 -1 osd.10 0 waiting for initial osdmap 2024-05-08T05:23:52.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:23:52 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:23:52.272+0000 7f296111d700 -1 osd.10 64 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:23:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:52 smithi059 ceph-mon[30304]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-05-08T05:23:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:52 smithi059 ceph-mon[30304]: from='osd.10 [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:23:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:52 smithi059 ceph-mon[30304]: osdmap e63: 11 total, 10 up, 11 in 2024-05-08T05:23:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:52 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:52.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:52 smithi059 ceph-mon[30304]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]: dispatch 2024-05-08T05:23:53.473 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:53 smithi143 ceph-mon[34337]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:53.473 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:53 smithi143 ceph-mon[34337]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:23:53.473 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:53 smithi143 ceph-mon[34337]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T05:23:53.473 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:53.473 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:53 smithi096 ceph-mon[34253]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:53.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:53 smithi096 ceph-mon[34253]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:23:53.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:53 smithi096 ceph-mon[34253]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T05:23:53.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:53.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:53 smithi059 ceph-mon[30304]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 290 MiB used, 894 GiB / 894 GiB avail 2024-05-08T05:23:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:53 smithi059 ceph-mon[30304]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi096", "root=default"]}]': finished 2024-05-08T05:23:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:53 smithi059 ceph-mon[30304]: osdmap e64: 11 total, 10 up, 11 in 2024-05-08T05:23:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-05-08T05:23:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-05-08T05:23:54.043 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10+0 records in 2024-05-08T05:23:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:10+0 records out 2024-05-08T05:23:54.043 INFO:teuthology.orchestra.run.smithi143.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113163 s, 927 MB/s 2024-05-08T05:23:54.043 INFO:teuthology.orchestra.run.smithi143.stderr:--> Zapping successful for: 2024-05-08T05:23:54.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:54 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:23:54.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:54 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:23:54.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:54 smithi096 ceph-mon[34253]: osd.10 [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] boot 2024-05-08T05:23:54.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:54 smithi096 ceph-mon[34253]: osdmap e65: 11 total, 11 up, 11 in 2024-05-08T05:23:54.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:54 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:54.341 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:54 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:23:54.341 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:54 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:23:54.341 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:54 smithi143 ceph-mon[34337]: osd.10 [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] boot 2024-05-08T05:23:54.341 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:54 smithi143 ceph-mon[34337]: osdmap e65: 11 total, 11 up, 11 in 2024-05-08T05:23:54.341 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:54.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:54 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:23:54.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:54 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:23:54.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:54 smithi059 ceph-mon[30304]: osd.10 [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] boot 2024-05-08T05:23:54.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:54 smithi059 ceph-mon[30304]: osdmap e65: 11 total, 11 up, 11 in 2024-05-08T05:23:54.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:23:54.721 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch daemon add osd smithi143:vg_nvme/lv_1 2024-05-08T05:23:54.968 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:23:55.424 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:23:55.424 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T05:23:55.424 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.424 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.424 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.424 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.424 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.425 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.425 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.425 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:55.425 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:55.425 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.425 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:55 smithi143 ceph-mon[34337]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T05:23:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:23:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T05:23:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:55.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:55.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:55 smithi096 ceph-mon[34253]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T05:23:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 317 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:23:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: osdmap e66: 11 total, 11 up, 11 in 2024-05-08T05:23:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:23:55.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:55.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:23:55.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:23:55.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:55 smithi059 ceph-mon[30304]: osdmap e67: 11 total, 11 up, 11 in 2024-05-08T05:23:56.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:56 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi096 2024-05-08T05:23:56.490 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:56 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi096 to 4272M 2024-05-08T05:23:56.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:56 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi096 2024-05-08T05:23:56.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:56 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi096 to 4272M 2024-05-08T05:23:56.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:56 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi096 2024-05-08T05:23:56.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:56 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi096 to 4272M 2024-05-08T05:23:56.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.713+0000 7f1dec832700 1 -- 172.21.15.143:0/3146973608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8109210 msgr2=0x7f1de810b680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:56.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.713+0000 7f1dec832700 1 --2- 172.21.15.143:0/3146973608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8109210 0x7f1de810b680 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f1de000b0a0 tx=0x7f1de000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:23:56.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.713+0000 7f1dec832700 1 -- 172.21.15.143:0/3146973608 shutdown_connections 2024-05-08T05:23:56.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.713+0000 7f1dec832700 1 --2- 172.21.15.143:0/3146973608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8109210 0x7f1de810b680 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:56.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.713+0000 7f1dec832700 1 --2- 172.21.15.143:0/3146973608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1de8103fb0 0x7f1de81063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:56.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.713+0000 7f1dec832700 1 --2- 172.21.15.143:0/3146973608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1de81068e0 0x7f1de8108cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:56.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.713+0000 7f1dec832700 1 -- 172.21.15.143:0/3146973608 >> 172.21.15.143:0/3146973608 conn(0x7f1de80fd9c0 msgr2=0x7f1de80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:23:56.714 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.713+0000 7f1dec832700 1 -- 172.21.15.143:0/3146973608 shutdown_connections 2024-05-08T05:23:56.715 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.713+0000 7f1dec832700 1 -- 172.21.15.143:0/3146973608 wait complete. 2024-05-08T05:23:56.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1dec832700 1 Processor -- start 2024-05-08T05:23:56.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1dec832700 1 -- start start 2024-05-08T05:23:56.716 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1dec832700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8103fb0 0x7f1de81995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1dec832700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1de81068e0 0x7f1de8199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1dec832700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1de8109210 0x7f1de81a0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1dec832700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1de810ef80 con 0x7f1de81068e0 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1dec832700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1de810ee00 con 0x7f1de8109210 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1dec832700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1de810f100 con 0x7f1de8103fb0 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8103fb0 0x7f1de81995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de659c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8103fb0 0x7f1de81995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:35560/0 (socket says 172.21.15.143:35560) 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de659c700 1 -- 172.21.15.143:0/1600498592 learned_addr learned my addr 172.21.15.143:0/1600498592 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1dddd9b700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1de81068e0 0x7f1de8199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de6d9d700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1de8109210 0x7f1de81a0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de659c700 1 -- 172.21.15.143:0/1600498592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1de8109210 msgr2=0x7f1de81a0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:56.717 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de659c700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1de8109210 0x7f1de81a0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de659c700 1 -- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1de81068e0 msgr2=0x7f1de8199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de659c700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1de81068e0 0x7f1de8199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de659c700 1 -- 172.21.15.143:0/1600498592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1de81a1200 con 0x7f1de8103fb0 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.714+0000 7f1de6d9d700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1de8109210 0x7f1de81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.715+0000 7f1de659c700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8103fb0 0x7f1de81995a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800e9d0 tx=0x7f1dd800ece0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.715+0000 7f1ddffff700 1 -- 172.21.15.143:0/1600498592 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800cbd0 con 0x7f1de8103fb0 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.715+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1de81a14f0 con 0x7f1de8103fb0 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.715+0000 7f1ddffff700 1 -- 172.21.15.143:0/1600498592 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dd800cd70 con 0x7f1de8103fb0 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.715+0000 7f1ddffff700 1 -- 172.21.15.143:0/1600498592 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8018680 con 0x7f1de8103fb0 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.715+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1de81a1a30 con 0x7f1de8103fb0 2024-05-08T05:23:56.718 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.716+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1de806a190 con 0x7f1de8103fb0 2024-05-08T05:23:56.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.717+0000 7f1ddffff700 1 -- 172.21.15.143:0/1600498592 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f1dd8010430 con 0x7f1de8103fb0 2024-05-08T05:23:56.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.718+0000 7f1ddffff700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1dd40b2880 0x7f1dd40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:23:56.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.718+0000 7f1dddd9b700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1dd40b2880 0x7f1dd40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:23:56.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.718+0000 7f1ddffff700 1 -- 172.21.15.143:0/1600498592 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(67..67 src has 1..67) ==== 6694+0+0 (secure 0 0 0) 0x7f1dd80e22f0 con 0x7f1de8103fb0 2024-05-08T05:23:56.719 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.719+0000 7f1dddd9b700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1dd40b2880 0x7f1dd40b4d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0005950 tx=0x7f1dd000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:23:56.722 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.722+0000 7f1ddffff700 1 -- 172.21.15.143:0/1600498592 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1dd80aa640 con 0x7f1de8103fb0 2024-05-08T05:23:56.934 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:23:56.934+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f1de80638e0 con 0x7f1dd40b2880 2024-05-08T05:23:57.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:57 smithi096 ceph-mon[34253]: pgmap v233: 1 pgs: 1 activating+degraded; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T05:23:57.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:57 smithi096 ceph-mon[34253]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-08T05:23:57.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:23:57.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:23:57.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:57 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:57.542 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:57 smithi143 ceph-mon[34337]: pgmap v233: 1 pgs: 1 activating+degraded; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T05:23:57.542 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:57 smithi143 ceph-mon[34337]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-08T05:23:57.542 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:23:57.543 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:23:57.543 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:57 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:57 smithi059 ceph-mon[30304]: pgmap v233: 1 pgs: 1 activating+degraded; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T05:23:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:57 smithi059 ceph-mon[30304]: Health check failed: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded (PG_DEGRADED) 2024-05-08T05:23:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-05-08T05:23:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-05-08T05:23:57.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:57 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:23:58.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:58 smithi143 ceph-mon[34337]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:23:58.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:58 smithi096 ceph-mon[34253]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:23:58.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:58 smithi059 ceph-mon[30304]: from='client.24343 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi143:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:23:59.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:23:59 smithi059 ceph-mon[30304]: pgmap v234: 1 pgs: 1 activating+degraded; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T05:23:59.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:23:59 smithi143 ceph-mon[34337]: pgmap v234: 1 pgs: 1 activating+degraded; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T05:23:59.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:23:59 smithi096 ceph-mon[34253]: pgmap v234: 1 pgs: 1 activating+degraded; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T05:24:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:00 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/757403918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c3b9ed-8849-4a99-a885-fed187c8aaf7"}]: dispatch 2024-05-08T05:24:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:00 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c3b9ed-8849-4a99-a885-fed187c8aaf7"}]: dispatch 2024-05-08T05:24:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:00 smithi143 ceph-mon[34337]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1c3b9ed-8849-4a99-a885-fed187c8aaf7"}]': finished 2024-05-08T05:24:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:00 smithi143 ceph-mon[34337]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T05:24:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:00 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:00.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:00 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/757403918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c3b9ed-8849-4a99-a885-fed187c8aaf7"}]: dispatch 2024-05-08T05:24:00.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:00 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c3b9ed-8849-4a99-a885-fed187c8aaf7"}]: dispatch 2024-05-08T05:24:00.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:00 smithi059 ceph-mon[30304]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1c3b9ed-8849-4a99-a885-fed187c8aaf7"}]': finished 2024-05-08T05:24:00.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:00 smithi059 ceph-mon[30304]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T05:24:00.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:00 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:00.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:00 smithi096 ceph-mon[34253]: from='client.? 172.21.15.143:0/757403918' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c3b9ed-8849-4a99-a885-fed187c8aaf7"}]: dispatch 2024-05-08T05:24:00.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:00 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "c1c3b9ed-8849-4a99-a885-fed187c8aaf7"}]: dispatch 2024-05-08T05:24:00.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:00 smithi096 ceph-mon[34253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c1c3b9ed-8849-4a99-a885-fed187c8aaf7"}]': finished 2024-05-08T05:24:00.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:00 smithi096 ceph-mon[34253]: osdmap e68: 12 total, 11 up, 12 in 2024-05-08T05:24:00.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:00 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:01.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:01 smithi059 ceph-mon[30304]: pgmap v236: 1 pgs: 1 activating+degraded; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T05:24:01.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:01 smithi059 ceph-mon[30304]: from='client.? 172.21.15.143:0/2429854452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:24:01.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:01 smithi143 ceph-mon[34337]: pgmap v236: 1 pgs: 1 activating+degraded; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T05:24:01.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:01 smithi143 ceph-mon[34337]: from='client.? 172.21.15.143:0/2429854452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:24:01.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:01 smithi096 ceph-mon[34253]: pgmap v236: 1 pgs: 1 activating+degraded; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 2/6 objects degraded (33.333%) 2024-05-08T05:24:01.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:01 smithi096 ceph-mon[34253]: from='client.? 172.21.15.143:0/2429854452' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-05-08T05:24:02.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:02 smithi143 ceph-mon[34337]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-08T05:24:02.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:02 smithi143 ceph-mon[34337]: Cluster is now healthy 2024-05-08T05:24:02.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:02 smithi096 ceph-mon[34253]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-08T05:24:02.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:02 smithi096 ceph-mon[34253]: Cluster is now healthy 2024-05-08T05:24:02.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:02 smithi059 ceph-mon[30304]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 2/6 objects degraded (33.333%), 1 pg degraded) 2024-05-08T05:24:02.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:02 smithi059 ceph-mon[30304]: Cluster is now healthy 2024-05-08T05:24:03.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:03 smithi143 ceph-mon[34337]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T05:24:03.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:03 smithi096 ceph-mon[34253]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T05:24:03.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:03 smithi059 ceph-mon[30304]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 75 KiB/s, 0 objects/s recovering 2024-05-08T05:24:05.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:05 smithi143 ceph-mon[34337]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-08T05:24:05.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:05 smithi096 ceph-mon[34253]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-08T05:24:05.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:05 smithi059 ceph-mon[30304]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 66 KiB/s, 0 objects/s recovering 2024-05-08T05:24:07.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:07 smithi143 ceph-mon[34337]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:24:07.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:07 smithi096 ceph-mon[34253]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:24:07.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:07 smithi059 ceph-mon[30304]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:24:08.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T05:24:08.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:08 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:08.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:08 smithi143 ceph-mon[34337]: Deploying daemon osd.11 on smithi143 2024-05-08T05:24:08.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T05:24:08.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:08 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:08.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:08 smithi096 ceph-mon[34253]: Deploying daemon osd.11 on smithi143 2024-05-08T05:24:08.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-05-08T05:24:08.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:08 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:08.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:08 smithi059 ceph-mon[30304]: Deploying daemon osd.11 on smithi143 2024-05-08T05:24:09.658 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:09 smithi143 ceph-mon[34337]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:24:09.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:09 smithi096 ceph-mon[34253]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:24:09.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:09 smithi059 ceph-mon[30304]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-05-08T05:24:11.560 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:11 smithi143 ceph-mon[34337]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-08T05:24:11.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:11 smithi096 ceph-mon[34253]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-08T05:24:11.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:11 smithi059 ceph-mon[30304]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 57 KiB/s, 0 objects/s recovering 2024-05-08T05:24:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:13 smithi143 ceph-mon[34337]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T05:24:13.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:13 smithi096 ceph-mon[34253]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T05:24:13.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:13 smithi059 ceph-mon[30304]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-05-08T05:24:14.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:14.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:14.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:24:14.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:14.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:14.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:24:14.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:14.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:14.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:24:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:15 smithi143 ceph-mon[34337]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:15.681 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:15 smithi096 ceph-mon[34253]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:15.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:15 smithi059 ceph-mon[30304]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:17.055 INFO:teuthology.orchestra.run.smithi143.stdout:Created osd(s) 11 on host 'smithi143' 2024-05-08T05:24:17.056 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.053+0000 7f1ddffff700 1 -- 172.21.15.143:0/1600498592 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f1de80638e0 con 0x7f1dd40b2880 2024-05-08T05:24:17.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.055+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1dd40b2880 msgr2=0x7f1dd40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:17.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1dd40b2880 0x7f1dd40b4d40 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1dd0005950 tx=0x7f1dd000a300 comp rx=0 tx=0).stop 2024-05-08T05:24:17.057 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8103fb0 msgr2=0x7f1de81995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:17.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8103fb0 0x7f1de81995a0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1dd800e9d0 tx=0x7f1dd800ece0 comp rx=0 tx=0).stop 2024-05-08T05:24:17.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 shutdown_connections 2024-05-08T05:24:17.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1dd40b2880 0x7f1dd40b4d40 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:17.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1de8109210 0x7f1de81a0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:17.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1de8103fb0 0x7f1de81995a0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:17.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 --2- 172.21.15.143:0/1600498592 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1de81068e0 0x7f1de8199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:17.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 >> 172.21.15.143:0/1600498592 conn(0x7f1de80fd9c0 msgr2=0x7f1de81048b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:17.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 shutdown_connections 2024-05-08T05:24:17.058 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:17.056+0000 7f1dec832700 1 -- 172.21.15.143:0/1600498592 wait complete. 2024-05-08T05:24:17.653 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:17 smithi143 ceph-mon[34337]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:17.653 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:17 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:17.653 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:17 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:17.653 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:17 smithi143 ceph-mon[34337]: from='osd.11 [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:24:17.653 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:17 smithi143 ceph-mon[34337]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:24:17.680 DEBUG:teuthology.orchestra.run.smithi143:osd.11> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.11.service 2024-05-08T05:24:17.683 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-05-08T05:24:17.683 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T05:24:17.712 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:17 smithi059 ceph-mon[30304]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:17.712 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:17.712 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:17 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:17.712 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:17 smithi059 ceph-mon[30304]: from='osd.11 [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:24:17.712 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:17 smithi059 ceph-mon[30304]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:24:17.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:17 smithi096 ceph-mon[34253]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:17.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:17 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:17.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:17 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:17.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:17 smithi096 ceph-mon[34253]: from='osd.11 [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:24:17.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:17 smithi096 ceph-mon[34253]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-05-08T05:24:17.948 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:24:19.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:19.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:19.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:18 smithi143 ceph-mon[34337]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:19.190 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:18 smithi143 ceph-mon[34337]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T05:24:19.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:18 smithi143 ceph-mon[34337]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T05:24:19.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:19.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:18 smithi143 ceph-mon[34337]: from='osd.11 [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:24:19.191 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:18 smithi143 ceph-mon[34337]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:24:19.191 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:24:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:24:19.037+0000 7fc6a27a6700 -1 osd.11 0 waiting for initial osdmap 2024-05-08T05:24:19.191 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:24:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:24:19.048+0000 7fc69e599700 -1 osd.11 70 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-05-08T05:24:19.229 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:19.229 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:19.229 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:18 smithi059 ceph-mon[30304]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:19.229 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:18 smithi059 ceph-mon[30304]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T05:24:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:18 smithi059 ceph-mon[30304]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T05:24:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:18 smithi059 ceph-mon[30304]: from='osd.11 [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:24:19.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:18 smithi059 ceph-mon[30304]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:24:19.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:19.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:19.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:18 smithi096 ceph-mon[34253]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:19.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:18 smithi096 ceph-mon[34253]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-05-08T05:24:19.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:18 smithi096 ceph-mon[34253]: osdmap e69: 12 total, 11 up, 12 in 2024-05-08T05:24:19.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:19.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:18 smithi096 ceph-mon[34253]: from='osd.11 [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:24:19.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:18 smithi096 ceph-mon[34253]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]: dispatch 2024-05-08T05:24:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.750+0000 7fc778f92700 1 -- 172.21.15.59:0/2938926607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc774104720 msgr2=0x7fc774104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.750+0000 7fc778f92700 1 --2- 172.21.15.59:0/2938926607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc774104720 0x7fc774104b80 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fc75c0099c0 tx=0x7fc75c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.750+0000 7fc778f92700 1 -- 172.21.15.59:0/2938926607 shutdown_connections 2024-05-08T05:24:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.750+0000 7fc778f92700 1 --2- 172.21.15.59:0/2938926607 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7741050c0 0x7fc77410b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.750+0000 7fc778f92700 1 --2- 172.21.15.59:0/2938926607 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc774103520 0x7fc774103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.750+0000 7fc778f92700 1 --2- 172.21.15.59:0/2938926607 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc774104720 0x7fc774104b80 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.750+0000 7fc778f92700 1 -- 172.21.15.59:0/2938926607 >> 172.21.15.59:0/2938926607 conn(0x7fc7740feac0 msgr2=0x7fc774100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:19.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.750+0000 7fc778f92700 1 -- 172.21.15.59:0/2938926607 shutdown_connections 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.750+0000 7fc778f92700 1 -- 172.21.15.59:0/2938926607 wait complete. 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc778f92700 1 Processor -- start 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc778f92700 1 -- start start 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc778f92700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc774103520 0x7fc774077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc778f92700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc774104720 0x7fc774075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc778f92700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7741050c0 0x7fc774075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc778f92700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc77410efa0 con 0x7fc7741050c0 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc778f92700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc77410ee20 con 0x7fc774104720 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc778f92700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc77410f120 con 0x7fc774103520 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc772d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7741050c0 0x7fc774075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc772d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7741050c0 0x7fc774075dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56016/0 (socket says 172.21.15.59:56016) 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.751+0000 7fc772d9d700 1 -- 172.21.15.59:0/3699098424 learned_addr learned my addr 172.21.15.59:0/3699098424 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc76bfff700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc774104720 0x7fc774075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:19.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc77259c700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc774103520 0x7fc774077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc772d9d700 1 -- 172.21.15.59:0/3699098424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc774103520 msgr2=0x7fc774077240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc772d9d700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc774103520 0x7fc774077240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc772d9d700 1 -- 172.21.15.59:0/3699098424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc774104720 msgr2=0x7fc774075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc772d9d700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc774104720 0x7fc774075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc772d9d700 1 -- 172.21.15.59:0/3699098424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc774076680 con 0x7fc7741050c0 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc772d9d700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7741050c0 0x7fc774075dd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00ada0 tx=0x7fc76c00c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc76b7fe700 1 -- 172.21.15.59:0/3699098424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c017020 con 0x7fc7741050c0 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc76b7fe700 1 -- 172.21.15.59:0/3699098424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc76c00cd80 con 0x7fc7741050c0 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc7741a3740 con 0x7fc7741050c0 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc76b7fe700 1 -- 172.21.15.59:0/3699098424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc76c005260 con 0x7fc7741050c0 2024-05-08T05:24:19.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.752+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7741a3c50 con 0x7fc7741050c0 2024-05-08T05:24:19.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.753+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc77406a190 con 0x7fc7741050c0 2024-05-08T05:24:19.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.755+0000 7fc76b7fe700 1 -- 172.21.15.59:0/3699098424 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc76c00c8d0 con 0x7fc7741050c0 2024-05-08T05:24:19.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.755+0000 7fc76b7fe700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc7600b2880 0x7fc7600b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:19.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.755+0000 7fc77259c700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc7600b2880 0x7fc7600b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:19.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.755+0000 7fc76b7fe700 1 -- 172.21.15.59:0/3699098424 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 6847+0+0 (secure 0 0 0) 0x7fc76c0e1350 con 0x7fc7741050c0 2024-05-08T05:24:19.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.756+0000 7fc77259c700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc7600b2880 0x7fc7600b4d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc774104580 tx=0x7fc76400a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:19.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.760+0000 7fc76b7fe700 1 -- 172.21.15.59:0/3699098424 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc76c021020 con 0x7fc7741050c0 2024-05-08T05:24:19.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.978+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fc77402e510 con 0x7fc7741050c0 2024-05-08T05:24:19.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.978+0000 7fc76b7fe700 1 -- 172.21.15.59:0/3699098424 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7fc76c0a96c0 con 0x7fc7741050c0 2024-05-08T05:24:19.979 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:24:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc7600b2880 msgr2=0x7fc7600b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc7600b2880 0x7fc7600b4d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc774104580 tx=0x7fc76400a3b0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7741050c0 msgr2=0x7fc774075dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7741050c0 0x7fc774075dd0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fc76c00ada0 tx=0x7fc76c00c330 comp rx=0 tx=0).stop 2024-05-08T05:24:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 shutdown_connections 2024-05-08T05:24:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7741050c0 0x7fc774075dd0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc774103520 0x7fc774077240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc7600b2880 0x7fc7600b4d40 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 --2- 172.21.15.59:0/3699098424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc774104720 0x7fc774075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 >> 172.21.15.59:0/3699098424 conn(0x7fc7740feac0 msgr2=0x7fc774107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 shutdown_connections 2024-05-08T05:24:19.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:19.981+0000 7fc778f92700 1 -- 172.21.15.59:0/3699098424 wait complete. 2024-05-08T05:24:20.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:20 smithi143 ceph-mon[34337]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:24:20.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:20 smithi143 ceph-mon[34337]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T05:24:20.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:20.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:20 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:20.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:20 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3699098424' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:24:20.279 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:20 smithi059 ceph-mon[30304]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:24:20.279 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:20 smithi059 ceph-mon[30304]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T05:24:20.279 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:20.279 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:20 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:20.279 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:20 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3699098424' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:24:20.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:20 smithi096 ceph-mon[34253]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi143", "root=default"]}]': finished 2024-05-08T05:24:20.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:20 smithi096 ceph-mon[34253]: osdmap e70: 12 total, 11 up, 12 in 2024-05-08T05:24:20.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:20.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:20 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:20.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:20 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3699098424' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:24:20.732 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":70,"num_osds":12,"num_up_osds":11,"osd_up_since":1715145833,"num_in_osds":12,"osd_in_since":1715145839,"num_remapped_pgs":0} 2024-05-08T05:24:21.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:21 smithi143 ceph-mon[34337]: purged_snaps scrub starts 2024-05-08T05:24:21.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:21 smithi143 ceph-mon[34337]: purged_snaps scrub ok 2024-05-08T05:24:21.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:21 smithi143 ceph-mon[34337]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:21.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:21 smithi143 ceph-mon[34337]: osd.11 [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] boot 2024-05-08T05:24:21.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:21 smithi143 ceph-mon[34337]: osdmap e71: 12 total, 12 up, 12 in 2024-05-08T05:24:21.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:21 smithi059 ceph-mon[30304]: purged_snaps scrub starts 2024-05-08T05:24:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:21 smithi059 ceph-mon[30304]: purged_snaps scrub ok 2024-05-08T05:24:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:21 smithi059 ceph-mon[30304]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:21.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:21 smithi059 ceph-mon[30304]: osd.11 [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] boot 2024-05-08T05:24:21.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:21 smithi059 ceph-mon[30304]: osdmap e71: 12 total, 12 up, 12 in 2024-05-08T05:24:21.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:21 smithi096 ceph-mon[34253]: purged_snaps scrub starts 2024-05-08T05:24:21.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:21 smithi096 ceph-mon[34253]: purged_snaps scrub ok 2024-05-08T05:24:21.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:21 smithi096 ceph-mon[34253]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 318 MiB used, 983 GiB / 983 GiB avail 2024-05-08T05:24:21.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:21 smithi096 ceph-mon[34253]: osd.11 [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] boot 2024-05-08T05:24:21.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:21 smithi096 ceph-mon[34253]: osdmap e71: 12 total, 12 up, 12 in 2024-05-08T05:24:21.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:24:21.734 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd stat -f json 2024-05-08T05:24:21.986 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:24:22.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T05:24:22.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: Detected new or changed devices on smithi143 2024-05-08T05:24:22.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi143 to 4272M 2024-05-08T05:24:22.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:22.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:24:22.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:24:22.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T05:24:22.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: Detected new or changed devices on smithi143 2024-05-08T05:24:22.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi143 to 4272M 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:24:22.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: osdmap e72: 12 total, 12 up, 12 in 2024-05-08T05:24:22.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: Detected new or changed devices on smithi143 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi143 to 4272M 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:22.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:24:23.304 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:23 smithi059 ceph-mon[30304]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:23.304 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:23 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:23.304 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:23 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:24:23.304 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:23 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:23 smithi143 ceph-mon[34337]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:23 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:23 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:24:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:23 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:23 smithi096 ceph-mon[34253]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:23 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:23 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:24:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:23 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:23.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.647+0000 7faeff7da700 1 -- 172.21.15.59:0/2596519467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 msgr2=0x7faef810a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:23.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.647+0000 7faeff7da700 1 --2- 172.21.15.59:0/2596519467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 0x7faef810a840 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7faef000b0d0 tx=0x7faef000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.647+0000 7faeff7da700 1 -- 172.21.15.59:0/2596519467 shutdown_connections 2024-05-08T05:24:23.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.647+0000 7faeff7da700 1 --2- 172.21.15.59:0/2596519467 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 0x7faef810a840 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.647+0000 7faeff7da700 1 --2- 172.21.15.59:0/2596519467 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef81023f0 0x7faef8102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.647+0000 7faeff7da700 1 --2- 172.21.15.59:0/2596519467 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faef81035f0 0x7faef8103a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.647+0000 7faeff7da700 1 -- 172.21.15.59:0/2596519467 >> 172.21.15.59:0/2596519467 conn(0x7faef80fd990 msgr2=0x7faef80ffdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 -- 172.21.15.59:0/2596519467 shutdown_connections 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 -- 172.21.15.59:0/2596519467 wait complete. 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 Processor -- start 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 -- start start 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faef81023f0 0x7faef8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef81035f0 0x7faef8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 0x7faef81a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faef810de60 con 0x7faef8103f90 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faef810dce0 con 0x7faef81023f0 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.648+0000 7faeff7da700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7faef810dfe0 con 0x7faef81035f0 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefdd77700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 0x7faef81a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefdd77700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 0x7faef81a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56040/0 (socket says 172.21.15.59:56040) 2024-05-08T05:24:23.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefdd77700 1 -- 172.21.15.59:0/2012192825 learned_addr learned my addr 172.21.15.59:0/2012192825 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefcd75700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef81035f0 0x7faef8199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefd576700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faef81023f0 0x7faef8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefdd77700 1 -- 172.21.15.59:0/2012192825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef81035f0 msgr2=0x7faef8199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefdd77700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef81035f0 0x7faef8199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefdd77700 1 -- 172.21.15.59:0/2012192825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faef81023f0 msgr2=0x7faef8199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefdd77700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faef81023f0 0x7faef8199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefdd77700 1 -- 172.21.15.59:0/2012192825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faef81a11e0 con 0x7faef8103f90 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faefdd77700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 0x7faef81a0b40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faef0009fd0 tx=0x7faef0004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faeea7fc700 1 -- 172.21.15.59:0/2012192825 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef001f030 con 0x7faef8103f90 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faef81a1470 con 0x7faef8103f90 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faeea7fc700 1 -- 172.21.15.59:0/2012192825 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faef0003a00 con 0x7faef8103f90 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faeea7fc700 1 -- 172.21.15.59:0/2012192825 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faef00046a0 con 0x7faef8103f90 2024-05-08T05:24:23.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.649+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faef81a1920 con 0x7faef8103f90 2024-05-08T05:24:23.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.650+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faef806a190 con 0x7faef8103f90 2024-05-08T05:24:23.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.652+0000 7faeea7fc700 1 -- 172.21.15.59:0/2012192825 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7faef0003bb0 con 0x7faef8103f90 2024-05-08T05:24:23.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.652+0000 7faeea7fc700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7faee40b2880 0x7faee40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:23.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.652+0000 7faefd576700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7faee40b2880 0x7faee40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:23.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.653+0000 7faeea7fc700 1 -- 172.21.15.59:0/2012192825 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7faef000f020 con 0x7faef8103f90 2024-05-08T05:24:23.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.653+0000 7faefd576700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7faee40b2880 0x7faee40b4d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faef4005fd0 tx=0x7faef4005dd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:23.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.656+0000 7faeea7fc700 1 -- 172.21.15.59:0/2012192825 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faef0029020 con 0x7faef8103f90 2024-05-08T05:24:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.869+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7faef81a1cf0 con 0x7faef8103f90 2024-05-08T05:24:23.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.870+0000 7faeea7fc700 1 -- 172.21.15.59:0/2012192825 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v72) ==== 74+0+133 (secure 0 0 0) 0x7faef00ab670 con 0x7faef8103f90 2024-05-08T05:24:23.870 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:24:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7faee40b2880 msgr2=0x7faee40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:23.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7faee40b2880 0x7faee40b4d40 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7faef4005fd0 tx=0x7faef4005dd0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 msgr2=0x7faef81a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 0x7faef81a0b40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7faef0009fd0 tx=0x7faef0004d50 comp rx=0 tx=0).stop 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 shutdown_connections 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faef8103f90 0x7faef81a0b40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7faee40b2880 0x7faee40b4d40 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faef81023f0 0x7faef8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 --2- 172.21.15.59:0/2012192825 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faef81035f0 0x7faef8199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 >> 172.21.15.59:0/2012192825 conn(0x7faef80fd990 msgr2=0x7faef80ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 shutdown_connections 2024-05-08T05:24:23.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:23.872+0000 7faeff7da700 1 -- 172.21.15.59:0/2012192825 wait complete. 2024-05-08T05:24:24.228 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:24 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2012192825' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:24:24.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:24 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2012192825' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:24:24.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:24 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2012192825' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-05-08T05:24:24.544 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"num_osds":12,"num_up_osds":12,"osd_up_since":1715145860,"num_in_osds":12,"osd_in_since":1715145839,"num_remapped_pgs":0} 2024-05-08T05:24:24.544 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:24:24.799 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:24:25.232 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:25 smithi059 ceph-mon[30304]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:25.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:25 smithi143 ceph-mon[34337]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:25.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:25 smithi096 ceph-mon[34253]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:26.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.473+0000 7f843c54f700 1 -- 172.21.15.59:0/161930132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8434102690 msgr2=0x7f8434102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.473+0000 7f843c54f700 1 --2- 172.21.15.59:0/161930132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8434102690 0x7f8434102a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f84300099c0 tx=0x7f8430009cd0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.473+0000 7f843c54f700 1 -- 172.21.15.59:0/161930132 shutdown_connections 2024-05-08T05:24:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.473+0000 7f843c54f700 1 --2- 172.21.15.59:0/161930132 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84341076e0 0x7f843410f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.473+0000 7f843c54f700 1 --2- 172.21.15.59:0/161930132 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8434102690 0x7f8434102a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.473+0000 7f843c54f700 1 --2- 172.21.15.59:0/161930132 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8434103040 0x7f8434107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.473+0000 7f843c54f700 1 -- 172.21.15.59:0/161930132 >> 172.21.15.59:0/161930132 conn(0x7f84340783f0 msgr2=0x7f8434078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.473+0000 7f843c54f700 1 -- 172.21.15.59:0/161930132 shutdown_connections 2024-05-08T05:24:26.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.474+0000 7f843c54f700 1 -- 172.21.15.59:0/161930132 wait complete. 2024-05-08T05:24:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843c54f700 1 Processor -- start 2024-05-08T05:24:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843c54f700 1 -- start start 2024-05-08T05:24:26.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843c54f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8434102690 0x7f84341040a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843c54f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8434103040 0x7f84341045e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843c54f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84341076e0 0x7f8434104b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843c54f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84341125f0 con 0x7f84341076e0 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843c54f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8434112470 con 0x7f8434103040 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843c54f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8434112770 con 0x7f8434102690 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843aaec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84341076e0 0x7f8434104b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843aaec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84341076e0 0x7f8434104b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56062/0 (socket says 172.21.15.59:56062) 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843aaec700 1 -- 172.21.15.59:0/3784932128 learned_addr learned my addr 172.21.15.59:0/3784932128 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f8439aea700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8434103040 0x7f84341045e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:26.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843a2eb700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8434102690 0x7f84341040a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843aaec700 1 -- 172.21.15.59:0/3784932128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8434102690 msgr2=0x7f84341040a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843aaec700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8434102690 0x7f84341040a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843aaec700 1 -- 172.21.15.59:0/3784932128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8434103040 msgr2=0x7f84341045e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843aaec700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8434103040 0x7f84341045e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843aaec700 1 -- 172.21.15.59:0/3784932128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84341a8150 con 0x7f84341076e0 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.475+0000 7f843a2eb700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8434102690 0x7f84341040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.476+0000 7f843aaec700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84341076e0 0x7f8434104b20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f842c007d70 tx=0x7f842c00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.476+0000 7f842b7fe700 1 -- 172.21.15.59:0/3784932128 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c016020 con 0x7f84341076e0 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.476+0000 7f842b7fe700 1 -- 172.21.15.59:0/3784932128 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f842c00ad40 con 0x7f84341076e0 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.476+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84341a8440 con 0x7f84341076e0 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.476+0000 7f842b7fe700 1 -- 172.21.15.59:0/3784932128 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f842c0115e0 con 0x7f84341076e0 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.476+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84341a8930 con 0x7f84341076e0 2024-05-08T05:24:26.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.477+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f843406a190 con 0x7f84341076e0 2024-05-08T05:24:26.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.478+0000 7f842b7fe700 1 -- 172.21.15.59:0/3784932128 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f842c019020 con 0x7f84341076e0 2024-05-08T05:24:26.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.479+0000 7f842b7fe700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84200b2880 0x7f84200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:26.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.480+0000 7f843a2eb700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84200b2880 0x7f84200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:26.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.480+0000 7f842b7fe700 1 -- 172.21.15.59:0/3784932128 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f842c0e2300 con 0x7f84341076e0 2024-05-08T05:24:26.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.480+0000 7f843a2eb700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84200b2880 0x7f84200b4d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8430003800 tx=0x7f8430009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:26.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.483+0000 7f842b7fe700 1 -- 172.21.15.59:0/3784932128 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f842c020020 con 0x7f84341076e0 2024-05-08T05:24:26.704 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.703+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f843402e510 con 0x7f84341076e0 2024-05-08T05:24:26.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.704+0000 7f842b7fe700 1 -- 172.21.15.59:0/3784932128 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v72) ==== 74+0+18403 (secure 0 0 0) 0x7f842c0aa420 con 0x7f84341076e0 2024-05-08T05:24:26.705 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:24:26.705 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":72,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:24:21.040310+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":37,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":47,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":66,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84200b2880 msgr2=0x7f84200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84200b2880 0x7f84200b4d40 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f8430003800 tx=0x7f8430009f10 comp rx=0 tx=0).stop 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84341076e0 msgr2=0x7f8434104b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84341076e0 0x7f8434104b20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f842c007d70 tx=0x7f842c00a340 comp rx=0 tx=0).stop 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 shutdown_connections 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84341076e0 0x7f8434104b20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8434102690 0x7f84341040a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84200b2880 0x7f84200b4d40 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 --2- 172.21.15.59:0/3784932128 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8434103040 0x7f84341045e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 >> 172.21.15.59:0/3784932128 conn(0x7f84340783f0 msgr2=0x7f843406bf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 shutdown_connections 2024-05-08T05:24:26.708 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:26.707+0000 7f843c54f700 1 -- 172.21.15.59:0/3784932128 wait complete. 2024-05-08T05:24:27.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:27 smithi096 ceph-mon[34253]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:27.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:27 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3784932128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:24:27.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:27 smithi059 ceph-mon[30304]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:27.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:27 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3784932128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:24:27.418 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:20:31.976415+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-05-08T05:24:27.418 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-05-08T05:24:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:27 smithi143 ceph-mon[34337]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:27 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3784932128' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:24:27.671 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:24:29.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.317+0000 7f4637b5c700 1 -- 172.21.15.59:0/3414007108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006ba40 msgr2=0x7f463006be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:29.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.317+0000 7f4637b5c700 1 --2- 172.21.15.59:0/3414007108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006ba40 0x7f463006be20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f462c009990 tx=0x7f462c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.317+0000 7f4637b5c700 1 -- 172.21.15.59:0/3414007108 shutdown_connections 2024-05-08T05:24:29.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.317+0000 7f4637b5c700 1 --2- 172.21.15.59:0/3414007108 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463010ff40 0x7f4630112330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.317+0000 7f4637b5c700 1 --2- 172.21.15.59:0/3414007108 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006ba40 0x7f463006be20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.317+0000 7f4637b5c700 1 --2- 172.21.15.59:0/3414007108 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463006c360 0x7f463010fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.317+0000 7f4637b5c700 1 -- 172.21.15.59:0/3414007108 >> 172.21.15.59:0/3414007108 conn(0x7f4630079310 msgr2=0x7f4630079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:29.318 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:29 smithi059 ceph-mon[30304]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.317+0000 7f4637b5c700 1 -- 172.21.15.59:0/3414007108 shutdown_connections 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.317+0000 7f4637b5c700 1 -- 172.21.15.59:0/3414007108 wait complete. 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f4637b5c700 1 Processor -- start 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f4637b5c700 1 -- start start 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f4637b5c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463006ba40 0x7f463019bd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f4637b5c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006c360 0x7f463019c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f4637b5c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463010ff40 0x7f46301a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f4637b5c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4630115870 con 0x7f463006c360 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f4637b5c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46301156f0 con 0x7f463006ba40 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f4637b5c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f46301159f0 con 0x7f463010ff40 2024-05-08T05:24:29.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f46350f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006c360 0x7f463019c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f46350f7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006c360 0x7f463019c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41748/0 (socket says 172.21.15.59:41748) 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f46350f7700 1 -- 172.21.15.59:0/1657065013 learned_addr learned my addr 172.21.15.59:0/1657065013 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.318+0000 7f46358f8700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463006ba40 0x7f463019bd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f46360f9700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463010ff40 0x7f46301a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f46350f7700 1 -- 172.21.15.59:0/1657065013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463010ff40 msgr2=0x7f46301a0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f46350f7700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463010ff40 0x7f46301a0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f46350f7700 1 -- 172.21.15.59:0/1657065013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463006ba40 msgr2=0x7f463019bd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f46350f7700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463006ba40 0x7f463019bd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f46350f7700 1 -- 172.21.15.59:0/1657065013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46301a0d70 con 0x7f463006c360 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f46350f7700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006c360 0x7f463019c2a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f462000c970 tx=0x7f462000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f4626ffd700 1 -- 172.21.15.59:0/1657065013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620007aa0 con 0x7f463006c360 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f4626ffd700 1 -- 172.21.15.59:0/1657065013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4620004ce0 con 0x7f463006c360 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46301a1060 con 0x7f463006c360 2024-05-08T05:24:29.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f4626ffd700 1 -- 172.21.15.59:0/1657065013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4620005510 con 0x7f463006c360 2024-05-08T05:24:29.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.319+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46301a88e0 con 0x7f463006c360 2024-05-08T05:24:29.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.320+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f463006a190 con 0x7f463006c360 2024-05-08T05:24:29.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.322+0000 7f4626ffd700 1 -- 172.21.15.59:0/1657065013 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4620007c40 con 0x7f463006c360 2024-05-08T05:24:29.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.323+0000 7f4626ffd700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f461c0b2900 0x7f461c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:29.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.323+0000 7f46358f8700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f461c0b2900 0x7f461c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:29.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.323+0000 7f4626ffd700 1 -- 172.21.15.59:0/1657065013 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f46200e1e00 con 0x7f463006c360 2024-05-08T05:24:29.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.323+0000 7f46358f8700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f461c0b2900 0x7f461c0b4dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f462c00b5c0 tx=0x7f462c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:29.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.326+0000 7f4626ffd700 1 -- 172.21.15.59:0/1657065013 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f46200aa040 con 0x7f463006c360 2024-05-08T05:24:29.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:29 smithi143 ceph-mon[34337]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:29 smithi096 ceph-mon[34253]: pgmap v254: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:29.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.539+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f463002e510 con 0x7f463006c360 2024-05-08T05:24:29.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.539+0000 7f4626ffd700 1 -- 172.21.15.59:0/1657065013 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v72) ==== 93+0+10 (secure 0 0 0) 0x7f46200a9940 con 0x7f463006c360 2024-05-08T05:24:29.540 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-05-08T05:24:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.541+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f461c0b2900 msgr2=0x7f461c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.541+0000 7f4637b5c700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f461c0b2900 0x7f461c0b4dc0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f462c00b5c0 tx=0x7f462c005960 comp rx=0 tx=0).stop 2024-05-08T05:24:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.541+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006c360 msgr2=0x7f463019c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.541+0000 7f4637b5c700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006c360 0x7f463019c2a0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f462000c970 tx=0x7f462000cc80 comp rx=0 tx=0).stop 2024-05-08T05:24:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.542+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 shutdown_connections 2024-05-08T05:24:29.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.542+0000 7f4637b5c700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f463010ff40 0x7f46301a0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.542+0000 7f4637b5c700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f461c0b2900 0x7f461c0b4dc0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.542+0000 7f4637b5c700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f463006ba40 0x7f463019bd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.542+0000 7f4637b5c700 1 --2- 172.21.15.59:0/1657065013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f463006c360 0x7f463019c2a0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.542+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 >> 172.21.15.59:0/1657065013 conn(0x7f4630079310 msgr2=0x7f46301113f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.542+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 shutdown_connections 2024-05-08T05:24:29.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:29.542+0000 7f4637b5c700 1 -- 172.21.15.59:0/1657065013 wait complete. 2024-05-08T05:24:30.230 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:30 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1657065013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:24:30.262 INFO:tasks.cephadm:Adding mds.a on smithi059 2024-05-08T05:24:30.262 INFO:tasks.cephadm:Adding mds.d on smithi059 2024-05-08T05:24:30.262 INFO:tasks.cephadm:Adding mds.g on smithi059 2024-05-08T05:24:30.263 INFO:tasks.cephadm:Adding mds.j on smithi059 2024-05-08T05:24:30.263 INFO:tasks.cephadm:Adding mds.b on smithi096 2024-05-08T05:24:30.263 INFO:tasks.cephadm:Adding mds.e on smithi096 2024-05-08T05:24:30.263 INFO:tasks.cephadm:Adding mds.h on smithi096 2024-05-08T05:24:30.263 INFO:tasks.cephadm:Adding mds.k on smithi096 2024-05-08T05:24:30.263 INFO:tasks.cephadm:Adding mds.c on smithi143 2024-05-08T05:24:30.263 INFO:tasks.cephadm:Adding mds.f on smithi143 2024-05-08T05:24:30.263 INFO:tasks.cephadm:Adding mds.i on smithi143 2024-05-08T05:24:30.263 INFO:tasks.cephadm:Adding mds.l on smithi143 2024-05-08T05:24:30.263 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph orch apply mds all '12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l' 2024-05-08T05:24:30.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:30 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1657065013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:24:30.516 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:24:30.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:30 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1657065013' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:24:31.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:31 smithi143 ceph-mon[34337]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:31.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:31 smithi059 ceph-mon[30304]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:31.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:31 smithi096 ceph-mon[34253]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.163+0000 7ff5870c5700 1 -- 172.21.15.143:0/3100007098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 msgr2=0x7ff580108d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.163+0000 7ff5870c5700 1 --2- 172.21.15.143:0/3100007098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 0x7ff580108d60 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff56c009990 tx=0x7ff56c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 -- 172.21.15.143:0/3100007098 shutdown_connections 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 --2- 172.21.15.143:0/3100007098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff5801092a0 0x7ff58010b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 --2- 172.21.15.143:0/3100007098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff580104040 0x7ff580106430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 --2- 172.21.15.143:0/3100007098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 0x7ff580108d60 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 -- 172.21.15.143:0/3100007098 >> 172.21.15.143:0/3100007098 conn(0x7ff5800fd9d0 msgr2=0x7ff5800ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 -- 172.21.15.143:0/3100007098 shutdown_connections 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 -- 172.21.15.143:0/3100007098 wait complete. 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 Processor -- start 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 -- start start 2024-05-08T05:24:32.167 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff580104040 0x7ff580199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.164+0000 7ff5870c5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 0x7ff580199c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff5870c5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5801092a0 0x7ff5801a0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff5870c5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010f010 con 0x7ff5801092a0 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff5870c5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010ee90 con 0x7ff580104040 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff5870c5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff58010f190 con 0x7ff580106970 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 0x7ff580199c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 0x7ff580199c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.143:49212/0 (socket says 172.21.15.143:49212) 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57ffff700 1 -- 172.21.15.143:0/1625860111 learned_addr learned my addr 172.21.15.143:0/1625860111 (peer_addr_for_me v2:172.21.15.143:0/0) 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff585662700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5801092a0 0x7ff5801a0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff584e61700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff580104040 0x7ff580199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57ffff700 1 -- 172.21.15.143:0/1625860111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff580104040 msgr2=0x7ff580199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57ffff700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff580104040 0x7ff580199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57ffff700 1 -- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5801092a0 msgr2=0x7ff5801a0d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:32.168 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57ffff700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5801092a0 0x7ff5801a0d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57ffff700 1 -- 172.21.15.143:0/1625860111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5801a13b0 con 0x7ff580106970 2024-05-08T05:24:32.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff584e61700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff580104040 0x7ff580199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:24:32.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57ffff700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 0x7ff580199c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0059d0 tx=0x7ff56c009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:32.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57dffb700 1 -- 172.21.15.143:0/1625860111 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c027070 con 0x7ff580106970 2024-05-08T05:24:32.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5801a1640 con 0x7ff580106970 2024-05-08T05:24:32.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57dffb700 1 -- 172.21.15.143:0/1625860111 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff56c004080 con 0x7ff580106970 2024-05-08T05:24:32.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff57dffb700 1 -- 172.21.15.143:0/1625860111 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff56c022cf0 con 0x7ff580106970 2024-05-08T05:24:32.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.165+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5801a1b70 con 0x7ff580106970 2024-05-08T05:24:32.169 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.166+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff58006a190 con 0x7ff580106970 2024-05-08T05:24:32.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.168+0000 7ff57dffb700 1 -- 172.21.15.143:0/1625860111 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff56c02a070 con 0x7ff580106970 2024-05-08T05:24:32.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.168+0000 7ff57dffb700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff5700b2880 0x7ff5700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:32.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.168+0000 7ff57dffb700 1 -- 172.21.15.143:0/1625860111 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7ff56c0e3400 con 0x7ff580106970 2024-05-08T05:24:32.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.168+0000 7ff584e61700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff5700b2880 0x7ff5700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:32.170 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.169+0000 7ff584e61700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff5700b2880 0x7ff5700b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff574007900 tx=0x7ff574008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:32.172 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.172+0000 7ff57dffb700 1 -- 172.21.15.143:0/1625860111 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff56c031070 con 0x7ff580106970 2024-05-08T05:24:32.391 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.390+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}) -- 0x7ff580063ae0 con 0x7ff5700b2880 2024-05-08T05:24:32.399 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.398+0000 7ff57dffb700 1 -- 172.21.15.143:0/1625860111 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7ff580063ae0 con 0x7ff5700b2880 2024-05-08T05:24:32.399 INFO:teuthology.orchestra.run.smithi143.stdout:Scheduled mds.all update... 2024-05-08T05:24:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff5700b2880 msgr2=0x7ff5700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff5700b2880 0x7ff5700b4d40 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7ff574007900 tx=0x7ff574008040 comp rx=0 tx=0).stop 2024-05-08T05:24:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 msgr2=0x7ff580199c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 0x7ff580199c90 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff56c0059d0 tx=0x7ff56c009f90 comp rx=0 tx=0).stop 2024-05-08T05:24:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 shutdown_connections 2024-05-08T05:24:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff5801092a0 0x7ff5801a0d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff5700b2880 0x7ff5700b4d40 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.402 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff580104040 0x7ff580199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 --2- 172.21.15.143:0/1625860111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff580106970 0x7ff580199c90 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 >> 172.21.15.143:0/1625860111 conn(0x7ff5800fd9d0 msgr2=0x7ff580105ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 shutdown_connections 2024-05-08T05:24:32.403 INFO:teuthology.orchestra.run.smithi143.stderr:2024-05-08T05:24:32.401+0000 7ff5870c5700 1 -- 172.21.15.143:0/1625860111 wait complete. 2024-05-08T05:24:33.130 DEBUG:teuthology.orchestra.run.smithi059:mds.a> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.a.service 2024-05-08T05:24:33.133 DEBUG:teuthology.orchestra.run.smithi059:mds.d> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.d.service 2024-05-08T05:24:33.176 DEBUG:teuthology.orchestra.run.smithi059:mds.g> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.g.service 2024-05-08T05:24:33.220 DEBUG:teuthology.orchestra.run.smithi059:mds.j> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.j.service 2024-05-08T05:24:33.264 DEBUG:teuthology.orchestra.run.smithi096:mds.b> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.b.service 2024-05-08T05:24:33.267 DEBUG:teuthology.orchestra.run.smithi096:mds.e> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.e.service 2024-05-08T05:24:33.311 DEBUG:teuthology.orchestra.run.smithi096:mds.h> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.h.service 2024-05-08T05:24:33.355 DEBUG:teuthology.orchestra.run.smithi096:mds.k> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.k.service 2024-05-08T05:24:33.398 DEBUG:teuthology.orchestra.run.smithi143:mds.c> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.c.service 2024-05-08T05:24:33.400 DEBUG:teuthology.orchestra.run.smithi143:mds.f> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.f.service 2024-05-08T05:24:33.443 DEBUG:teuthology.orchestra.run.smithi143:mds.i> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.i.service 2024-05-08T05:24:33.486 DEBUG:teuthology.orchestra.run.smithi143:mds.l> sudo journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.l.service 2024-05-08T05:24:33.529 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-05-08T05:24:33.529 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:24:33.556 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:33.556 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:24:33.556 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-05-08T05:24:33.557 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:33.557 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:24:33.557 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:33.557 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:24:33.557 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:33.557 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:33.557 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:33.557 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:33.557 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:33 smithi059 ceph-mon[30304]: Deploying daemon mds.l on smithi143 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:33.670 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:33.671 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:33 smithi143 ceph-mon[34337]: Deploying daemon mds.l on smithi143 2024-05-08T05:24:33.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: from='client.24379 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: Saving service mds.all spec with placement smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:33 smithi096 ceph-mon[34253]: Deploying daemon mds.l on smithi143 2024-05-08T05:24:33.794 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:24:34.993 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:24:34 smithi143 systemd[1]: Starting Ceph mds.l for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:24:35.325 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:24:35 smithi143 podman[64355]: 2024-05-08 05:24:34.936103064 +0000 UTC m=+0.017266776 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:24:35.325 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:24:35 smithi143 podman[64355]: 2024-05-08 05:24:35.04660813 +0000 UTC m=+0.127771821 container create d840ca7eb67bfa48c07ed3c6f3291fdf89b41641ced2668be9b151b3750eb5bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-08T05:24:35.325 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:24:35 smithi143 podman[64355]: 2024-05-08 05:24:35.324821891 +0000 UTC m=+0.405985581 container init d840ca7eb67bfa48c07ed3c6f3291fdf89b41641ced2668be9b151b3750eb5bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-08T05:24:35.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:35 smithi059 ceph-mon[30304]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:35.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:35 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6832/1879950931,v1:172.21.15.143:6833/1879950931] up:boot 2024-05-08T05:24:35.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:35 smithi059 ceph-mon[30304]: fsmap 1 up:standby 2024-05-08T05:24:35.476 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:35 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:24:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.474+0000 7f20392cf700 1 -- 172.21.15.59:0/1163732380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341026a0 msgr2=0x7f2034102b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.474+0000 7f20392cf700 1 --2- 172.21.15.59:0/1163732380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341026a0 0x7f2034102b20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2028009990 tx=0x7f2028009ca0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.475+0000 7f20392cf700 1 -- 172.21.15.59:0/1163732380 shutdown_connections 2024-05-08T05:24:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.475+0000 7f20392cf700 1 --2- 172.21.15.59:0/1163732380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2034103060 0x7f203410f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.475+0000 7f20392cf700 1 --2- 172.21.15.59:0/1163732380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20341084b0 0x7f2034108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.475+0000 7f20392cf700 1 --2- 172.21.15.59:0/1163732380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341026a0 0x7f2034102b20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.475+0000 7f20392cf700 1 -- 172.21.15.59:0/1163732380 >> 172.21.15.59:0/1163732380 conn(0x7f20340791a0 msgr2=0x7f20340795b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.475+0000 7f20392cf700 1 -- 172.21.15.59:0/1163732380 shutdown_connections 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.475+0000 7f20392cf700 1 -- 172.21.15.59:0/1163732380 wait complete. 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.475+0000 7f20392cf700 1 Processor -- start 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.475+0000 7f20392cf700 1 -- start start 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20392cf700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20341026a0 0x7f203419aac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20392cf700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2034103060 0x7f203419b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20392cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341084b0 0x7f20341a5b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20392cf700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20341125e0 con 0x7f20341084b0 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20392cf700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2034112460 con 0x7f20341026a0 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20392cf700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2034112760 con 0x7f2034103060 2024-05-08T05:24:35.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341084b0 0x7f20341a5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20327fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2034103060 0x7f203419b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20337fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341084b0 0x7f20341a5b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41764/0 (socket says 172.21.15.59:41764) 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20327fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2034103060 0x7f203419b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:33974/0 (socket says 172.21.15.59:33974) 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20337fe700 1 -- 172.21.15.59:0/112392176 learned_addr learned my addr 172.21.15.59:0/112392176 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f2032ffd700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20341026a0 0x7f203419aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20327fc700 1 -- 172.21.15.59:0/112392176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20341026a0 msgr2=0x7f203419aac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20327fc700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20341026a0 0x7f203419aac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20327fc700 1 -- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341084b0 msgr2=0x7f20341a5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20327fc700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341084b0 0x7f20341a5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20327fc700 1 -- 172.21.15.59:0/112392176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20341a6080 con 0x7f2034103060 2024-05-08T05:24:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20337fe700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341084b0 0x7f20341a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:24:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.476+0000 7f20327fc700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2034103060 0x7f203419b000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20280059d0 tx=0x7f20280047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.477+0000 7f201bfff700 1 -- 172.21.15.59:0/112392176 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028027070 con 0x7f2034103060 2024-05-08T05:24:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.477+0000 7f201bfff700 1 -- 172.21.15.59:0/112392176 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2028022470 con 0x7f2034103060 2024-05-08T05:24:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.477+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20341a6250 con 0x7f2034103060 2024-05-08T05:24:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.477+0000 7f201bfff700 1 -- 172.21.15.59:0/112392176 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2028017650 con 0x7f2034103060 2024-05-08T05:24:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.477+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20341a66a0 con 0x7f2034103060 2024-05-08T05:24:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.478+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203406a190 con 0x7f2034103060 2024-05-08T05:24:35.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.479+0000 7f201bfff700 1 -- 172.21.15.59:0/112392176 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f202802a070 con 0x7f2034103060 2024-05-08T05:24:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.480+0000 7f201bfff700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f201c0b2900 0x7f201c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.480+0000 7f2032ffd700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f201c0b2900 0x7f201c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.480+0000 7f201bfff700 1 -- 172.21.15.59:0/112392176 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f20280e2d20 con 0x7f2034103060 2024-05-08T05:24:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.480+0000 7f2032ffd700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f201c0b2900 0x7f201c0b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2024009bd0 tx=0x7f2024009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:35.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.483+0000 7f201bfff700 1 -- 172.21.15.59:0/112392176 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2028013070 con 0x7f2034103060 2024-05-08T05:24:35.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:35 smithi143 ceph-mon[34337]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:35.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:35 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6832/1879950931,v1:172.21.15.143:6833/1879950931] up:boot 2024-05-08T05:24:35.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:35 smithi143 ceph-mon[34337]: fsmap 1 up:standby 2024-05-08T05:24:35.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:35 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:24:35.744 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:24:35 smithi143 podman[64355]: 2024-05-08 05:24:35.327743709 +0000 UTC m=+0.408907405 container start d840ca7eb67bfa48c07ed3c6f3291fdf89b41641ced2668be9b151b3750eb5bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:24:35.744 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:24:35 smithi143 ceph-mds[64373]: -- 172.21.15.143:0/3459149225 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557b4ad3da40 con 0x557b4ad10c00 2024-05-08T05:24:35.744 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:24:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l[64369]: starting mds.l at 2024-05-08T05:24:35.744 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:24:35 smithi143 bash[64355]: d840ca7eb67bfa48c07ed3c6f3291fdf89b41641ced2668be9b151b3750eb5bd 2024-05-08T05:24:35.744 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:24:35 smithi143 systemd[1]: Started Ceph mds.l for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:24:35.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.757+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f203402e510 con 0x7f2034103060 2024-05-08T05:24:35.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.757+0000 7f201bfff700 1 -- 172.21.15.59:0/112392176 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f20280aaed0 con 0x7f2034103060 2024-05-08T05:24:35.758 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:24:35.758 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-05-08T05:24:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f201c0b2900 msgr2=0x7f201c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f201c0b2900 0x7f201c0b4dc0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f2024009bd0 tx=0x7f2024009260 comp rx=0 tx=0).stop 2024-05-08T05:24:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2034103060 msgr2=0x7f203419b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2034103060 0x7f203419b000 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f20280059d0 tx=0x7f20280047b0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 shutdown_connections 2024-05-08T05:24:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20341084b0 0x7f20341a5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f201c0b2900 0x7f201c0b4dc0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20341026a0 0x7f203419aac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 --2- 172.21.15.59:0/112392176 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2034103060 0x7f203419b000 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.759+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 >> 172.21.15.59:0/112392176 conn(0x7f20340791a0 msgr2=0x7f2034100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.760+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 shutdown_connections 2024-05-08T05:24:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:35.760+0000 7f20392cf700 1 -- 172.21.15.59:0/112392176 wait complete. 2024-05-08T05:24:35.761 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 2 2024-05-08T05:24:35.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:35 smithi096 ceph-mon[34253]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:35.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:35 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6832/1879950931,v1:172.21.15.143:6833/1879950931] up:boot 2024-05-08T05:24:35.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:35 smithi096 ceph-mon[34253]: fsmap 1 up:standby 2024-05-08T05:24:35.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:35 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:24:36.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:36 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:36.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:36 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:36.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:36 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:36.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:36 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:36.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:36 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:36.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:36 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:36.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:36 smithi143 ceph-mon[34337]: Deploying daemon mds.k on smithi096 2024-05-08T05:24:36.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:36 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/112392176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:36.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:36 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:36.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:36 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:36.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:36 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:36.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:36 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:36.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:36 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:36.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:36 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:36.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:36 smithi096 ceph-mon[34253]: Deploying daemon mds.k on smithi096 2024-05-08T05:24:36.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:36 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/112392176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:36.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:36.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:36.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:36.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:36.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:36.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:36 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:36.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:36 smithi059 ceph-mon[30304]: Deploying daemon mds.k on smithi096 2024-05-08T05:24:36.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:36 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/112392176' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:37.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:37 smithi096 ceph-mon[34253]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:37.780 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:24:37 smithi096 systemd[1]: Starting Ceph mds.k for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:24:37.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:37 smithi059 ceph-mon[30304]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:37.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:37 smithi143 ceph-mon[34337]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:38.237 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:24:37 smithi096 podman[63503]: 2024-05-08 05:24:37.827395279 +0000 UTC m=+0.017548118 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:24:38.238 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:24:37 smithi096 podman[63503]: 2024-05-08 05:24:37.972418055 +0000 UTC m=+0.162570892 container create e8fde80f3d5533d5cdd65658667f4a585540e4636585314bb1bb54677ff9f745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T05:24:38.238 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:24:38 smithi096 podman[63503]: 2024-05-08 05:24:38.237849315 +0000 UTC m=+0.428002152 container init e8fde80f3d5533d5cdd65658667f4a585540e4636585314bb1bb54677ff9f745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:24:38.530 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:24:38 smithi096 podman[63503]: 2024-05-08 05:24:38.240711552 +0000 UTC m=+0.430864391 container start e8fde80f3d5533d5cdd65658667f4a585540e4636585314bb1bb54677ff9f745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-05-08T05:24:38.530 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:24:38 smithi096 ceph-mds[63522]: -- 172.21.15.96:0/89703272 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a87fd07a40 con 0x55a87fcda800 2024-05-08T05:24:38.530 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:24:38 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k[63518]: starting mds.k at 2024-05-08T05:24:38.530 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:24:38 smithi096 bash[63503]: e8fde80f3d5533d5cdd65658667f4a585540e4636585314bb1bb54677ff9f745 2024-05-08T05:24:38.530 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:24:38 smithi096 systemd[1]: Started Ceph mds.k for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:24:38.558 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:24:38.821 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:24:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: Deploying daemon mds.j on smithi059 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6832/2593895617,v1:172.21.15.96:6833/2593895617] up:boot 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: fsmap 2 up:standby 2024-05-08T05:24:39.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:39 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:24:39.663 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: Deploying daemon mds.j on smithi059 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6832/2593895617,v1:172.21.15.96:6833/2593895617] up:boot 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: fsmap 2 up:standby 2024-05-08T05:24:39.664 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:39 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:24:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:39.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:39.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:39.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:39.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:39.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: Deploying daemon mds.j on smithi059 2024-05-08T05:24:39.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6832/2593895617,v1:172.21.15.96:6833/2593895617] up:boot 2024-05-08T05:24:39.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: fsmap 2 up:standby 2024-05-08T05:24:39.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:39 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:24:40.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 -- 172.21.15.59:0/982352339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc9806c590 msgr2=0x7fcc98110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:40.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/982352339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc9806c590 0x7fcc98110330 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b0a0 tx=0x7fcc9000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:24:40.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 -- 172.21.15.59:0/982352339 shutdown_connections 2024-05-08T05:24:40.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/982352339 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc9806c590 0x7fcc98110330 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:40.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/982352339 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc98109510 0x7fcc981098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:40.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/982352339 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9806bbd0 0x7fcc9806c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:40.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 -- 172.21.15.59:0/982352339 >> 172.21.15.59:0/982352339 conn(0x7fcc98079350 msgr2=0x7fcc98079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:40.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 -- 172.21.15.59:0/982352339 shutdown_connections 2024-05-08T05:24:40.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 -- 172.21.15.59:0/982352339 wait complete. 2024-05-08T05:24:40.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.853+0000 7fcc9dac0700 1 Processor -- start 2024-05-08T05:24:40.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc9dac0700 1 -- start start 2024-05-08T05:24:40.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc9dac0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc9806bbd0 0x7fcc9819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:40.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc9dac0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9806c590 0x7fcc9819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:40.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc9dac0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc98109510 0x7fcc98199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc9dac0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcc98113620 con 0x7fcc98109510 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc9dac0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981134a0 con 0x7fcc9806bbd0 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc9dac0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcc981137a0 con 0x7fcc9806c590 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc97fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc98109510 0x7fcc98199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc97fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc98109510 0x7fcc98199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37188/0 (socket says 172.21.15.59:37188) 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc97fff700 1 -- 172.21.15.59:0/3914656923 learned_addr learned my addr 172.21.15.59:0/3914656923 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc977fe700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc9806bbd0 0x7fcc9819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc96ffd700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9806c590 0x7fcc9819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc97fff700 1 -- 172.21.15.59:0/3914656923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9806c590 msgr2=0x7fcc9819fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc97fff700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9806c590 0x7fcc9819fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc97fff700 1 -- 172.21.15.59:0/3914656923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc9806bbd0 msgr2=0x7fcc9819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc97fff700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc9806bbd0 0x7fcc9819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc97fff700 1 -- 172.21.15.59:0/3914656923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9819a080 con 0x7fcc98109510 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc977fe700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc9806bbd0 0x7fcc9819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:24:40.857 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.854+0000 7fcc97fff700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc98109510 0x7fcc98199920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b830 tx=0x7fcc90004d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:40.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.855+0000 7fcc94ff9700 1 -- 172.21.15.59:0/3914656923 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90021070 con 0x7fcc98109510 2024-05-08T05:24:40.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.855+0000 7fcc94ff9700 1 -- 172.21.15.59:0/3914656923 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc90003c00 con 0x7fcc98109510 2024-05-08T05:24:40.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.855+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9819a310 con 0x7fcc98109510 2024-05-08T05:24:40.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.855+0000 7fcc94ff9700 1 -- 172.21.15.59:0/3914656923 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc90004730 con 0x7fcc98109510 2024-05-08T05:24:40.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.855+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc981a65e0 con 0x7fcc98109510 2024-05-08T05:24:40.858 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.856+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc9806a190 con 0x7fcc98109510 2024-05-08T05:24:40.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.857+0000 7fcc94ff9700 1 -- 172.21.15.59:0/3914656923 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fcc90003db0 con 0x7fcc98109510 2024-05-08T05:24:40.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.858+0000 7fcc94ff9700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc800b2900 0x7fcc800b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:40.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.858+0000 7fcc977fe700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc800b2900 0x7fcc800b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:40.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.858+0000 7fcc94ff9700 1 -- 172.21.15.59:0/3914656923 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fcc9000f020 con 0x7fcc98109510 2024-05-08T05:24:40.859 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.858+0000 7fcc977fe700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc800b2900 0x7fcc800b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005e90 tx=0x7fcc88005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:40.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:40.861+0000 7fcc94ff9700 1 -- 172.21.15.59:0/3914656923 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcc9002a020 con 0x7fcc98109510 2024-05-08T05:24:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.133+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcc9802e510 con 0x7fcc98109510 2024-05-08T05:24:41.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.133+0000 7fcc94ff9700 1 -- 172.21.15.59:0/3914656923 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fcc900aba30 con 0x7fcc98109510 2024-05-08T05:24:41.134 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:24:41.134 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-08T05:24:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc800b2900 msgr2=0x7fcc800b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc800b2900 0x7fcc800b4dc0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005e90 tx=0x7fcc88005d40 comp rx=0 tx=0).stop 2024-05-08T05:24:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc98109510 msgr2=0x7fcc98199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc98109510 0x7fcc98199920 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000b830 tx=0x7fcc90004d90 comp rx=0 tx=0).stop 2024-05-08T05:24:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 shutdown_connections 2024-05-08T05:24:41.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcc98109510 0x7fcc98199920 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:41.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcc800b2900 0x7fcc800b4dc0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:41.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcc9806bbd0 0x7fcc9819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:41.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 --2- 172.21.15.59:0/3914656923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcc9806c590 0x7fcc9819fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:41.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 >> 172.21.15.59:0/3914656923 conn(0x7fcc98079350 msgr2=0x7fcc98101140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:41.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 shutdown_connections 2024-05-08T05:24:41.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:41.136+0000 7fcc9dac0700 1 -- 172.21.15.59:0/3914656923 wait complete. 2024-05-08T05:24:41.137 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 3 2024-05-08T05:24:41.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:41 smithi143 ceph-mon[34337]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:41.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:41 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3914656923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:41.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:41 smithi096 ceph-mon[34253]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:41.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:41 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3914656923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:41.835 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:41 smithi059 ceph-mon[30304]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:41.835 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:41 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3914656923' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:43.036 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:24:42 smithi059 systemd[1]: Starting Ceph mds.j for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:24:43.302 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:24:43 smithi059 podman[70900]: 2024-05-08 05:24:42.939275801 +0000 UTC m=+0.018612296 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:24:43.303 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:24:43 smithi059 podman[70900]: 2024-05-08 05:24:43.058495516 +0000 UTC m=+0.137831984 container create 283f0a6ceacb0d265b1e61f1777c54e8aad1ad203e11725ae3ee249f5d267d69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-08T05:24:43.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:43 smithi059 ceph-mon[30304]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:43.586 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:24:43 smithi059 podman[70900]: 2024-05-08 05:24:43.302958227 +0000 UTC m=+0.382294696 container init 283f0a6ceacb0d265b1e61f1777c54e8aad1ad203e11725ae3ee249f5d267d69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:24:43.586 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:24:43 smithi059 podman[70900]: 2024-05-08 05:24:43.305961505 +0000 UTC m=+0.385297982 container start 283f0a6ceacb0d265b1e61f1777c54e8aad1ad203e11725ae3ee249f5d267d69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:24:43.587 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:24:43 smithi059 ceph-mds[70920]: -- 172.21.15.59:0/412129595 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c3a009fa40 con 0x55c3a0072800 2024-05-08T05:24:43.587 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:24:43 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j[70916]: starting mds.j at 2024-05-08T05:24:43.587 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:24:43 smithi059 bash[70900]: 283f0a6ceacb0d265b1e61f1777c54e8aad1ad203e11725ae3ee249f5d267d69 2024-05-08T05:24:43.587 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:24:43 smithi059 systemd[1]: Started Ceph mds.j for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:24:43.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:43 smithi143 ceph-mon[34337]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:43.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:43 smithi096 ceph-mon[34253]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:43.869 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:24:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.59:6834/3565849854,v1:172.21.15.59:6835/3565849854] up:boot 2024-05-08T05:24:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: fsmap 3 up:standby 2024-05-08T05:24:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:24:44.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:44.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:44.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:44.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:44.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:44.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:44.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:44 smithi143 ceph-mon[34337]: Deploying daemon mds.d on smithi059 2024-05-08T05:24:44.762 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.59:6834/3565849854,v1:172.21.15.59:6835/3565849854] up:boot 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: fsmap 3 up:standby 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:44.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:44 smithi096 ceph-mon[34253]: Deploying daemon mds.d on smithi059 2024-05-08T05:24:44.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.59:6834/3565849854,v1:172.21.15.59:6835/3565849854] up:boot 2024-05-08T05:24:44.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: fsmap 3 up:standby 2024-05-08T05:24:44.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:24:44.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:44.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:44.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:44.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:44.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:44.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:44.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:44 smithi059 ceph-mon[30304]: Deploying daemon mds.d on smithi059 2024-05-08T05:24:45.752 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:45 smithi059 ceph-mon[30304]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:45.752 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:24:45 smithi059 systemd[1]: Starting Ceph mds.d for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:24:45.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:45 smithi096 ceph-mon[34253]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:45.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:45 smithi143 ceph-mon[34337]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:46.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 -- 172.21.15.59:0/3431868661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102fe0 msgr2=0x7f51f0106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/3431868661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102fe0 0x7f51f0106fe0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f51d80099c0 tx=0x7f51d8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 -- 172.21.15.59:0/3431868661 shutdown_connections 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/3431868661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f01076c0 0x7f51f010f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/3431868661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51f0102630 0x7f51f0102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/3431868661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102fe0 0x7f51f0106fe0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 -- 172.21.15.59:0/3431868661 >> 172.21.15.59:0/3431868661 conn(0x7f51f0079350 msgr2=0x7f51f0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 -- 172.21.15.59:0/3431868661 shutdown_connections 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 -- 172.21.15.59:0/3431868661 wait complete. 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.971+0000 7f51f4e7e700 1 Processor -- start 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51f4e7e700 1 -- start start 2024-05-08T05:24:46.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51f4e7e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102630 0x7f51f019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51f4e7e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51f0102fe0 0x7f51f019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51f4e7e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f01076c0 0x7f51f0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51f4e7e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51f0112550 con 0x7f51f0102630 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51f4e7e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51f01123d0 con 0x7f51f01076c0 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51f4e7e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f51f01126d0 con 0x7f51f0102fe0 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51ee59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102630 0x7f51f019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51ee59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102630 0x7f51f019f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37222/0 (socket says 172.21.15.59:37222) 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51ee59c700 1 -- 172.21.15.59:0/78823412 learned_addr learned my addr 172.21.15.59:0/78823412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51e5d9b700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51f0102fe0 0x7f51f019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51eed9d700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f01076c0 0x7f51f0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51ee59c700 1 -- 172.21.15.59:0/78823412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51f0102fe0 msgr2=0x7f51f019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:46.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51ee59c700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51f0102fe0 0x7f51f019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:46.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51ee59c700 1 -- 172.21.15.59:0/78823412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f01076c0 msgr2=0x7f51f0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:46.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51ee59c700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f01076c0 0x7f51f0199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:46.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.972+0000 7f51ee59c700 1 -- 172.21.15.59:0/78823412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f51f019a0c0 con 0x7f51f0102630 2024-05-08T05:24:46.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.973+0000 7f51ee59c700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102630 0x7f51f019f7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c970 tx=0x7f51e000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:46.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.973+0000 7f51e7fff700 1 -- 172.21.15.59:0/78823412 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e00078d0 con 0x7f51f0102630 2024-05-08T05:24:46.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.973+0000 7f51e7fff700 1 -- 172.21.15.59:0/78823412 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51e0018480 con 0x7f51f0102630 2024-05-08T05:24:46.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.973+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f51f019a3b0 con 0x7f51f0102630 2024-05-08T05:24:46.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.973+0000 7f51e7fff700 1 -- 172.21.15.59:0/78823412 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51e000f6b0 con 0x7f51f0102630 2024-05-08T05:24:46.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.973+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51f01a6670 con 0x7f51f0102630 2024-05-08T05:24:46.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.974+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f51f006a190 con 0x7f51f0102630 2024-05-08T05:24:46.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.975+0000 7f51e7fff700 1 -- 172.21.15.59:0/78823412 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f51e0018930 con 0x7f51f0102630 2024-05-08T05:24:46.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.976+0000 7f51e7fff700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f51dc0b2900 0x7f51dc0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:46.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.976+0000 7f51e5d9b700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f51dc0b2900 0x7f51dc0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:46.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.976+0000 7f51e7fff700 1 -- 172.21.15.59:0/78823412 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f51e00e2070 con 0x7f51f0102630 2024-05-08T05:24:46.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.977+0000 7f51e5d9b700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f51dc0b2900 0x7f51dc0b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f51d8006010 tx=0x7f51d8009630 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:46.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:46.980+0000 7f51e7fff700 1 -- 172.21.15.59:0/78823412 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f51e00aa350 con 0x7f51f0102630 2024-05-08T05:24:47.086 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:24:46 smithi059 podman[71316]: 2024-05-08 05:24:46.737477388 +0000 UTC m=+0.265489253 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:24:47.086 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:24:47 smithi059 podman[71316]: 2024-05-08 05:24:47.067209462 +0000 UTC m=+0.595221308 container create 16357abb3bc047c131158640104819c73c814dd167a95a5ab4ef29f63700dbf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-d, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.license=GPLv2) 2024-05-08T05:24:47.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.245+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f51f019ae90 con 0x7f51f0102630 2024-05-08T05:24:47.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.245+0000 7f51e7fff700 1 -- 172.21.15.59:0/78823412 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f51e00a9c50 con 0x7f51f0102630 2024-05-08T05:24:47.247 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:24:47.247 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-05-08T05:24:47.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f51dc0b2900 msgr2=0x7f51dc0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:47.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f51dc0b2900 0x7f51dc0b4dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f51d8006010 tx=0x7f51d8009630 comp rx=0 tx=0).stop 2024-05-08T05:24:47.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102630 msgr2=0x7f51f019f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:47.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102630 0x7f51f019f7a0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f51e000c970 tx=0x7f51e000cc80 comp rx=0 tx=0).stop 2024-05-08T05:24:47.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 shutdown_connections 2024-05-08T05:24:47.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f51dc0b2900 0x7f51dc0b4dc0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:47.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51f01076c0 0x7f51f0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:47.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51f0102630 0x7f51f019f7a0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:47.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 --2- 172.21.15.59:0/78823412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51f0102fe0 0x7f51f019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:47.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 >> 172.21.15.59:0/78823412 conn(0x7f51f0079350 msgr2=0x7f51f006beb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:47.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 shutdown_connections 2024-05-08T05:24:47.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:47.248+0000 7f51f4e7e700 1 -- 172.21.15.59:0/78823412 wait complete. 2024-05-08T05:24:47.253 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 4 2024-05-08T05:24:47.508 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:24:47 smithi059 podman[71316]: 2024-05-08 05:24:47.329349417 +0000 UTC m=+0.857361264 container init 16357abb3bc047c131158640104819c73c814dd167a95a5ab4ef29f63700dbf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-d, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, ceph=True) 2024-05-08T05:24:47.509 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:24:47 smithi059 podman[71316]: 2024-05-08 05:24:47.332387868 +0000 UTC m=+0.860399708 container start 16357abb3bc047c131158640104819c73c814dd167a95a5ab4ef29f63700dbf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-d, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-08T05:24:47.509 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:24:47 smithi059 ceph-mds[71375]: -- 172.21.15.59:0/3052321505 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564980eb3a40 con 0x564980e86c00 2024-05-08T05:24:47.509 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:24:47 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-d[71358]: starting mds.d at 2024-05-08T05:24:47.509 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:24:47 smithi059 bash[71316]: 16357abb3bc047c131158640104819c73c814dd167a95a5ab4ef29f63700dbf8 2024-05-08T05:24:47.509 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:24:47 smithi059 systemd[1]: Started Ceph mds.d for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:24:47.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:47 smithi096 ceph-mon[34253]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:47.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:47 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/78823412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:47.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:47.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:47 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:47.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:47 smithi059 ceph-mon[30304]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:47.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:47 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/78823412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:47.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:47.799 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:47 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:47.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:47 smithi143 ceph-mon[34337]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:47.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:47 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/78823412' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:47.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:47.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:47 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:48.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:48 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.59:6836/4212370259,v1:172.21.15.59:6837/4212370259] up:boot 2024-05-08T05:24:48.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:48 smithi096 ceph-mon[34253]: fsmap 4 up:standby 2024-05-08T05:24:48.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:24:48.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:48.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:48.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:48.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:48 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:48.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:48 smithi096 ceph-mon[34253]: Deploying daemon mds.e on smithi096 2024-05-08T05:24:48.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:48 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.59:6836/4212370259,v1:172.21.15.59:6837/4212370259] up:boot 2024-05-08T05:24:48.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:48 smithi059 ceph-mon[30304]: fsmap 4 up:standby 2024-05-08T05:24:48.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:24:48.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:48.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:48.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:48.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:48 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:48.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:48 smithi059 ceph-mon[30304]: Deploying daemon mds.e on smithi096 2024-05-08T05:24:48.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:48 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.59:6836/4212370259,v1:172.21.15.59:6837/4212370259] up:boot 2024-05-08T05:24:48.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:48 smithi143 ceph-mon[34337]: fsmap 4 up:standby 2024-05-08T05:24:48.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:24:48.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:48.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:48.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:48.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:48 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:48.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:48 smithi143 ceph-mon[34337]: Deploying daemon mds.e on smithi096 2024-05-08T05:24:49.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:49 smithi059 ceph-mon[30304]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:49.977 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:49 smithi096 ceph-mon[34253]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:49.977 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:24:49 smithi096 systemd[1]: Starting Ceph mds.e for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:24:49.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:49 smithi143 ceph-mon[34337]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:50.091 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:24:50.275 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:24:49 smithi096 podman[63821]: 2024-05-08 05:24:49.879969192 +0000 UTC m=+0.017656300 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:24:50.276 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:24:50 smithi096 podman[63821]: 2024-05-08 05:24:50.013206774 +0000 UTC m=+0.150893942 container create e2474414a20601912da2a7e4ca99646eb189a3961a78574dc2baa6c462b7faa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-08T05:24:50.360 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:24:50.530 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:24:50 smithi096 podman[63821]: 2024-05-08 05:24:50.275995921 +0000 UTC m=+0.413683020 container init e2474414a20601912da2a7e4ca99646eb189a3961a78574dc2baa6c462b7faa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:24:50.530 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:24:50 smithi096 podman[63821]: 2024-05-08 05:24:50.278904325 +0000 UTC m=+0.416591425 container start e2474414a20601912da2a7e4ca99646eb189a3961a78574dc2baa6c462b7faa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-08T05:24:50.530 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:24:50 smithi096 ceph-mds[63840]: -- 172.21.15.96:0/2083020163 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558576aeda40 con 0x558576ac0800 2024-05-08T05:24:50.530 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:24:50 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e[63836]: starting mds.e at 2024-05-08T05:24:50.530 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:24:50 smithi096 bash[63821]: e2474414a20601912da2a7e4ca99646eb189a3961a78574dc2baa6c462b7faa9 2024-05-08T05:24:50.530 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:24:50 smithi096 systemd[1]: Started Ceph mds.e for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:24:51.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: Deploying daemon mds.f on smithi143 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] up:boot 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: fsmap 5 up:standby 2024-05-08T05:24:51.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:51 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:24:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: Deploying daemon mds.f on smithi143 2024-05-08T05:24:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] up:boot 2024-05-08T05:24:51.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: fsmap 5 up:standby 2024-05-08T05:24:51.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:51 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:24:51.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:51.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:51.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:51.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:51.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:51.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:51.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:51.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: Deploying daemon mds.f on smithi143 2024-05-08T05:24:51.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] up:boot 2024-05-08T05:24:51.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: fsmap 5 up:standby 2024-05-08T05:24:51.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:51 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:24:52.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 -- 172.21.15.59:0/2685070659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514103f00 msgr2=0x7f25141103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:52.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 --2- 172.21.15.59:0/2685070659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514103f00 0x7f25141103b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f250c00b0d0 tx=0x7f250c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 -- 172.21.15.59:0/2685070659 shutdown_connections 2024-05-08T05:24:52.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 --2- 172.21.15.59:0/2685070659 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2514103f00 0x7f25141103b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 --2- 172.21.15.59:0/2685070659 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f25141095e0 0x7f25141099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 --2- 172.21.15.59:0/2685070659 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2514103540 0x7f25141039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 -- 172.21.15.59:0/2685070659 >> 172.21.15.59:0/2685070659 conn(0x7f251407a650 msgr2=0x7f251407aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 -- 172.21.15.59:0/2685070659 shutdown_connections 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 -- 172.21.15.59:0/2685070659 wait complete. 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.120+0000 7f251a8fd700 1 Processor -- start 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f251a8fd700 1 -- start start 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f251a8fd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2514103540 0x7f25140757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f251a8fd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2514103f00 0x7f2514075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f251a8fd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25141095e0 0x7f2514076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f251a8fd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2514113700 con 0x7f25141095e0 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f251a8fd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2514113580 con 0x7f2514103540 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f251a8fd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2514113880 con 0x7f2514103f00 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f2518e9a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25141095e0 0x7f2514076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:52.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f2518e9a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25141095e0 0x7f2514076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32990/0 (socket says 172.21.15.59:32990) 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f2518e9a700 1 -- 172.21.15.59:0/688139758 learned_addr learned my addr 172.21.15.59:0/688139758 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f2513fff700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2514103540 0x7f25140757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f25137fe700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2514103f00 0x7f2514075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f2518e9a700 1 -- 172.21.15.59:0/688139758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2514103f00 msgr2=0x7f2514075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f2518e9a700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2514103f00 0x7f2514075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f2518e9a700 1 -- 172.21.15.59:0/688139758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2514103540 msgr2=0x7f25140757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f2518e9a700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2514103540 0x7f25140757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.121+0000 7f2518e9a700 1 -- 172.21.15.59:0/688139758 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f25141a80a0 con 0x7f25141095e0 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.122+0000 7f2513fff700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2514103540 0x7f25140757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.122+0000 7f2518e9a700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25141095e0 0x7f2514076220 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f250c009850 tx=0x7f250c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.122+0000 7f25117fa700 1 -- 172.21.15.59:0/688139758 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c01f070 con 0x7f25141095e0 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.122+0000 7f25117fa700 1 -- 172.21.15.59:0/688139758 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f250c004c50 con 0x7f25141095e0 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.122+0000 7f251a8fd700 1 -- 172.21.15.59:0/688139758 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f25141a8330 con 0x7f25141095e0 2024-05-08T05:24:52.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.122+0000 7f25117fa700 1 -- 172.21.15.59:0/688139758 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f250c01b7b0 con 0x7f25141095e0 2024-05-08T05:24:52.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.122+0000 7f251a8fd700 1 -- 172.21.15.59:0/688139758 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25141a8780 con 0x7f25141095e0 2024-05-08T05:24:52.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.123+0000 7f24faffd700 1 -- 172.21.15.59:0/688139758 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24f4005330 con 0x7f25141095e0 2024-05-08T05:24:52.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.125+0000 7f25117fa700 1 -- 172.21.15.59:0/688139758 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f250c022070 con 0x7f25141095e0 2024-05-08T05:24:52.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.125+0000 7f25117fa700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f24fc0b2880 0x7f24fc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:52.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.125+0000 7f2513fff700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f24fc0b2880 0x7f24fc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:52.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.125+0000 7f25117fa700 1 -- 172.21.15.59:0/688139758 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f250c00f020 con 0x7f25141095e0 2024-05-08T05:24:52.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.126+0000 7f2513fff700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f24fc0b2880 0x7f24fc0b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2504005950 tx=0x7f250400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:52.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.129+0000 7f25117fa700 1 -- 172.21.15.59:0/688139758 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f250c029020 con 0x7f25141095e0 2024-05-08T05:24:52.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.391+0000 7f24faffd700 1 -- 172.21.15.59:0/688139758 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f24f4005620 con 0x7f25141095e0 2024-05-08T05:24:52.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.392+0000 7f25117fa700 1 -- 172.21.15.59:0/688139758 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4696 (secure 0 0 0) 0x7f250c009cb0 con 0x7f25141095e0 2024-05-08T05:24:52.394 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:24:52.394 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-05-08T05:24:52.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.394+0000 7f24faffd700 1 -- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f24fc0b2880 msgr2=0x7f24fc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:52.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f24fc0b2880 0x7f24fc0b4d40 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f2504005950 tx=0x7f250400a300 comp rx=0 tx=0).stop 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 -- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25141095e0 msgr2=0x7f2514076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25141095e0 0x7f2514076220 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f250c009850 tx=0x7f250c007a20 comp rx=0 tx=0).stop 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 -- 172.21.15.59:0/688139758 shutdown_connections 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f25141095e0 0x7f2514076220 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f24fc0b2880 0x7f24fc0b4d40 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2514103540 0x7f25140757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 --2- 172.21.15.59:0/688139758 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2514103f00 0x7f2514075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 -- 172.21.15.59:0/688139758 >> 172.21.15.59:0/688139758 conn(0x7f251407a650 msgr2=0x7f2514102100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 -- 172.21.15.59:0/688139758 shutdown_connections 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:52.395+0000 7f24faffd700 1 -- 172.21.15.59:0/688139758 wait complete. 2024-05-08T05:24:52.397 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 6 2024-05-08T05:24:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:52 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/688139758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:52.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:52 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/688139758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:52.743 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:24:52 smithi143 systemd[1]: Starting Ceph mds.f for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:24:52.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:52 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/688139758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:53.111 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:24:52 smithi143 podman[64670]: 2024-05-08 05:24:52.738873577 +0000 UTC m=+0.018212666 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:24:53.111 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:24:52 smithi143 podman[64670]: 2024-05-08 05:24:52.841446201 +0000 UTC m=+0.120785343 container create 33cb737861dada2f2d0525358a3b0dadcb89f4f8efda1a3e1451c2bafcaa36d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-08T05:24:53.111 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:24:53 smithi143 podman[64670]: 2024-05-08 05:24:53.110683145 +0000 UTC m=+0.390022228 container init 33cb737861dada2f2d0525358a3b0dadcb89f4f8efda1a3e1451c2bafcaa36d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:24:53.434 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:24:53 smithi143 podman[64670]: 2024-05-08 05:24:53.113681644 +0000 UTC m=+0.393020722 container start 33cb737861dada2f2d0525358a3b0dadcb89f4f8efda1a3e1451c2bafcaa36d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5) 2024-05-08T05:24:53.435 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:24:53 smithi143 ceph-mds[64689]: -- 172.21.15.143:0/2779658807 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557c0d4eba40 con 0x557c0d4bf000 2024-05-08T05:24:53.435 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:24:53 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f[64685]: starting mds.f at 2024-05-08T05:24:53.435 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:24:53 smithi143 bash[64670]: 33cb737861dada2f2d0525358a3b0dadcb89f4f8efda1a3e1451c2bafcaa36d2 2024-05-08T05:24:53.435 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:24:53 smithi143 systemd[1]: Started Ceph mds.f for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:24:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:53 smithi059 ceph-mon[30304]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:53 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:53.702 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:53 smithi143 ceph-mon[34337]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:53.703 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:53.703 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:53.703 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:53.703 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:53.703 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:53.703 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:53 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:53.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:53 smithi096 ceph-mon[34253]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:53.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:53.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:53.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:53.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:53.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:53.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:53 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:54.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:54 smithi143 ceph-mon[34337]: Deploying daemon mds.c on smithi143 2024-05-08T05:24:54.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:54 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6834/2504523054,v1:172.21.15.143:6835/2504523054] up:boot 2024-05-08T05:24:54.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:54 smithi143 ceph-mon[34337]: fsmap 6 up:standby 2024-05-08T05:24:54.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:54 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:24:54.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:54 smithi096 ceph-mon[34253]: Deploying daemon mds.c on smithi143 2024-05-08T05:24:54.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:54 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6834/2504523054,v1:172.21.15.143:6835/2504523054] up:boot 2024-05-08T05:24:54.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:54 smithi096 ceph-mon[34253]: fsmap 6 up:standby 2024-05-08T05:24:54.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:54 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:24:54.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:54 smithi059 ceph-mon[30304]: Deploying daemon mds.c on smithi143 2024-05-08T05:24:54.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:54 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6834/2504523054,v1:172.21.15.143:6835/2504523054] up:boot 2024-05-08T05:24:54.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:54 smithi059 ceph-mon[30304]: fsmap 6 up:standby 2024-05-08T05:24:54.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:54 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:24:55.045 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:24:55.314 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:24:55.447 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 systemd[1]: Starting Ceph mds.c for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:24:55.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:55 smithi059 ceph-mon[30304]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:55.733 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:55 smithi096 ceph-mon[34253]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:55.733 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:55 smithi143 ceph-mon[34337]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:55.734 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 podman[64983]: 2024-05-08 05:24:55.481337996 +0000 UTC m=+0.018125714 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:24:55.734 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 podman[64983]: 2024-05-08 05:24:55.612411888 +0000 UTC m=+0.149199607 container create 8388243eb8233eaae9dfb70a81d3ee08734c23305c61b7138d86972ab51fd573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD) 2024-05-08T05:24:56.243 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 podman[64983]: 2024-05-08 05:24:55.856882823 +0000 UTC m=+0.393670542 container init 8388243eb8233eaae9dfb70a81d3ee08734c23305c61b7138d86972ab51fd573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2) 2024-05-08T05:24:56.243 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 podman[64983]: 2024-05-08 05:24:55.859385186 +0000 UTC m=+0.396172905 container start 8388243eb8233eaae9dfb70a81d3ee08734c23305c61b7138d86972ab51fd573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0) 2024-05-08T05:24:56.243 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 ceph-mds[65001]: -- 172.21.15.143:0/3534110281 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x56301db164e0 con 0x56301daecc00 2024-05-08T05:24:56.243 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 ceph-mds[65001]: -- 172.21.15.143:0/3534110281 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56301db19a40 con 0x56301daecc00 2024-05-08T05:24:56.243 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c[64997]: starting mds.c at 2024-05-08T05:24:56.244 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 bash[64983]: 8388243eb8233eaae9dfb70a81d3ee08734c23305c61b7138d86972ab51fd573 2024-05-08T05:24:56.244 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:24:55 smithi143 systemd[1]: Started Ceph mds.c for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:24:57.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.036+0000 7fb212db5700 1 -- 172.21.15.59:0/1281353173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c109510 msgr2=0x7fb20c1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:57.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.036+0000 7fb212db5700 1 --2- 172.21.15.59:0/1281353173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c109510 0x7fb20c1098f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc009990 tx=0x7fb1fc009ca0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.036+0000 7fb212db5700 1 -- 172.21.15.59:0/1281353173 shutdown_connections 2024-05-08T05:24:57.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.036+0000 7fb212db5700 1 --2- 172.21.15.59:0/1281353173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb20c06bb40 0x7fb20c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.036+0000 7fb212db5700 1 --2- 172.21.15.59:0/1281353173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c109510 0x7fb20c1098f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.036+0000 7fb212db5700 1 --2- 172.21.15.59:0/1281353173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c06b180 0x7fb20c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.036+0000 7fb212db5700 1 -- 172.21.15.59:0/1281353173 >> 172.21.15.59:0/1281353173 conn(0x7fb20c079350 msgr2=0x7fb20c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.036+0000 7fb212db5700 1 -- 172.21.15.59:0/1281353173 shutdown_connections 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.036+0000 7fb212db5700 1 -- 172.21.15.59:0/1281353173 wait complete. 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.037+0000 7fb212db5700 1 Processor -- start 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.037+0000 7fb212db5700 1 -- start start 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.037+0000 7fb212db5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c06b180 0x7fb20c19bd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.037+0000 7fb212db5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c06bb40 0x7fb20c19c2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb212db5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb20c109510 0x7fb20c1a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb212db5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c113640 con 0x7fb20c06bb40 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb212db5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1134c0 con 0x7fb20c06b180 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb212db5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb20c1137c0 con 0x7fb20c109510 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb20bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c06bb40 0x7fb20c19c2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb20bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c06bb40 0x7fb20c19c2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33004/0 (socket says 172.21.15.59:33004) 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb20bfff700 1 -- 172.21.15.59:0/756486643 learned_addr learned my addr 172.21.15.59:0/756486643 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:24:57.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb210b51700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c06b180 0x7fb20c19bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb211352700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb20c109510 0x7fb20c1a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb20bfff700 1 -- 172.21.15.59:0/756486643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb20c109510 msgr2=0x7fb20c1a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb20bfff700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb20c109510 0x7fb20c1a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb20bfff700 1 -- 172.21.15.59:0/756486643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c06b180 msgr2=0x7fb20c19bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb20bfff700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c06b180 0x7fb20c19bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb20bfff700 1 -- 172.21.15.59:0/756486643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb20c1a0d80 con 0x7fb20c06bb40 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb20bfff700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c06bb40 0x7fb20c19c2b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb20000e9a0 tx=0x7fb20000ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb209ffb700 1 -- 172.21.15.59:0/756486643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20000cbd0 con 0x7fb20c06bb40 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb20c1a1070 con 0x7fb20c06bb40 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb209ffb700 1 -- 172.21.15.59:0/756486643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb20000cd70 con 0x7fb20c06bb40 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.038+0000 7fb209ffb700 1 -- 172.21.15.59:0/756486643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200018680 con 0x7fb20c06bb40 2024-05-08T05:24:57.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.039+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb20c1a88f0 con 0x7fb20c06bb40 2024-05-08T05:24:57.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.039+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb20c06a190 con 0x7fb20c06bb40 2024-05-08T05:24:57.043 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:57.043 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:57.043 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:57.043 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:57.043 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:57.043 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:57.043 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:57.043 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: Deploying daemon mds.h on smithi096 2024-05-08T05:24:57.044 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] up:boot 2024-05-08T05:24:57.044 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: fsmap 7 up:standby 2024-05-08T05:24:57.044 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:56 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:24:57.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.041+0000 7fb209ffb700 1 -- 172.21.15.59:0/756486643 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb200010970 con 0x7fb20c06bb40 2024-05-08T05:24:57.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.042+0000 7fb209ffb700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb1f40b2880 0x7fb1f40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:24:57.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.042+0000 7fb210b51700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb1f40b2880 0x7fb1f40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:24:57.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.042+0000 7fb209ffb700 1 -- 172.21.15.59:0/756486643 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fb2000e2590 con 0x7fb20c06bb40 2024-05-08T05:24:57.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.042+0000 7fb210b51700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb1f40b2880 0x7fb1f40b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006010 tx=0x7fb1fc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:24:57.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.047+0000 7fb209ffb700 1 -- 172.21.15.59:0/756486643 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb2000aa750 con 0x7fb20c06bb40 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: Deploying daemon mds.h on smithi096 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] up:boot 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: fsmap 7 up:standby 2024-05-08T05:24:57.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:56 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:24:57.309 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.308+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb20c19d0d0 con 0x7fb20c06bb40 2024-05-08T05:24:57.310 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.309+0000 7fb209ffb700 1 -- 172.21.15.59:0/756486643 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fb20c19d0d0 con 0x7fb20c06bb40 2024-05-08T05:24:57.310 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:24:57.313 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-05-08T05:24:57.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb1f40b2880 msgr2=0x7fb1f40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:57.314 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb1f40b2880 0x7fb1f40b4d40 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc006010 tx=0x7fb1fc01a040 comp rx=0 tx=0).stop 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c06bb40 msgr2=0x7fb20c19c2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c06bb40 0x7fb20c19c2b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb20000e9a0 tx=0x7fb20000ed60 comp rx=0 tx=0).stop 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 shutdown_connections 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb20c109510 0x7fb20c1a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb1f40b2880 0x7fb1f40b4d40 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb20c06b180 0x7fb20c19bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 --2- 172.21.15.59:0/756486643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb20c06bb40 0x7fb20c19c2b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 >> 172.21.15.59:0/756486643 conn(0x7fb20c079350 msgr2=0x7fb20c10e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 shutdown_connections 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:24:57.311+0000 7fb212db5700 1 -- 172.21.15.59:0/756486643 wait complete. 2024-05-08T05:24:57.315 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 8 2024-05-08T05:24:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:57.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:57.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: Deploying daemon mds.h on smithi096 2024-05-08T05:24:57.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] up:boot 2024-05-08T05:24:57.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: fsmap 7 up:standby 2024-05-08T05:24:57.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:56 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:24:58.255 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:58 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/756486643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:58.255 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:24:58 smithi096 systemd[1]: Starting Ceph mds.h for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:24:58.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:58 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/756486643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:58.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:58 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/756486643' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:24:58.774 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:24:58 smithi096 podman[64133]: 2024-05-08 05:24:58.374813276 +0000 UTC m=+0.018414449 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:24:58.774 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:24:58 smithi096 podman[64133]: 2024-05-08 05:24:58.499366778 +0000 UTC m=+0.142967927 container create e5e7468a1174ef77d39f609e00b4427d6cb0bbe205924a7d8d5295096844232c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-05-08T05:24:58.774 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:24:58 smithi096 podman[64133]: 2024-05-08 05:24:58.773890698 +0000 UTC m=+0.417491855 container init e5e7468a1174ef77d39f609e00b4427d6cb0bbe205924a7d8d5295096844232c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-08T05:24:59.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:59 smithi096 ceph-mon[34253]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:59.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:59 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:59.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:59 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:59.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:59 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:59.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:59 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:59.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:59 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:59.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:24:59 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:59.030 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:24:58 smithi096 podman[64133]: 2024-05-08 05:24:58.776639349 +0000 UTC m=+0.420240497 container start e5e7468a1174ef77d39f609e00b4427d6cb0bbe205924a7d8d5295096844232c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:24:59.030 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:24:58 smithi096 ceph-mds[64151]: -- 172.21.15.96:0/1544837807 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560aa127ba40 con 0x560aa124ec00 2024-05-08T05:24:59.031 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:24:58 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h[64147]: starting mds.h at 2024-05-08T05:24:59.031 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:24:58 smithi096 bash[64133]: e5e7468a1174ef77d39f609e00b4427d6cb0bbe205924a7d8d5295096844232c 2024-05-08T05:24:59.031 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:24:58 smithi096 systemd[1]: Started Ceph mds.h for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:24:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:59 smithi059 ceph-mon[30304]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:59 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:59 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:59 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:59 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:59.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:59 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:59.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:24:59 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:24:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:59 smithi143 ceph-mon[34337]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:24:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:59 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:59 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:59 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:24:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:59 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:24:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:59 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:24:59.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:24:59 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:00.047 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:25:00.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:00 smithi143 ceph-mon[34337]: Deploying daemon mds.i on smithi143 2024-05-08T05:25:00.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:00 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] up:boot 2024-05-08T05:25:00.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:00 smithi143 ceph-mon[34337]: fsmap 8 up:standby 2024-05-08T05:25:00.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:00 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:25:00.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:00 smithi096 ceph-mon[34253]: Deploying daemon mds.i on smithi143 2024-05-08T05:25:00.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:00 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] up:boot 2024-05-08T05:25:00.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:00 smithi096 ceph-mon[34253]: fsmap 8 up:standby 2024-05-08T05:25:00.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:00 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:25:00.313 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:00 smithi059 ceph-mon[30304]: Deploying daemon mds.i on smithi143 2024-05-08T05:25:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:00 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] up:boot 2024-05-08T05:25:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:00 smithi059 ceph-mon[30304]: fsmap 8 up:standby 2024-05-08T05:25:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:00 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:25:01.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:01 smithi143 ceph-mon[34337]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:01.244 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:25:01 smithi143 systemd[1]: Starting Ceph mds.i for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:25:01.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:01 smithi096 ceph-mon[34253]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:01.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:01 smithi059 ceph-mon[30304]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:01.528 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:25:01 smithi143 podman[65289]: 2024-05-08 05:25:01.157643694 +0000 UTC m=+0.018440754 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:25:01.528 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:25:01 smithi143 podman[65289]: 2024-05-08 05:25:01.266482124 +0000 UTC m=+0.127279184 container create dcd9b13ab8a790385d7260297b33c521e854764ec425fe13df1518e04879b97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:25:01.528 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:25:01 smithi143 podman[65289]: 2024-05-08 05:25:01.50235747 +0000 UTC m=+0.363154529 container init dcd9b13ab8a790385d7260297b33c521e854764ec425fe13df1518e04879b97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:25:01.528 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:25:01 smithi143 podman[65289]: 2024-05-08 05:25:01.504920793 +0000 UTC m=+0.365717853 container start dcd9b13ab8a790385d7260297b33c521e854764ec425fe13df1518e04879b97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-05-08T05:25:01.993 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:25:01 smithi143 ceph-mds[65308]: -- 172.21.15.143:0/2667695772 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x564cb6f79a40 con 0x564cb6f4c800 2024-05-08T05:25:01.993 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:25:01 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i[65304]: starting mds.i at 2024-05-08T05:25:01.993 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:25:01 smithi143 bash[65289]: dcd9b13ab8a790385d7260297b33c521e854764ec425fe13df1518e04879b97a 2024-05-08T05:25:01.993 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:25:01 smithi143 systemd[1]: Started Ceph mds.i for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:25:02.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.007+0000 7fc220399700 1 -- 172.21.15.59:0/1714203171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218104100 msgr2=0x7fc218108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:02.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.007+0000 7fc220399700 1 --2- 172.21.15.59:0/1714203171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218104100 0x7fc218108120 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fc2080099c0 tx=0x7fc208009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.007+0000 7fc220399700 1 -- 172.21.15.59:0/1714203171 shutdown_connections 2024-05-08T05:25:02.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.007+0000 7fc220399700 1 --2- 172.21.15.59:0/1714203171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218108850 0x7fc2181103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.007+0000 7fc220399700 1 --2- 172.21.15.59:0/1714203171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc218103750 0x7fc218103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.007+0000 7fc220399700 1 --2- 172.21.15.59:0/1714203171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218104100 0x7fc218108120 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.007+0000 7fc220399700 1 -- 172.21.15.59:0/1714203171 >> 172.21.15.59:0/1714203171 conn(0x7fc21807a600 msgr2=0x7fc21807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:02.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.007+0000 7fc220399700 1 -- 172.21.15.59:0/1714203171 shutdown_connections 2024-05-08T05:25:02.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.007+0000 7fc220399700 1 -- 172.21.15.59:0/1714203171 wait complete. 2024-05-08T05:25:02.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc220399700 1 Processor -- start 2024-05-08T05:25:02.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc220399700 1 -- start start 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc220399700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218103750 0x7fc2180757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc220399700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218104100 0x7fc218075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc220399700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc218108850 0x7fc218076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc220399700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc218113630 con 0x7fc218103750 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc220399700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2181134b0 con 0x7fc218108850 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc220399700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2181137b0 con 0x7fc218104100 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc21e135700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218103750 0x7fc2180757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc21e135700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218103750 0x7fc2180757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52026/0 (socket says 172.21.15.59:52026) 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc21e135700 1 -- 172.21.15.59:0/3860253706 learned_addr learned my addr 172.21.15.59:0/3860253706 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:02.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc21d934700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218104100 0x7fc218075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.008+0000 7fc21e936700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc218108850 0x7fc218076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc21e135700 1 -- 172.21.15.59:0/3860253706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218104100 msgr2=0x7fc218075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc21e135700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218104100 0x7fc218075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc21e135700 1 -- 172.21.15.59:0/3860253706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc218108850 msgr2=0x7fc218076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc21e135700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc218108850 0x7fc218076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc21e135700 1 -- 172.21.15.59:0/3860253706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2181a8010 con 0x7fc218103750 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc21e135700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218103750 0x7fc2180757a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc21400d950 tx=0x7fc21400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc20f7fe700 1 -- 172.21.15.59:0/3860253706 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc214016070 con 0x7fc218103750 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc20f7fe700 1 -- 172.21.15.59:0/3860253706 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc21400fd40 con 0x7fc218103750 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2181a8300 con 0x7fc218103750 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc20f7fe700 1 -- 172.21.15.59:0/3860253706 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc21400e3b0 con 0x7fc218103750 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.009+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2181a8810 con 0x7fc218103750 2024-05-08T05:25:02.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.010+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc218104be0 con 0x7fc218103750 2024-05-08T05:25:02.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.013+0000 7fc20f7fe700 1 -- 172.21.15.59:0/3860253706 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc214019070 con 0x7fc218103750 2024-05-08T05:25:02.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.013+0000 7fc20f7fe700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2040b2880 0x7fc2040b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:02.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.014+0000 7fc21d934700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2040b2880 0x7fc2040b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:02.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.014+0000 7fc20f7fe700 1 -- 172.21.15.59:0/3860253706 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc2140e17c0 con 0x7fc218103750 2024-05-08T05:25:02.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.014+0000 7fc21d934700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2040b2880 0x7fc2040b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc208009990 tx=0x7fc20800b790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:02.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.016+0000 7fc20f7fe700 1 -- 172.21.15.59:0/3860253706 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc2140a9980 con 0x7fc218103750 2024-05-08T05:25:02.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.302+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc218076cd0 con 0x7fc218103750 2024-05-08T05:25:02.304 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.303+0000 7fc20f7fe700 1 -- 172.21.15.59:0/3860253706 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8001 (secure 0 0 0) 0x7fc2140a9280 con 0x7fc218103750 2024-05-08T05:25:02.305 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:25:02.305 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-08T05:25:02.306 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2040b2880 msgr2=0x7fc2040b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2040b2880 0x7fc2040b4d40 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc208009990 tx=0x7fc20800b790 comp rx=0 tx=0).stop 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218103750 msgr2=0x7fc2180757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218103750 0x7fc2180757a0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc21400d950 tx=0x7fc21400dc60 comp rx=0 tx=0).stop 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 shutdown_connections 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc2040b2880 0x7fc2040b4d40 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc218108850 0x7fc218076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc218103750 0x7fc2180757a0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 --2- 172.21.15.59:0/3860253706 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc218104100 0x7fc218075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 >> 172.21.15.59:0/3860253706 conn(0x7fc21807a600 msgr2=0x7fc2181015a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 shutdown_connections 2024-05-08T05:25:02.307 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:02.305+0000 7fc220399700 1 -- 172.21.15.59:0/3860253706 wait complete. 2024-05-08T05:25:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 10 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: Deploying daemon mds.g on smithi059 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] up:boot 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: fsmap 9 up:standby 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:25:02.925 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:02 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3860253706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:02.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:02.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:02.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:02.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:25:02.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:25:02.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:02.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: Deploying daemon mds.g on smithi059 2024-05-08T05:25:02.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:02.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] up:boot 2024-05-08T05:25:02.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: fsmap 9 up:standby 2024-05-08T05:25:02.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:25:02.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:02 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3860253706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:03.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: Deploying daemon mds.g on smithi059 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] up:boot 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: fsmap 9 up:standby 2024-05-08T05:25:03.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:25:03.031 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:02 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3860253706' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:04.586 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:25:04 smithi059 systemd[1]: Starting Ceph mds.g for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:25:05.013 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:25:04 smithi059 podman[72150]: 2024-05-08 05:25:04.643700988 +0000 UTC m=+0.018810376 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:25:05.013 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:25:04 smithi059 podman[72150]: 2024-05-08 05:25:04.76090539 +0000 UTC m=+0.136014762 container create bbacb2744df2a0696c00a24e3b2bdc97046d60b8083d02b43b8afbb1194c9022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g, ceph=True, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-08T05:25:05.013 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:25:05 smithi059 podman[72150]: 2024-05-08 05:25:05.013512798 +0000 UTC m=+0.388622171 container init bbacb2744df2a0696c00a24e3b2bdc97046d60b8083d02b43b8afbb1194c9022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-08T05:25:05.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:05 smithi096 ceph-mon[34253]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:05 smithi059 ceph-mon[30304]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:05.336 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:25:05 smithi059 podman[72150]: 2024-05-08 05:25:05.016597255 +0000 UTC m=+0.391706631 container start bbacb2744df2a0696c00a24e3b2bdc97046d60b8083d02b43b8afbb1194c9022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:25:05.336 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:25:05 smithi059 ceph-mds[72169]: -- 172.21.15.59:0/3098683889 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5561fc721a40 con 0x5561fc6f4800 2024-05-08T05:25:05.336 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:25:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g[72165]: starting mds.g at 2024-05-08T05:25:05.336 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:25:05 smithi059 bash[72150]: bbacb2744df2a0696c00a24e3b2bdc97046d60b8083d02b43b8afbb1194c9022 2024-05-08T05:25:05.336 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:25:05 smithi059 systemd[1]: Started Ceph mds.g for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:25:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:05 smithi143 ceph-mon[34337]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:05.599 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:25:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:25:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:25:06.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:06.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: Deploying daemon mds.a on smithi059 2024-05-08T05:25:06.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.59:6838/2756426260,v1:172.21.15.59:6839/2756426260] up:boot 2024-05-08T05:25:06.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: fsmap 10 up:standby 2024-05-08T05:25:06.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: Deploying daemon mds.a on smithi059 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.59:6838/2756426260,v1:172.21.15.59:6839/2756426260] up:boot 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: fsmap 10 up:standby 2024-05-08T05:25:06.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:25:06.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:06.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:06.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:06.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:25:06.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:25:06.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:06.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: Deploying daemon mds.a on smithi059 2024-05-08T05:25:06.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.59:6838/2756426260,v1:172.21.15.59:6839/2756426260] up:boot 2024-05-08T05:25:06.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: fsmap 10 up:standby 2024-05-08T05:25:06.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:25:06.955 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:07.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:07 smithi059 ceph-mon[30304]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:07 smithi143 ceph-mon[34337]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:07 smithi096 ceph-mon[34253]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:09.145 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:25:08 smithi059 systemd[1]: Started Ceph mds.a for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:25:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.317+0000 7fca06f4f700 1 -- 172.21.15.59:0/3974292311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00103750 msgr2=0x7fca00103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.317+0000 7fca06f4f700 1 --2- 172.21.15.59:0/3974292311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00103750 0x7fca00103b30 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc9f00099c0 tx=0x7fc9f0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.317+0000 7fca06f4f700 1 -- 172.21.15.59:0/3974292311 shutdown_connections 2024-05-08T05:25:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.317+0000 7fca06f4f700 1 --2- 172.21.15.59:0/3974292311 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca00108850 0x7fca001103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.317+0000 7fca06f4f700 1 --2- 172.21.15.59:0/3974292311 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00103750 0x7fca00103b30 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.317+0000 7fca06f4f700 1 --2- 172.21.15.59:0/3974292311 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca00104100 0x7fca00108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.317+0000 7fca06f4f700 1 -- 172.21.15.59:0/3974292311 >> 172.21.15.59:0/3974292311 conn(0x7fca0007a600 msgr2=0x7fca0007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.317+0000 7fca06f4f700 1 -- 172.21.15.59:0/3974292311 shutdown_connections 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.317+0000 7fca06f4f700 1 -- 172.21.15.59:0/3974292311 wait complete. 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca06f4f700 1 Processor -- start 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca06f4f700 1 -- start start 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca06f4f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca00103750 0x7fca000757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca06f4f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca00104100 0x7fca00075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca06f4f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00108850 0x7fca00076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca06f4f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca00113640 con 0x7fca00108850 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca06f4f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca001134c0 con 0x7fca00103750 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca06f4f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fca001137c0 con 0x7fca00104100 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca054ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00108850 0x7fca00076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca054ec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00108850 0x7fca00076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33996/0 (socket says 172.21.15.59:33996) 2024-05-08T05:25:09.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca04ceb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca00103750 0x7fca000757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca054ec700 1 -- 172.21.15.59:0/2013819494 learned_addr learned my addr 172.21.15.59:0/2013819494 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fc9fffff700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca00104100 0x7fca00075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca054ec700 1 -- 172.21.15.59:0/2013819494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca00104100 msgr2=0x7fca00075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca054ec700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca00104100 0x7fca00075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca054ec700 1 -- 172.21.15.59:0/2013819494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca00103750 msgr2=0x7fca000757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca054ec700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca00103750 0x7fca000757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca054ec700 1 -- 172.21.15.59:0/2013819494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca001a8090 con 0x7fca00108850 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.318+0000 7fca04ceb700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca00103750 0x7fca000757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.319+0000 7fca054ec700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00108850 0x7fca00076220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8007ee0 tx=0x7fc9f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.319+0000 7fc9fdffb700 1 -- 172.21.15.59:0/2013819494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f800dbd0 con 0x7fca00108850 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.319+0000 7fc9fdffb700 1 -- 172.21.15.59:0/2013819494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9f800dd70 con 0x7fca00108850 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.319+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca001a8380 con 0x7fca00108850 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.319+0000 7fc9fdffb700 1 -- 172.21.15.59:0/2013819494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8005990 con 0x7fca00108850 2024-05-08T05:25:09.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.319+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca001a88c0 con 0x7fca00108850 2024-05-08T05:25:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.320+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca0010ba90 con 0x7fca00108850 2024-05-08T05:25:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.321+0000 7fc9fdffb700 1 -- 172.21.15.59:0/2013819494 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc9f801a020 con 0x7fca00108850 2024-05-08T05:25:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.322+0000 7fc9fdffb700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9e80b2900 0x7fc9e80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.323+0000 7fca04ceb700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9e80b2900 0x7fc9e80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.323+0000 7fc9fdffb700 1 -- 172.21.15.59:0/2013819494 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc9f80e14c0 con 0x7fca00108850 2024-05-08T05:25:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.323+0000 7fca04ceb700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9e80b2900 0x7fc9e80b4dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009990 tx=0x7fc9f0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:09.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.327+0000 7fc9fdffb700 1 -- 172.21.15.59:0/2013819494 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9f8010070 con 0x7fca00108850 2024-05-08T05:25:09.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:09 smithi059 ceph-mon[30304]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:09.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:09 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:09.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:09 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:09.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:09 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:09.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:09 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:25:09.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:09 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:25:09.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:09 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:09 smithi143 ceph-mon[34337]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:09 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:09 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:09 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:09 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:25:09.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:09 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:25:09.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:09 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:09.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:09 smithi096 ceph-mon[34253]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:09.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:09 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:09.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:09 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:09.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:09 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:09.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:09 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-05-08T05:25:09.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:09 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-05-08T05:25:09.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:09 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:09.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.589+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca0010b470 con 0x7fca00108850 2024-05-08T05:25:09.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.589+0000 7fc9fdffb700 1 -- 172.21.15.59:0/2013819494 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9653 (secure 0 0 0) 0x7fc9f80a95e0 con 0x7fca00108850 2024-05-08T05:25:09.590 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:25:09.590 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-05-08T05:25:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.591+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9e80b2900 msgr2=0x7fc9e80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.591+0000 7fca06f4f700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9e80b2900 0x7fc9e80b4dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fc9f0009990 tx=0x7fc9f0000bc0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.591+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00108850 msgr2=0x7fca00076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.591+0000 7fca06f4f700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00108850 0x7fca00076220 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8007ee0 tx=0x7fc9f800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.591+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 shutdown_connections 2024-05-08T05:25:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.591+0000 7fca06f4f700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca00108850 0x7fca00076220 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.591+0000 7fca06f4f700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9e80b2900 0x7fc9e80b4dc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.592+0000 7fca06f4f700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca00103750 0x7fca000757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.592+0000 7fca06f4f700 1 --2- 172.21.15.59:0/2013819494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca00104100 0x7fca00075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.592+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 >> 172.21.15.59:0/2013819494 conn(0x7fca0007a600 msgr2=0x7fca001099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.592+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 shutdown_connections 2024-05-08T05:25:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:09.592+0000 7fca06f4f700 1 -- 172.21.15.59:0/2013819494 wait complete. 2024-05-08T05:25:09.593 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 12 2024-05-08T05:25:10.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:10 smithi096 ceph-mon[34253]: Deploying daemon mds.b on smithi096 2024-05-08T05:25:10.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:10 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.59:6840/2496793439,v1:172.21.15.59:6841/2496793439] up:boot 2024-05-08T05:25:10.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:10 smithi096 ceph-mon[34253]: fsmap 11 up:standby 2024-05-08T05:25:10.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:10 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:25:10.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:10 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2013819494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:10.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:10 smithi143 ceph-mon[34337]: Deploying daemon mds.b on smithi096 2024-05-08T05:25:10.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:10 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.59:6840/2496793439,v1:172.21.15.59:6841/2496793439] up:boot 2024-05-08T05:25:10.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:10 smithi143 ceph-mon[34337]: fsmap 11 up:standby 2024-05-08T05:25:10.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:10 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:25:10.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:10 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2013819494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:10.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:10 smithi059 ceph-mon[30304]: Deploying daemon mds.b on smithi096 2024-05-08T05:25:10.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:10 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.59:6840/2496793439,v1:172.21.15.59:6841/2496793439] up:boot 2024-05-08T05:25:10.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:10 smithi059 ceph-mon[30304]: fsmap 11 up:standby 2024-05-08T05:25:10.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:10 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:25:10.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:10 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2013819494' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:11.451 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:11 smithi096 ceph-mon[34253]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:11 smithi143 ceph-mon[34337]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:11.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:11 smithi059 ceph-mon[30304]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:12.029 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:25:11 smithi096 systemd[1]: Started Ceph mds.b for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:25:12.308 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:25:12.575 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] up:boot 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: fsmap 12 up:standby 2024-05-08T05:25:12.952 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:12 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:25:13.178 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.179 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.179 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.179 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.179 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.179 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:13.179 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:25:13.179 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] up:boot 2024-05-08T05:25:13.179 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: fsmap 12 up:standby 2024-05-08T05:25:13.179 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:12 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:25:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.196 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:25:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] up:boot 2024-05-08T05:25:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: fsmap 12 up:standby 2024-05-08T05:25:13.197 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:12 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:25:14.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.258+0000 7fc9ec81c700 1 -- 172.21.15.59:0/3256889903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e81076c0 msgr2=0x7fc9e810f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.258+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/3256889903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e81076c0 0x7fc9e810f290 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fc9e000b0d0 tx=0x7fc9e000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.258+0000 7fc9ec81c700 1 -- 172.21.15.59:0/3256889903 shutdown_connections 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.258+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/3256889903 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e81076c0 0x7fc9e810f290 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.258+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/3256889903 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9e8102630 0x7fc9e8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.258+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/3256889903 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9e8102fe0 0x7fc9e8106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.258+0000 7fc9ec81c700 1 -- 172.21.15.59:0/3256889903 >> 172.21.15.59:0/3256889903 conn(0x7fc9e8079350 msgr2=0x7fc9e8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.258+0000 7fc9ec81c700 1 -- 172.21.15.59:0/3256889903 shutdown_connections 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.258+0000 7fc9ec81c700 1 -- 172.21.15.59:0/3256889903 wait complete. 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9ec81c700 1 Processor -- start 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9ec81c700 1 -- start start 2024-05-08T05:25:14.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9ec81c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e8102630 0x7fc9e8104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9ec81c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9e8102fe0 0x7fc9e81045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9ec81c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9e81076c0 0x7fc9e8104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9ec81c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e8112560 con 0x7fc9e8102630 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9ec81c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81123e0 con 0x7fc9e8102fe0 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9ec81c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc9e81126e0 con 0x7fc9e81076c0 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9e659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e8102630 0x7fc9e8104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9e659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e8102630 0x7fc9e8104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34014/0 (socket says 172.21.15.59:34014) 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9e659c700 1 -- 172.21.15.59:0/950909308 learned_addr learned my addr 172.21.15.59:0/950909308 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9e6d9d700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9e81076c0 0x7fc9e8104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.259+0000 7fc9e5d9b700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9e8102fe0 0x7fc9e81045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9e659c700 1 -- 172.21.15.59:0/950909308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9e81076c0 msgr2=0x7fc9e8104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:14.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9e659c700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9e81076c0 0x7fc9e8104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9e659c700 1 -- 172.21.15.59:0/950909308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9e8102fe0 msgr2=0x7fc9e81045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9e659c700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9e8102fe0 0x7fc9e81045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9e659c700 1 -- 172.21.15.59:0/950909308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9e81a8080 con 0x7fc9e8102630 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9e659c700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e8102630 0x7fc9e8104080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800d980 tx=0x7fc9d800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9d77fe700 1 -- 172.21.15.59:0/950909308 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8016070 con 0x7fc9e8102630 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9e81a8370 con 0x7fc9e8102630 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9d77fe700 1 -- 172.21.15.59:0/950909308 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9d8004830 con 0x7fc9e8102630 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9d77fe700 1 -- 172.21.15.59:0/950909308 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9d8005540 con 0x7fc9e8102630 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.260+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9e81a8900 con 0x7fc9e8102630 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.261+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9e806a190 con 0x7fc9e8102630 2024-05-08T05:25:14.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.262+0000 7fc9d77fe700 1 -- 172.21.15.59:0/950909308 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fc9d8004220 con 0x7fc9e8102630 2024-05-08T05:25:14.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.263+0000 7fc9d77fe700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9d00b2880 0x7fc9d00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:14.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.263+0000 7fc9e5d9b700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9d00b2880 0x7fc9d00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:14.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.263+0000 7fc9d77fe700 1 -- 172.21.15.59:0/950909308 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7fc9d80e1b30 con 0x7fc9e8102630 2024-05-08T05:25:14.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.263+0000 7fc9e5d9b700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9d00b2880 0x7fc9d00b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8108cb0 tx=0x7fc9dc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:14.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.268+0000 7fc9d77fe700 1 -- 172.21.15.59:0/950909308 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc9d801f020 con 0x7fc9e8102630 2024-05-08T05:25:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.544+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc9e802e510 con 0x7fc9e8102630 2024-05-08T05:25:14.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.544+0000 7fc9d77fe700 1 -- 172.21.15.59:0/950909308 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10476 (secure 0 0 0) 0x7fc9d80a9cf0 con 0x7fc9e8102630 2024-05-08T05:25:14.545 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:25:14.545 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24473,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-05-08T05:25:14.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9d00b2880 msgr2=0x7fc9d00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:14.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9d00b2880 0x7fc9d00b4d40 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8108cb0 tx=0x7fc9dc009260 comp rx=0 tx=0).stop 2024-05-08T05:25:14.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e8102630 msgr2=0x7fc9e8104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:14.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e8102630 0x7fc9e8104080 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fc9d800d980 tx=0x7fc9d800dc90 comp rx=0 tx=0).stop 2024-05-08T05:25:14.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 shutdown_connections 2024-05-08T05:25:14.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc9e81076c0 0x7fc9e8104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc9e8102630 0x7fc9e8104080 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc9d00b2880 0x7fc9d00b4d40 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 --2- 172.21.15.59:0/950909308 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc9e8102fe0 0x7fc9e81045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 >> 172.21.15.59:0/950909308 conn(0x7fc9e8079350 msgr2=0x7fc9e806bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 shutdown_connections 2024-05-08T05:25:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:14.547+0000 7fc9ec81c700 1 -- 172.21.15.59:0/950909308 wait complete. 2024-05-08T05:25:14.548 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 13 2024-05-08T05:25:15.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:14 smithi059 ceph-mon[30304]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:15.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:14 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.177 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:14 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/950909308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:15.205 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph --format=json-pretty osd lspools 2024-05-08T05:25:15.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:14 smithi143 ceph-mon[34337]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:15.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:14 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:14 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/950909308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:15.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:14 smithi096 ceph-mon[34253]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:15.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:14 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:15.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:14 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/950909308' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:15.798 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:17.266 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:17 smithi059 ceph-mon[30304]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:17.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:17 smithi096 ceph-mon[34253]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:17.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:17 smithi143 ceph-mon[34337]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:17.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.496+0000 7f90b0b2f700 1 -- 172.21.15.59:0/4078339159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac1076c0 msgr2=0x7f90ac10f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:17.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.496+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/4078339159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac1076c0 0x7f90ac10f290 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f90a40099c0 tx=0x7f90a4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.496+0000 7f90b0b2f700 1 -- 172.21.15.59:0/4078339159 shutdown_connections 2024-05-08T05:25:17.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.496+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/4078339159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac1076c0 0x7f90ac10f290 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.496+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/4078339159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac102630 0x7f90ac102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.496+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/4078339159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac102fe0 0x7f90ac106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.496+0000 7f90b0b2f700 1 -- 172.21.15.59:0/4078339159 >> 172.21.15.59:0/4078339159 conn(0x7f90ac079350 msgr2=0x7f90ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:17.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.496+0000 7f90b0b2f700 1 -- 172.21.15.59:0/4078339159 shutdown_connections 2024-05-08T05:25:17.497 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.496+0000 7f90b0b2f700 1 -- 172.21.15.59:0/4078339159 wait complete. 2024-05-08T05:25:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90b0b2f700 1 Processor -- start 2024-05-08T05:25:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90b0b2f700 1 -- start start 2024-05-08T05:25:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90b0b2f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac102630 0x7f90ac104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90b0b2f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac102fe0 0x7f90ac1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90b0b2f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac1076c0 0x7f90ac104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90b0b2f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac112580 con 0x7f90ac102fe0 2024-05-08T05:25:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90b0b2f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac112400 con 0x7f90ac102630 2024-05-08T05:25:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90b0b2f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f90ac112700 con 0x7f90ac1076c0 2024-05-08T05:25:17.499 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90a9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac102fe0 0x7f90ac1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90a9d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac102fe0 0x7f90ac1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34028/0 (socket says 172.21.15.59:34028) 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90a9d9b700 1 -- 172.21.15.59:0/1601174988 learned_addr learned my addr 172.21.15.59:0/1601174988 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90aa59c700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac102630 0x7f90ac104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90aad9d700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac1076c0 0x7f90ac104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90a9d9b700 1 -- 172.21.15.59:0/1601174988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac1076c0 msgr2=0x7f90ac104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90a9d9b700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac1076c0 0x7f90ac104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90a9d9b700 1 -- 172.21.15.59:0/1601174988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac102630 msgr2=0x7f90ac104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90a9d9b700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac102630 0x7f90ac104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.497+0000 7f90a9d9b700 1 -- 172.21.15.59:0/1601174988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90ac1a80a0 con 0x7f90ac102fe0 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.498+0000 7f90a9d9b700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac102fe0 0x7f90ac1045c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f90a000d980 tx=0x7f90a000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.498+0000 7f909b7fe700 1 -- 172.21.15.59:0/1601174988 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a0017070 con 0x7f90ac102fe0 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.498+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90ac1a8390 con 0x7f90ac102fe0 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.498+0000 7f909b7fe700 1 -- 172.21.15.59:0/1601174988 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90a0009de0 con 0x7f90ac102fe0 2024-05-08T05:25:17.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.498+0000 7f909b7fe700 1 -- 172.21.15.59:0/1601174988 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90a000f3c0 con 0x7f90ac102fe0 2024-05-08T05:25:17.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.498+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90ac1a88d0 con 0x7f90ac102fe0 2024-05-08T05:25:17.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.499+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f90ac06a190 con 0x7f90ac102fe0 2024-05-08T05:25:17.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.500+0000 7f909b7fe700 1 -- 172.21.15.59:0/1601174988 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f90a0009930 con 0x7f90ac102fe0 2024-05-08T05:25:17.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.501+0000 7f909b7fe700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90940b2880 0x7f90940b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:17.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.502+0000 7f90aa59c700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90940b2880 0x7f90940b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:17.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.502+0000 7f909b7fe700 1 -- 172.21.15.59:0/1601174988 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f90a00e18c0 con 0x7f90ac102fe0 2024-05-08T05:25:17.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.502+0000 7f90aa59c700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90940b2880 0x7f90940b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f909c00f540 tx=0x7f909c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:17.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.505+0000 7f909b7fe700 1 -- 172.21.15.59:0/1601174988 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f90a00a99e0 con 0x7f90ac102fe0 2024-05-08T05:25:17.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.736+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f90ac1055b0 con 0x7f90ac102fe0 2024-05-08T05:25:17.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.736+0000 7f909b7fe700 1 -- 172.21.15.59:0/1601174988 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v72) ==== 84+0+65 (secure 0 0 0) 0x7f90a00a92e0 con 0x7f90ac102fe0 2024-05-08T05:25:17.737 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:25:17.737 INFO:teuthology.orchestra.run.smithi059.stdout:[ 2024-05-08T05:25:17.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:25:17.737 INFO:teuthology.orchestra.run.smithi059.stdout: "poolnum": 1, 2024-05-08T05:25:17.737 INFO:teuthology.orchestra.run.smithi059.stdout: "poolname": ".mgr" 2024-05-08T05:25:17.737 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:25:17.738 INFO:teuthology.orchestra.run.smithi059.stdout:] 2024-05-08T05:25:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90940b2880 msgr2=0x7f90940b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:17.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90940b2880 0x7f90940b4d40 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f909c00f540 tx=0x7f909c00b040 comp rx=0 tx=0).stop 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac102fe0 msgr2=0x7f90ac1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac102fe0 0x7f90ac1045c0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f90a000d980 tx=0x7f90a000dc90 comp rx=0 tx=0).stop 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 shutdown_connections 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90ac1076c0 0x7f90ac104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90940b2880 0x7f90940b4d40 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90ac102630 0x7f90ac104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 --2- 172.21.15.59:0/1601174988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90ac102fe0 0x7f90ac1045c0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 >> 172.21.15.59:0/1601174988 conn(0x7f90ac079350 msgr2=0x7f90ac06bee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 shutdown_connections 2024-05-08T05:25:17.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:17.739+0000 7f90b0b2f700 1 -- 172.21.15.59:0/1601174988 wait complete. 2024-05-08T05:25:18.364 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-05-08T05:25:18.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-05-08T05:25:18.626 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:18.867 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:18.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:18.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:18 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1601174988' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T05:25:18.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:18 smithi059 ceph-mon[30304]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:18.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:18.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:25:18.868 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:18 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:18.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:18.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:18.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:18 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1601174988' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T05:25:18.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:18 smithi143 ceph-mon[34337]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:18.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:18.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:25:18.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:18 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:19.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:19.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:19.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:18 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1601174988' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-05-08T05:25:19.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:18 smithi096 ceph-mon[34253]: pgmap v279: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:19.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:19.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:25:19.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:18 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:19.770 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:19 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:25:19.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:19 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:25:20.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:19 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:25:20.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.962+0000 7f5390239700 1 -- 172.21.15.59:0/3491148380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 msgr2=0x7f53881103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:20.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.962+0000 7f5390239700 1 --2- 172.21.15.59:0/3491148380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 0x7f53881103f0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f538000b0d0 tx=0x7f538000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:25:20.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.962+0000 7f5390239700 1 -- 172.21.15.59:0/3491148380 shutdown_connections 2024-05-08T05:25:20.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.962+0000 7f5390239700 1 --2- 172.21.15.59:0/3491148380 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 0x7f53881103f0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:20.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.962+0000 7f5390239700 1 --2- 172.21.15.59:0/3491148380 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5388103780 0x7f5388103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.962+0000 7f5390239700 1 --2- 172.21.15.59:0/3491148380 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5388104130 0x7f5388108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.962+0000 7f5390239700 1 -- 172.21.15.59:0/3491148380 >> 172.21.15.59:0/3491148380 conn(0x7f538807a610 msgr2=0x7f538807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.962+0000 7f5390239700 1 -- 172.21.15.59:0/3491148380 shutdown_connections 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.962+0000 7f5390239700 1 -- 172.21.15.59:0/3491148380 wait complete. 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.963+0000 7f5390239700 1 Processor -- start 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.963+0000 7f5390239700 1 -- start start 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.963+0000 7f5390239700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5388103780 0x7f538819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f5390239700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5388104130 0x7f538819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f5390239700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 0x7f53881998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538e7d6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 0x7f53881998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:20.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f5390239700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5388113640 con 0x7f5388108870 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f5390239700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53881134c0 con 0x7f5388103780 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f5390239700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f53881137c0 con 0x7f5388104130 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538e7d6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 0x7f53881998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36600/0 (socket says 172.21.15.59:36600) 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538e7d6700 1 -- 172.21.15.59:0/2553164254 learned_addr learned my addr 172.21.15.59:0/2553164254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538dfd5700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5388103780 0x7f538819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538d7d4700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5388104130 0x7f538819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538e7d6700 1 -- 172.21.15.59:0/2553164254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5388104130 msgr2=0x7f538819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538e7d6700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5388104130 0x7f538819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538e7d6700 1 -- 172.21.15.59:0/2553164254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5388103780 msgr2=0x7f538819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538e7d6700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5388103780 0x7f538819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538e7d6700 1 -- 172.21.15.59:0/2553164254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f538819a020 con 0x7f5388108870 2024-05-08T05:25:20.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538d7d4700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5388104130 0x7f538819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:25:20.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f538e7d6700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 0x7f53881998c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5380009850 tx=0x7f5380007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:20.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.964+0000 7f537effd700 1 -- 172.21.15.59:0/2553164254 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538001f070 con 0x7f5388108870 2024-05-08T05:25:20.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.965+0000 7f537effd700 1 -- 172.21.15.59:0/2553164254 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5380005520 con 0x7f5388108870 2024-05-08T05:25:20.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.965+0000 7f537effd700 1 -- 172.21.15.59:0/2553164254 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f538001b870 con 0x7f5388108870 2024-05-08T05:25:20.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.965+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f538819a2b0 con 0x7f5388108870 2024-05-08T05:25:20.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.965+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53881a6580 con 0x7f5388108870 2024-05-08T05:25:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.966+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f538806a190 con 0x7f5388108870 2024-05-08T05:25:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.967+0000 7f537effd700 1 -- 172.21.15.59:0/2553164254 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f5380022070 con 0x7f5388108870 2024-05-08T05:25:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.968+0000 7f537effd700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53740b2980 0x7f53740b4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.968+0000 7f537effd700 1 -- 172.21.15.59:0/2553164254 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7127+0+0 (secure 0 0 0) 0x7f538000f020 con 0x7f5388108870 2024-05-08T05:25:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.968+0000 7f538dfd5700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53740b2980 0x7f53740b4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:20.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.969+0000 7f538dfd5700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53740b2980 0x7f53740b4e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5388108370 tx=0x7f5384009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:20.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:20.973+0000 7f537effd700 1 -- 172.21.15.59:0/2553164254 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5380029020 con 0x7f5388108870 2024-05-08T05:25:21.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.244+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f538819ada0 con 0x7f5388108870 2024-05-08T05:25:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2553164254' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:25:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:25:21.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:21 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.620+0000 7f537effd700 1 -- 172.21.15.59:0/2553164254 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v73) ==== 138+0+0 (secure 0 0 0) 0x7f53800ab0c0 con 0x7f5388108870 2024-05-08T05:25:21.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53740b2980 msgr2=0x7f53740b4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:21.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53740b2980 0x7f53740b4e40 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f5388108370 tx=0x7f5384009260 comp rx=0 tx=0).stop 2024-05-08T05:25:21.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 msgr2=0x7f53881998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:21.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 0x7f53881998c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f5380009850 tx=0x7f5380007ad0 comp rx=0 tx=0).stop 2024-05-08T05:25:21.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 shutdown_connections 2024-05-08T05:25:21.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5388108870 0x7f53881998c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:21.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53740b2980 0x7f53740b4e40 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:21.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5388103780 0x7f538819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:21.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 --2- 172.21.15.59:0/2553164254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5388104130 0x7f538819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:21.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.624+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 >> 172.21.15.59:0/2553164254 conn(0x7f538807a610 msgr2=0x7f53881016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:21.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.625+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 shutdown_connections 2024-05-08T05:25:21.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:21.625+0000 7f5390239700 1 -- 172.21.15.59:0/2553164254 wait complete. 2024-05-08T05:25:21.628 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' created 2024-05-08T05:25:21.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2553164254' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:25:21.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:21 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: pgmap v280: 1 pgs: 1 active+clean; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2553164254' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:25:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:21 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:25:22.351 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-05-08T05:25:22.610 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:22.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:22 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2553164254' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T05:25:22.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:22 smithi059 ceph-mon[30304]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T05:25:22.886 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:22 smithi059 ceph-mon[30304]: pgmap v282: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:22.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:22 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2553164254' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T05:25:22.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:22 smithi143 ceph-mon[34337]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T05:25:22.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:22 smithi143 ceph-mon[34337]: pgmap v282: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:23.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:22 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2553164254' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-05-08T05:25:23.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:22 smithi096 ceph-mon[34253]: osdmap e73: 12 total, 12 up, 12 in 2024-05-08T05:25:23.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:22 smithi096 ceph-mon[34253]: pgmap v282: 33 pgs: 3 creating+peering, 1 active+clean, 29 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:23.690 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:23 smithi059 ceph-mon[30304]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:25:23.691 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:23 smithi059 ceph-mon[30304]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T05:25:23.951 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:23 smithi096 ceph-mon[34253]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:25:23.952 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:23 smithi096 ceph-mon[34253]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T05:25:23.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:23 smithi143 ceph-mon[34337]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:25:23.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:23 smithi143 ceph-mon[34337]: osdmap e74: 12 total, 12 up, 12 in 2024-05-08T05:25:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.249+0000 7f4162399700 1 -- 172.21.15.59:0/3853324922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 msgr2=0x7f415c1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.249+0000 7f4162399700 1 --2- 172.21.15.59:0/3853324922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 0x7f415c1103d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f415400b0d0 tx=0x7f415400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.249+0000 7f4162399700 1 -- 172.21.15.59:0/3853324922 shutdown_connections 2024-05-08T05:25:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.249+0000 7f4162399700 1 --2- 172.21.15.59:0/3853324922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 0x7f415c1103d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.249+0000 7f4162399700 1 --2- 172.21.15.59:0/3853324922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c103750 0x7f415c103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.249+0000 7f4162399700 1 --2- 172.21.15.59:0/3853324922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f415c104100 0x7f415c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.249+0000 7f4162399700 1 -- 172.21.15.59:0/3853324922 >> 172.21.15.59:0/3853324922 conn(0x7f415c07a600 msgr2=0x7f415c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.249+0000 7f4162399700 1 -- 172.21.15.59:0/3853324922 shutdown_connections 2024-05-08T05:25:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.249+0000 7f4162399700 1 -- 172.21.15.59:0/3853324922 wait complete. 2024-05-08T05:25:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.250+0000 7f4162399700 1 Processor -- start 2024-05-08T05:25:24.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.250+0000 7f4162399700 1 -- start start 2024-05-08T05:25:24.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4162399700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f415c103750 0x7f415c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:24.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4162399700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c104100 0x7f415c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:24.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4162399700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 0x7f415c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:24.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4162399700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f415c113640 con 0x7f415c108850 2024-05-08T05:25:24.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4162399700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f415c1134c0 con 0x7f415c103750 2024-05-08T05:25:24.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4162399700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f415c1137c0 con 0x7f415c104100 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4160936700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 0x7f415c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f415bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f415c103750 0x7f415c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4160936700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 0x7f415c076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36606/0 (socket says 172.21.15.59:36606) 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f415bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f415c103750 0x7f415c0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:41328/0 (socket says 172.21.15.59:41328) 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4160936700 1 -- 172.21.15.59:0/889774412 learned_addr learned my addr 172.21.15.59:0/889774412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f415b7fe700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c104100 0x7f415c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4160936700 1 -- 172.21.15.59:0/889774412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c104100 msgr2=0x7f415c075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4160936700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c104100 0x7f415c075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4160936700 1 -- 172.21.15.59:0/889774412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f415c103750 msgr2=0x7f415c0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4160936700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f415c103750 0x7f415c0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.251+0000 7f4160936700 1 -- 172.21.15.59:0/889774412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f415c1a7fe0 con 0x7f415c108850 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.252+0000 7f415bfff700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f415c103750 0x7f415c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.252+0000 7f4160936700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 0x7f415c076220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f41540095c0 tx=0x7f4154007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.252+0000 7f41597fa700 1 -- 172.21.15.59:0/889774412 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415401f070 con 0x7f415c108850 2024-05-08T05:25:24.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.252+0000 7f41597fa700 1 -- 172.21.15.59:0/889774412 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4154003c00 con 0x7f415c108850 2024-05-08T05:25:24.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.252+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f415c1a8270 con 0x7f415c108850 2024-05-08T05:25:24.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.252+0000 7f41597fa700 1 -- 172.21.15.59:0/889774412 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f415401b6a0 con 0x7f415c108850 2024-05-08T05:25:24.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.252+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f415c1a86c0 con 0x7f415c108850 2024-05-08T05:25:24.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.254+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f415c10ba20 con 0x7f415c108850 2024-05-08T05:25:24.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.255+0000 7f41597fa700 1 -- 172.21.15.59:0/889774412 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f4154022070 con 0x7f415c108850 2024-05-08T05:25:24.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.255+0000 7f41597fa700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f41440b2880 0x7f41440b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:24.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.255+0000 7f415bfff700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f41440b2880 0x7f41440b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:24.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.255+0000 7f41597fa700 1 -- 172.21.15.59:0/889774412 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 7497+0+0 (secure 0 0 0) 0x7f415400f020 con 0x7f415c108850 2024-05-08T05:25:24.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.256+0000 7f415bfff700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f41440b2880 0x7f41440b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f414c0104e0 tx=0x7f414c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:24.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.260+0000 7f41597fa700 1 -- 172.21.15.59:0/889774412 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4154029020 con 0x7f415c108850 2024-05-08T05:25:24.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.476+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f415c02e5b0 con 0x7f415c108850 2024-05-08T05:25:24.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.642+0000 7f41597fa700 1 -- 172.21.15.59:0/889774412 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v76) ==== 171+0+0 (secure 0 0 0) 0x7f41540ab230 con 0x7f415c108850 2024-05-08T05:25:24.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f41440b2880 msgr2=0x7f41440b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:24.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f41440b2880 0x7f41440b4d40 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f414c0104e0 tx=0x7f414c00b3c0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 msgr2=0x7f415c076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:24.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 0x7f415c076220 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f41540095c0 tx=0x7f4154007a20 comp rx=0 tx=0).stop 2024-05-08T05:25:24.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 shutdown_connections 2024-05-08T05:25:24.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f415c108850 0x7f415c076220 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f41440b2880 0x7f41440b4d40 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f415c103750 0x7f415c0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 --2- 172.21.15.59:0/889774412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f415c104100 0x7f415c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.646+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 >> 172.21.15.59:0/889774412 conn(0x7f415c07a600 msgr2=0x7f415c101710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.647+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 shutdown_connections 2024-05-08T05:25:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:24.647+0000 7f4162399700 1 -- 172.21.15.59:0/889774412 wait complete. 2024-05-08T05:25:24.648 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' created 2024-05-08T05:25:24.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:24 smithi096 ceph-mon[34253]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T05:25:24.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:24 smithi096 ceph-mon[34253]: pgmap v285: 33 pgs: 9 creating+peering, 7 active+clean, 17 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:24.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:24 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/889774412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T05:25:24.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:24 smithi059 ceph-mon[30304]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T05:25:24.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:24 smithi059 ceph-mon[30304]: pgmap v285: 33 pgs: 9 creating+peering, 7 active+clean, 17 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:24.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:24 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/889774412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T05:25:24.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:24 smithi143 ceph-mon[34337]: osdmap e75: 12 total, 12 up, 12 in 2024-05-08T05:25:24.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:24 smithi143 ceph-mon[34337]: pgmap v285: 33 pgs: 9 creating+peering, 7 active+clean, 17 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:24.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:24 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/889774412' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-05-08T05:25:25.360 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-05-08T05:25:25.625 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:25.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:25 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/889774412' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T05:25:25.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:25 smithi143 ceph-mon[34337]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T05:25:26.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:25 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/889774412' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T05:25:26.003 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:25 smithi059 ceph-mon[30304]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T05:25:26.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:25 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/889774412' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-05-08T05:25:26.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:25 smithi096 ceph-mon[34253]: osdmap e76: 12 total, 12 up, 12 in 2024-05-08T05:25:26.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:26 smithi059 ceph-mon[30304]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T05:25:26.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:26 smithi059 ceph-mon[30304]: pgmap v288: 97 pgs: 21 creating+peering, 37 active+clean, 39 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:26.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:26 smithi143 ceph-mon[34337]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T05:25:26.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:26 smithi143 ceph-mon[34337]: pgmap v288: 97 pgs: 21 creating+peering, 37 active+clean, 39 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:27.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:26 smithi096 ceph-mon[34253]: osdmap e77: 12 total, 12 up, 12 in 2024-05-08T05:25:27.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:26 smithi096 ceph-mon[34253]: pgmap v288: 97 pgs: 21 creating+peering, 37 active+clean, 39 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.329+0000 7f4269460700 1 -- 172.21.15.59:0/3504511149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264104100 msgr2=0x7f4264108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.329+0000 7f4269460700 1 --2- 172.21.15.59:0/3504511149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264104100 0x7f4264108120 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f42580099c0 tx=0x7f4258009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.329+0000 7f4269460700 1 -- 172.21.15.59:0/3504511149 shutdown_connections 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.329+0000 7f4269460700 1 --2- 172.21.15.59:0/3504511149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4264108850 0x7f42641103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.329+0000 7f4269460700 1 --2- 172.21.15.59:0/3504511149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4264103750 0x7f4264103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.329+0000 7f4269460700 1 --2- 172.21.15.59:0/3504511149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264104100 0x7f4264108120 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.329+0000 7f4269460700 1 -- 172.21.15.59:0/3504511149 >> 172.21.15.59:0/3504511149 conn(0x7f426407a600 msgr2=0x7f426407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.329+0000 7f4269460700 1 -- 172.21.15.59:0/3504511149 shutdown_connections 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.329+0000 7f4269460700 1 -- 172.21.15.59:0/3504511149 wait complete. 2024-05-08T05:25:27.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4269460700 1 Processor -- start 2024-05-08T05:25:27.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4269460700 1 -- start start 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4269460700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264103750 0x7f426419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4269460700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4264104100 0x7f426419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4269460700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4264108850 0x7f42641998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4269460700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4264113690 con 0x7f4264103750 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4269460700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4264113510 con 0x7f4264108850 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4262ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264103750 0x7f426419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4269460700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4264113810 con 0x7f4264104100 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4262ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264103750 0x7f426419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:36626/0 (socket says 172.21.15.59:36626) 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4262ffd700 1 -- 172.21.15.59:0/639427409 learned_addr learned my addr 172.21.15.59:0/639427409 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f42637fe700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4264108850 0x7f42641998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f42627fc700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4264104100 0x7f426419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4262ffd700 1 -- 172.21.15.59:0/639427409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4264104100 msgr2=0x7f426419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:27.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4262ffd700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4264104100 0x7f426419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4262ffd700 1 -- 172.21.15.59:0/639427409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4264108850 msgr2=0x7f42641998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4262ffd700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4264108850 0x7f42641998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f4262ffd700 1 -- 172.21.15.59:0/639427409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426419a050 con 0x7f4264103750 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.330+0000 7f42627fc700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4264104100 0x7f426419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.331+0000 7f4262ffd700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264103750 0x7f426419f730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f425400c9b0 tx=0x7f425400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.331+0000 7f424bfff700 1 -- 172.21.15.59:0/639427409 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254007b00 con 0x7f4264103750 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.331+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426419a340 con 0x7f4264103750 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.331+0000 7f424bfff700 1 -- 172.21.15.59:0/639427409 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4254007ca0 con 0x7f4264103750 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.331+0000 7f424bfff700 1 -- 172.21.15.59:0/639427409 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4254018660 con 0x7f4264103750 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.331+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42641a6600 con 0x7f4264103750 2024-05-08T05:25:27.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.332+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f426406a190 con 0x7f4264103750 2024-05-08T05:25:27.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.333+0000 7f424bfff700 1 -- 172.21.15.59:0/639427409 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f425400f450 con 0x7f4264103750 2024-05-08T05:25:27.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.334+0000 7f424bfff700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f424c0b2880 0x7f424c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:27.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.334+0000 7f42627fc700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f424c0b2880 0x7f424c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:27.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.334+0000 7f424bfff700 1 -- 172.21.15.59:0/639427409 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 7879+0+0 (secure 0 0 0) 0x7f42540e2040 con 0x7f4264103750 2024-05-08T05:25:27.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.335+0000 7f42627fc700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f424c0b2880 0x7f424c0b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4258003800 tx=0x7f4258009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:27.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.339+0000 7f424bfff700 1 -- 172.21.15.59:0/639427409 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f4254014030 con 0x7f4264103750 2024-05-08T05:25:27.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.600+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f426402e510 con 0x7f4264103750 2024-05-08T05:25:27.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.661+0000 7f424bfff700 1 -- 172.21.15.59:0/639427409 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f42540a9f10 con 0x7f4264103750 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f424c0b2880 msgr2=0x7f424c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f424c0b2880 0x7f424c0b4d40 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f4258003800 tx=0x7f4258009f10 comp rx=0 tx=0).stop 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264103750 msgr2=0x7f426419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264103750 0x7f426419f730 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f425400c9b0 tx=0x7f425400ccc0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 shutdown_connections 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f424c0b2880 0x7f424c0b4d40 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4264108850 0x7f42641998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4264103750 0x7f426419f730 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 --2- 172.21.15.59:0/639427409 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4264104100 0x7f426419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 >> 172.21.15.59:0/639427409 conn(0x7f426407a600 msgr2=0x7f4264102c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 shutdown_connections 2024-05-08T05:25:27.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:27.664+0000 7f4269460700 1 -- 172.21.15.59:0/639427409 wait complete. 2024-05-08T05:25:27.666 INFO:teuthology.orchestra.run.smithi059.stderr:new fs with metadata pool 2 and data pool 3 2024-05-08T05:25:27.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:27 smithi143 ceph-mon[34337]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T05:25:27.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:27 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/639427409' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T05:25:27.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:27 smithi143 ceph-mon[34337]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T05:25:28.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:27 smithi096 ceph-mon[34253]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T05:25:28.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:27 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/639427409' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T05:25:28.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:27 smithi096 ceph-mon[34253]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T05:25:28.039 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:27 smithi059 ceph-mon[30304]: osdmap e78: 12 total, 12 up, 12 in 2024-05-08T05:25:28.040 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:27 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/639427409' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-05-08T05:25:28.040 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:27 smithi059 ceph-mon[30304]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-05-08T05:25:28.513 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-05-08T05:25:28.513 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-05-08T05:25:28.771 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:28.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:28 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/639427409' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T05:25:28.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:28 smithi059 ceph-mon[30304]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T05:25:28.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:28 smithi059 ceph-mon[30304]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T05:25:28.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:28 smithi059 ceph-mon[30304]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T05:25:28.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:28 smithi059 ceph-mon[30304]: pgmap v291: 97 pgs: 12 creating+peering, 71 active+clean, 14 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:28 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/639427409' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T05:25:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:28 smithi143 ceph-mon[34337]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T05:25:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:28 smithi143 ceph-mon[34337]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T05:25:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:28 smithi143 ceph-mon[34337]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T05:25:28.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:28 smithi143 ceph-mon[34337]: pgmap v291: 97 pgs: 12 creating+peering, 71 active+clean, 14 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:29.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:28 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/639427409' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-05-08T05:25:29.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:28 smithi096 ceph-mon[34253]: osdmap e79: 12 total, 12 up, 12 in 2024-05-08T05:25:29.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:28 smithi096 ceph-mon[34253]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-05-08T05:25:29.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:28 smithi096 ceph-mon[34253]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-05-08T05:25:29.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:28 smithi096 ceph-mon[34253]: pgmap v291: 97 pgs: 12 creating+peering, 71 active+clean, 14 unknown; 577 KiB data, 346 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:29 smithi143 ceph-mon[34337]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T05:25:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:29 smithi143 ceph-mon[34337]: Cluster is now healthy 2024-05-08T05:25:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:29 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] up:active 2024-05-08T05:25:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:29 smithi143 ceph-mon[34337]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:25:29.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:29 smithi143 ceph-mon[34337]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:25:30.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:29 smithi096 ceph-mon[34253]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T05:25:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:29 smithi096 ceph-mon[34253]: Cluster is now healthy 2024-05-08T05:25:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:29 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] up:active 2024-05-08T05:25:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:29 smithi096 ceph-mon[34253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:25:30.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:29 smithi096 ceph-mon[34253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:25:30.053 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:29 smithi059 ceph-mon[30304]: Health check cleared: POOL_APP_NOT_ENABLED (was: 2 pool(s) do not have an application enabled) 2024-05-08T05:25:30.053 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:29 smithi059 ceph-mon[30304]: Cluster is now healthy 2024-05-08T05:25:30.053 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:29 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] up:active 2024-05-08T05:25:30.053 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:29 smithi059 ceph-mon[30304]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:25:30.053 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:29 smithi059 ceph-mon[30304]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:25:30.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.474+0000 7fdd05111700 1 -- 172.21.15.59:0/3492876406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00104100 msgr2=0x7fdd00108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:30.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.474+0000 7fdd05111700 1 --2- 172.21.15.59:0/3492876406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00104100 0x7fdd00108120 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7fdcf40099c0 tx=0x7fdcf4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:30.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.474+0000 7fdd05111700 1 -- 172.21.15.59:0/3492876406 shutdown_connections 2024-05-08T05:25:30.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.474+0000 7fdd05111700 1 --2- 172.21.15.59:0/3492876406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd00108850 0x7fdd001103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:30.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.474+0000 7fdd05111700 1 --2- 172.21.15.59:0/3492876406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd00103750 0x7fdd00103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:30.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.474+0000 7fdd05111700 1 --2- 172.21.15.59:0/3492876406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00104100 0x7fdd00108120 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:30.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.474+0000 7fdd05111700 1 -- 172.21.15.59:0/3492876406 >> 172.21.15.59:0/3492876406 conn(0x7fdd0007a600 msgr2=0x7fdd0007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:30.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.474+0000 7fdd05111700 1 -- 172.21.15.59:0/3492876406 shutdown_connections 2024-05-08T05:25:30.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.474+0000 7fdd05111700 1 -- 172.21.15.59:0/3492876406 wait complete. 2024-05-08T05:25:30.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdd05111700 1 Processor -- start 2024-05-08T05:25:30.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdd05111700 1 -- start start 2024-05-08T05:25:30.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdd05111700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd00103750 0x7fdd0019f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:30.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdd05111700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd00104100 0x7fdd0019fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdd05111700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00108850 0x7fdd001999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdd05111700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00113710 con 0x7fdd00108850 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdd05111700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00113590 con 0x7fdd00103750 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdd05111700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdd00113890 con 0x7fdd00104100 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdcff59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00108850 0x7fdd001999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdcff59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00108850 0x7fdd001999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41796/0 (socket says 172.21.15.59:41796) 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdcff59e700 1 -- 172.21.15.59:0/1737271019 learned_addr learned my addr 172.21.15.59:0/1737271019 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdcfed9d700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd00103750 0x7fdd0019f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.475+0000 7fdcfe59c700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd00104100 0x7fdd0019fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdcff59e700 1 -- 172.21.15.59:0/1737271019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd00104100 msgr2=0x7fdd0019fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdcff59e700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd00104100 0x7fdd0019fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdcff59e700 1 -- 172.21.15.59:0/1737271019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd00103750 msgr2=0x7fdd0019f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:30.478 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdcff59e700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd00103750 0x7fdd0019f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:30.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdcff59e700 1 -- 172.21.15.59:0/1737271019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0019a130 con 0x7fdd00108850 2024-05-08T05:25:30.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdcfed9d700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd00103750 0x7fdd0019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:25:30.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdcff59e700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00108850 0x7fdd001999d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800b970 tx=0x7fdcf800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:30.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdce7fff700 1 -- 172.21.15.59:0/1737271019 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800d860 con 0x7fdd00108850 2024-05-08T05:25:30.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdce7fff700 1 -- 172.21.15.59:0/1737271019 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdcf800e430 con 0x7fdd00108850 2024-05-08T05:25:30.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdce7fff700 1 -- 172.21.15.59:0/1737271019 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8017660 con 0x7fdd00108850 2024-05-08T05:25:30.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0019a420 con 0x7fdd00108850 2024-05-08T05:25:30.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.476+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd001a66e0 con 0x7fdd00108850 2024-05-08T05:25:30.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.477+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0006a190 con 0x7fdd00108850 2024-05-08T05:25:30.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.479+0000 7fdce7fff700 1 -- 172.21.15.59:0/1737271019 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fdcf800da00 con 0x7fdd00108850 2024-05-08T05:25:30.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.479+0000 7fdce7fff700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fdce80b2880 0x7fdce80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:30.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.479+0000 7fdcfed9d700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fdce80b2880 0x7fdce80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:30.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.479+0000 7fdce7fff700 1 -- 172.21.15.59:0/1737271019 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 7979+0+0 (secure 0 0 0) 0x7fdcf80e1e80 con 0x7fdd00108850 2024-05-08T05:25:30.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.480+0000 7fdcfed9d700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fdce80b2880 0x7fdce80b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd0019ab10 tx=0x7fdcf0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:30.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.483+0000 7fdce7fff700 1 -- 172.21.15.59:0/1737271019 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdcf8053df0 con 0x7fdd00108850 2024-05-08T05:25:30.713 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:30.712+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7fdd0019af00 con 0x7fdd00108850 2024-05-08T05:25:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:30 smithi143 ceph-mon[34337]: pgmap v292: 97 pgs: 12 creating+peering, 85 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-05-08T05:25:31.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:30 smithi096 ceph-mon[34253]: pgmap v292: 97 pgs: 12 creating+peering, 85 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-05-08T05:25:31.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:30 smithi059 ceph-mon[30304]: pgmap v292: 97 pgs: 12 creating+peering, 85 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 1 op/s 2024-05-08T05:25:31.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.687+0000 7fdce7fff700 1 -- 172.21.15.59:0/1737271019 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v80) ==== 157+0+0 (secure 0 0 0) 0x7fdcf8013030 con 0x7fdd00108850 2024-05-08T05:25:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fdce80b2880 msgr2=0x7fdce80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:31.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fdce80b2880 0x7fdce80b4d40 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fdd0019ab10 tx=0x7fdcf0009190 comp rx=0 tx=0).stop 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00108850 msgr2=0x7fdd001999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00108850 0x7fdd001999d0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800b970 tx=0x7fdcf800bc80 comp rx=0 tx=0).stop 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 shutdown_connections 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdd00108850 0x7fdd001999d0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fdce80b2880 0x7fdce80b4d40 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdd00103750 0x7fdd0019f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 --2- 172.21.15.59:0/1737271019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdd00104100 0x7fdd0019fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 >> 172.21.15.59:0/1737271019 conn(0x7fdd0007a600 msgr2=0x7fdd00101800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 shutdown_connections 2024-05-08T05:25:31.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:31.692+0000 7fdd05111700 1 -- 172.21.15.59:0/1737271019 wait complete. 2024-05-08T05:25:31.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:31 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1737271019' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T05:25:32.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:31 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1737271019' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T05:25:32.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:31 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1737271019' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-05-08T05:25:32.530 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-05-08T05:25:32.531 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:25:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b[64457]: 2024-05-08T05:25:32.248+0000 7f00205c7700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-05-08T05:25:32.792 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:32.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:32 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1737271019' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T05:25:32.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:32 smithi059 ceph-mon[30304]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T05:25:32.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:32 smithi059 ceph-mon[30304]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-05-08T05:25:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:32 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1737271019' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T05:25:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:32 smithi143 ceph-mon[34337]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T05:25:32.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:32 smithi143 ceph-mon[34337]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-05-08T05:25:33.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:32 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1737271019' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-05-08T05:25:33.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:32 smithi096 ceph-mon[34253]: osdmap e80: 12 total, 12 up, 12 in 2024-05-08T05:25:33.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:32 smithi096 ceph-mon[34253]: pgmap v294: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s wr, 3 op/s 2024-05-08T05:25:34.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.445+0000 7f8f6759e700 1 -- 172.21.15.59:0/757844720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68103e30 msgr2=0x7f8f681042b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:34.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.445+0000 7f8f6759e700 1 --2- 172.21.15.59:0/757844720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68103e30 0x7f8f681042b0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f8f500099c0 tx=0x7f8f50009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:34.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.445+0000 7f8f6759e700 1 -- 172.21.15.59:0/757844720 shutdown_connections 2024-05-08T05:25:34.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.445+0000 7f8f6759e700 1 --2- 172.21.15.59:0/757844720 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f681047f0 0x7f8f6810f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:34.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.445+0000 7f8f6759e700 1 --2- 172.21.15.59:0/757844720 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68102540 0x7f8f68102920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:34.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.445+0000 7f8f6759e700 1 --2- 172.21.15.59:0/757844720 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68103e30 0x7f8f681042b0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:34.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.445+0000 7f8f6759e700 1 -- 172.21.15.59:0/757844720 >> 172.21.15.59:0/757844720 conn(0x7f8f680fdcf0 msgr2=0x7f8f68100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:34.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.445+0000 7f8f6759e700 1 -- 172.21.15.59:0/757844720 shutdown_connections 2024-05-08T05:25:34.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.445+0000 7f8f6759e700 1 -- 172.21.15.59:0/757844720 wait complete. 2024-05-08T05:25:34.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.446+0000 7f8f6759e700 1 Processor -- start 2024-05-08T05:25:34.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.446+0000 7f8f6759e700 1 -- start start 2024-05-08T05:25:34.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6759e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68102540 0x7f8f6819f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:34.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6759e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68103e30 0x7f8f6819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:34.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6759e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f681047f0 0x7f8f68199770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:34.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6759e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8f681124a0 con 0x7f8f68102540 2024-05-08T05:25:34.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6759e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8f68112320 con 0x7f8f68103e30 2024-05-08T05:25:34.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6759e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8f68112620 con 0x7f8f681047f0 2024-05-08T05:25:34.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68102540 0x7f8f6819f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68102540 0x7f8f6819f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41814/0 (socket says 172.21.15.59:41814) 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6659c700 1 -- 172.21.15.59:0/3013043232 learned_addr learned my addr 172.21.15.59:0/3013043232 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f5dd9b700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68103e30 0x7f8f6819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f66d9d700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f681047f0 0x7f8f68199770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6659c700 1 -- 172.21.15.59:0/3013043232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f681047f0 msgr2=0x7f8f68199770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6659c700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f681047f0 0x7f8f68199770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6659c700 1 -- 172.21.15.59:0/3013043232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68103e30 msgr2=0x7f8f6819fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6659c700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68103e30 0x7f8f6819fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6659c700 1 -- 172.21.15.59:0/3013043232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8f6819a050 con 0x7f8f68102540 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f5dd9b700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68103e30 0x7f8f6819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.447+0000 7f8f6659c700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68102540 0x7f8f6819f6a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800c920 tx=0x7f8f5800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:34.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.448+0000 7f8f5ffff700 1 -- 172.21.15.59:0/3013043232 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58004a30 con 0x7f8f68102540 2024-05-08T05:25:34.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.448+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8f6819a340 con 0x7f8f68102540 2024-05-08T05:25:34.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.448+0000 7f8f5ffff700 1 -- 172.21.15.59:0/3013043232 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8f580054d0 con 0x7f8f68102540 2024-05-08T05:25:34.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.448+0000 7f8f5ffff700 1 -- 172.21.15.59:0/3013043232 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8f58007c70 con 0x7f8f68102540 2024-05-08T05:25:34.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.448+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8f681a6620 con 0x7f8f68102540 2024-05-08T05:25:34.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.449+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8f68103940 con 0x7f8f68102540 2024-05-08T05:25:34.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.450+0000 7f8f5ffff700 1 -- 172.21.15.59:0/3013043232 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8f58004c60 con 0x7f8f68102540 2024-05-08T05:25:34.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.451+0000 7f8f5ffff700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8f540b2930 0x7f8f540b4df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:34.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.451+0000 7f8f5dd9b700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8f540b2930 0x7f8f540b4df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:34.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.451+0000 7f8f5ffff700 1 -- 172.21.15.59:0/3013043232 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8277+0+0 (secure 0 0 0) 0x7f8f580e2100 con 0x7f8f68102540 2024-05-08T05:25:34.452 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.451+0000 7f8f5dd9b700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8f540b2930 0x7f8f540b4df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009990 tx=0x7f8f50000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:34.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.455+0000 7f8f5ffff700 1 -- 172.21.15.59:0/3013043232 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8f58014030 con 0x7f8f68102540 2024-05-08T05:25:34.671 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:34.670+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f8f6819aca0 con 0x7f8f68102540 2024-05-08T05:25:35.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:35 smithi059 ceph-mon[30304]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-08T05:25:35.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:35 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3013043232' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:25:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:35 smithi143 ceph-mon[34337]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-08T05:25:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:35 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3013043232' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:25:35.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:35 smithi096 ceph-mon[34253]: pgmap v295: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 6 op/s 2024-05-08T05:25:35.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:35 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3013043232' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-05-08T05:25:36.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.034+0000 7f8f5ffff700 1 -- 172.21.15.59:0/3013043232 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v82) ==== 228+0+0 (secure 0 0 0) 0x7f8f580b0e20 con 0x7f8f68102540 2024-05-08T05:25:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8f540b2930 msgr2=0x7f8f540b4df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:36.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8f540b2930 0x7f8f540b4df0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f8f50009990 tx=0x7f8f50000bc0 comp rx=0 tx=0).stop 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68102540 msgr2=0x7f8f6819f6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68102540 0x7f8f6819f6a0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f8f5800c920 tx=0x7f8f5800cc30 comp rx=0 tx=0).stop 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 shutdown_connections 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8f681047f0 0x7f8f68199770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8f68102540 0x7f8f6819f6a0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8f540b2930 0x7f8f540b4df0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 --2- 172.21.15.59:0/3013043232 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8f68103e30 0x7f8f6819fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 >> 172.21.15.59:0/3013043232 conn(0x7f8f680fdcf0 msgr2=0x7f8f681071b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 shutdown_connections 2024-05-08T05:25:36.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:36.038+0000 7f8f6759e700 1 -- 172.21.15.59:0/3013043232 wait complete. 2024-05-08T05:25:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data_ec' created 2024-05-08T05:25:36.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:36 smithi059 ceph-mon[30304]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T05:25:36.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:36 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3013043232' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T05:25:36.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:36 smithi059 ceph-mon[30304]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T05:25:36.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:36 smithi143 ceph-mon[34337]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T05:25:36.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:36 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3013043232' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T05:25:36.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:36 smithi143 ceph-mon[34337]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T05:25:36.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:36 smithi096 ceph-mon[34253]: osdmap e81: 12 total, 12 up, 12 in 2024-05-08T05:25:36.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:36 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3013043232' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-05-08T05:25:36.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:36 smithi096 ceph-mon[34253]: osdmap e82: 12 total, 12 up, 12 in 2024-05-08T05:25:36.735 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-05-08T05:25:36.991 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:37.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:37 smithi059 ceph-mon[30304]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-08T05:25:37.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:37 smithi059 ceph-mon[30304]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T05:25:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:37 smithi143 ceph-mon[34337]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-08T05:25:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:37 smithi143 ceph-mon[34337]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T05:25:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:37 smithi096 ceph-mon[34253]: pgmap v297: 97 pgs: 97 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-05-08T05:25:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:37 smithi096 ceph-mon[34253]: osdmap e83: 12 total, 12 up, 12 in 2024-05-08T05:25:38.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.756+0000 7f81ab9a5700 1 -- 172.21.15.59:0/1019927298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a410ff00 msgr2=0x7f81a41122f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:38.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.756+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/1019927298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a410ff00 0x7f81a41122f0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f819c00b0a0 tx=0x7f819c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:25:38.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.756+0000 7f81ab9a5700 1 -- 172.21.15.59:0/1019927298 shutdown_connections 2024-05-08T05:25:38.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.756+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/1019927298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a410ff00 0x7f81a41122f0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:38.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.756+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/1019927298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a40756d0 0x7f81a4075ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:38.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.756+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/1019927298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81a4075ff0 0x7f81a410f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:38.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.756+0000 7f81ab9a5700 1 -- 172.21.15.59:0/1019927298 >> 172.21.15.59:0/1019927298 conn(0x7f81a407a580 msgr2=0x7f81a407a990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:38.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.756+0000 7f81ab9a5700 1 -- 172.21.15.59:0/1019927298 shutdown_connections 2024-05-08T05:25:38.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.756+0000 7f81ab9a5700 1 -- 172.21.15.59:0/1019927298 wait complete. 2024-05-08T05:25:38.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.757+0000 7f81ab9a5700 1 Processor -- start 2024-05-08T05:25:38.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.757+0000 7f81ab9a5700 1 -- start start 2024-05-08T05:25:38.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.757+0000 7f81ab9a5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a40756d0 0x7f81a419bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:38.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.757+0000 7f81ab9a5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a4075ff0 0x7f81a419c300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:38.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.757+0000 7f81ab9a5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81a410ff00 0x7f81a41a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.757+0000 7f81ab9a5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81a4115870 con 0x7f81a4075ff0 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.757+0000 7f81ab9a5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41156f0 con 0x7f81a40756d0 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.757+0000 7f81ab9a5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f81a41159f0 con 0x7f81a410ff00 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a8f40700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a4075ff0 0x7f81a419c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a8f40700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a4075ff0 0x7f81a419c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56034/0 (socket says 172.21.15.59:56034) 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a8f40700 1 -- 172.21.15.59:0/683794356 learned_addr learned my addr 172.21.15.59:0/683794356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a9741700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a40756d0 0x7f81a419bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a9f42700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81a410ff00 0x7f81a41a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a8f40700 1 -- 172.21.15.59:0/683794356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81a410ff00 msgr2=0x7f81a41a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a8f40700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81a410ff00 0x7f81a41a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a8f40700 1 -- 172.21.15.59:0/683794356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a40756d0 msgr2=0x7f81a419bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a8f40700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a40756d0 0x7f81a419bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:38.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a8f40700 1 -- 172.21.15.59:0/683794356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81a41a0d80 con 0x7f81a4075ff0 2024-05-08T05:25:38.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81a8f40700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a4075ff0 0x7f81a419c300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f819400d980 tx=0x7f819400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:38.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f819a7fc700 1 -- 172.21.15.59:0/683794356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8194017070 con 0x7f81a4075ff0 2024-05-08T05:25:38.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f819a7fc700 1 -- 172.21.15.59:0/683794356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8194005020 con 0x7f81a4075ff0 2024-05-08T05:25:38.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81a41a1070 con 0x7f81a4075ff0 2024-05-08T05:25:38.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f819a7fc700 1 -- 172.21.15.59:0/683794356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f819400fa10 con 0x7f81a4075ff0 2024-05-08T05:25:38.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.758+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81a41a88f0 con 0x7f81a4075ff0 2024-05-08T05:25:38.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.759+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81a406a190 con 0x7f81a4075ff0 2024-05-08T05:25:38.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.761+0000 7f819a7fc700 1 -- 172.21.15.59:0/683794356 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f8194004a30 con 0x7f81a4075ff0 2024-05-08T05:25:38.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.762+0000 7f819a7fc700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81900b2900 0x7f81900b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:38.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.762+0000 7f81a9741700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81900b2900 0x7f81900b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:38.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.762+0000 7f819a7fc700 1 -- 172.21.15.59:0/683794356 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8770+0+0 (secure 0 0 0) 0x7f81940e2600 con 0x7f81a4075ff0 2024-05-08T05:25:38.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.762+0000 7f81a9741700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81900b2900 0x7f81900b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81a0005950 tx=0x7f81a000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:38.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.765+0000 7f819a7fc700 1 -- 172.21.15.59:0/683794356 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8194020020 con 0x7f81a4075ff0 2024-05-08T05:25:38.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:38.978+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7f81a419d0d0 con 0x7f81a4075ff0 2024-05-08T05:25:39.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.066+0000 7f819a7fc700 1 -- 172.21.15.59:0/683794356 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v85) ==== 169+0+0 (secure 0 0 0) 0x7f81940b2030 con 0x7f81a4075ff0 2024-05-08T05:25:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81900b2900 msgr2=0x7f81900b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81900b2900 0x7f81900b4dc0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f81a0005950 tx=0x7f81a000b410 comp rx=0 tx=0).stop 2024-05-08T05:25:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a4075ff0 msgr2=0x7f81a419c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:39.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a4075ff0 0x7f81a419c300 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f819400d980 tx=0x7f819400dc90 comp rx=0 tx=0).stop 2024-05-08T05:25:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 shutdown_connections 2024-05-08T05:25:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81a410ff00 0x7f81a41a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81900b2900 0x7f81900b4dc0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81a40756d0 0x7f81a419bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 --2- 172.21.15.59:0/683794356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81a4075ff0 0x7f81a419c300 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.068+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 >> 172.21.15.59:0/683794356 conn(0x7f81a407a580 msgr2=0x7f81a4101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.069+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 shutdown_connections 2024-05-08T05:25:39.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:39.069+0000 7f81ab9a5700 1 -- 172.21.15.59:0/683794356 wait complete. 2024-05-08T05:25:39.071 INFO:teuthology.orchestra.run.smithi059.stderr:set pool 4 allow_ec_overwrites to true 2024-05-08T05:25:39.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:39 smithi059 ceph-mon[30304]: pgmap v300: 129 pgs: 6 creating+peering, 25 unknown, 98 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-05-08T05:25:39.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:39 smithi059 ceph-mon[30304]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T05:25:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:39 smithi059 ceph-mon[30304]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:25:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:39 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/683794356' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T05:25:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:39 smithi143 ceph-mon[34337]: pgmap v300: 129 pgs: 6 creating+peering, 25 unknown, 98 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-05-08T05:25:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:39 smithi143 ceph-mon[34337]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T05:25:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:39 smithi143 ceph-mon[34337]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:25:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:39 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/683794356' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T05:25:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:39 smithi096 ceph-mon[34253]: pgmap v300: 129 pgs: 6 creating+peering, 25 unknown, 98 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s wr, 3 op/s 2024-05-08T05:25:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:39 smithi096 ceph-mon[34253]: osdmap e84: 12 total, 12 up, 12 in 2024-05-08T05:25:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:39 smithi096 ceph-mon[34253]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-05-08T05:25:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:39 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/683794356' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-05-08T05:25:39.878 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs add_data_pool cephfs cephfs_data_ec 2024-05-08T05:25:40.139 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:40.261 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:40 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/683794356' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T05:25:40.262 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:40 smithi059 ceph-mon[30304]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T05:25:40.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:40 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/683794356' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T05:25:40.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:40 smithi143 ceph-mon[34337]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T05:25:40.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:40 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/683794356' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-05-08T05:25:40.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:40 smithi096 ceph-mon[34253]: osdmap e85: 12 total, 12 up, 12 in 2024-05-08T05:25:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:41 smithi059 ceph-mon[30304]: pgmap v303: 129 pgs: 6 creating+peering, 19 unknown, 104 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:41 smithi143 ceph-mon[34337]: pgmap v303: 129 pgs: 6 creating+peering, 19 unknown, 104 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:41 smithi096 ceph-mon[34253]: pgmap v303: 129 pgs: 6 creating+peering, 19 unknown, 104 active+clean; 579 KiB data, 347 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:41.868 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.866+0000 7f6748de9700 1 -- 172.21.15.59:0/2176403913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6744109590 msgr2=0x7f6744109970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.866+0000 7f6748de9700 1 --2- 172.21.15.59:0/2176403913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6744109590 0x7f6744109970 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f67340099c0 tx=0x7f6734009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.866+0000 7f6748de9700 1 -- 172.21.15.59:0/2176403913 shutdown_connections 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.866+0000 7f6748de9700 1 --2- 172.21.15.59:0/2176403913 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674406c630 0x7f6744110360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.866+0000 7f6748de9700 1 --2- 172.21.15.59:0/2176403913 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6744109590 0x7f6744109970 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.866+0000 7f6748de9700 1 --2- 172.21.15.59:0/2176403913 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f674406bc70 0x7f674406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.866+0000 7f6748de9700 1 -- 172.21.15.59:0/2176403913 >> 172.21.15.59:0/2176403913 conn(0x7f6744079440 msgr2=0x7f6744079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.866+0000 7f6748de9700 1 -- 172.21.15.59:0/2176403913 shutdown_connections 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.866+0000 7f6748de9700 1 -- 172.21.15.59:0/2176403913 wait complete. 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6748de9700 1 Processor -- start 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6748de9700 1 -- start start 2024-05-08T05:25:41.869 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6748de9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674406bc70 0x7f6744102410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6748de9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674406c630 0x7f6744102950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6748de9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6744109590 0x7f6744102e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6748de9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6744113690 con 0x7f674406bc70 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6748de9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6744113510 con 0x7f674406c630 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6748de9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6744113810 con 0x7f6744109590 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f674259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674406bc70 0x7f6744102410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f674259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674406bc70 0x7f6744102410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56054/0 (socket says 172.21.15.59:56054) 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f674259c700 1 -- 172.21.15.59:0/2924025493 learned_addr learned my addr 172.21.15.59:0/2924025493 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6742d9d700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6744109590 0x7f6744102e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6741d9b700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674406c630 0x7f6744102950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f674259c700 1 -- 172.21.15.59:0/2924025493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6744109590 msgr2=0x7f6744102e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:41.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f674259c700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6744109590 0x7f6744102e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f674259c700 1 -- 172.21.15.59:0/2924025493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674406c630 msgr2=0x7f6744102950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f674259c700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674406c630 0x7f6744102950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f674259c700 1 -- 172.21.15.59:0/2924025493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67441a80e0 con 0x7f674406bc70 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f6741d9b700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674406c630 0x7f6744102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.867+0000 7f674259c700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674406bc70 0x7f6744102410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f673400b580 tx=0x7f67340047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.868+0000 7f67337fe700 1 -- 172.21.15.59:0/2924025493 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734027070 con 0x7f674406bc70 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.868+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67441a8370 con 0x7f674406bc70 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.868+0000 7f67337fe700 1 -- 172.21.15.59:0/2924025493 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6734003f20 con 0x7f674406bc70 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.868+0000 7f67337fe700 1 -- 172.21.15.59:0/2924025493 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6734022d70 con 0x7f674406bc70 2024-05-08T05:25:41.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.868+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67441a87c0 con 0x7f674406bc70 2024-05-08T05:25:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.869+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f674410ba20 con 0x7f674406bc70 2024-05-08T05:25:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.870+0000 7f67337fe700 1 -- 172.21.15.59:0/2924025493 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f6734022500 con 0x7f674406bc70 2024-05-08T05:25:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.871+0000 7f67337fe700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f672c0b2880 0x7f672c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.871+0000 7f6741d9b700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f672c0b2880 0x7f672c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.871+0000 7f67337fe700 1 -- 172.21.15.59:0/2924025493 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8770+0+0 (secure 0 0 0) 0x7f67340e3520 con 0x7f674406bc70 2024-05-08T05:25:41.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.871+0000 7f6741d9b700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f672c0b2880 0x7f672c0b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6744079a90 tx=0x7f673800b2c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:41.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:41.875+0000 7f67337fe700 1 -- 172.21.15.59:0/2924025493 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6734013070 con 0x7f674406bc70 2024-05-08T05:25:42.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:42.141+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f674406a190 con 0x7f674406bc70 2024-05-08T05:25:43.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.084+0000 7f67337fe700 1 -- 172.21.15.59:0/2924025493 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f67340ab0b0 con 0x7f674406bc70 2024-05-08T05:25:43.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f672c0b2880 msgr2=0x7f672c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f672c0b2880 0x7f672c0b4d40 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f6744079a90 tx=0x7f673800b2c0 comp rx=0 tx=0).stop 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674406bc70 msgr2=0x7f6744102410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674406bc70 0x7f6744102410 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f673400b580 tx=0x7f67340047b0 comp rx=0 tx=0).stop 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 shutdown_connections 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6744109590 0x7f6744102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f674406bc70 0x7f6744102410 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f672c0b2880 0x7f672c0b4d40 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 --2- 172.21.15.59:0/2924025493 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f674406c630 0x7f6744102950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 >> 172.21.15.59:0/2924025493 conn(0x7f6744079440 msgr2=0x7f6744100fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 shutdown_connections 2024-05-08T05:25:43.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:43.087+0000 7f6748de9700 1 -- 172.21.15.59:0/2924025493 wait complete. 2024-05-08T05:25:43.089 INFO:teuthology.orchestra.run.smithi059.stderr:added data pool 4 to fsmap 2024-05-08T05:25:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:43 smithi059 ceph-mon[30304]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:43 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2924025493' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T05:25:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:43 smithi143 ceph-mon[34337]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:43 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2924025493' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T05:25:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:43 smithi096 ceph-mon[34253]: pgmap v304: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:43.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:43 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2924025493' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-05-08T05:25:43.916 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:25:44.181 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:44.304 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:44 smithi059 ceph-mon[30304]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T05:25:44.304 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:44 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2924025493' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T05:25:44.304 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:44 smithi059 ceph-mon[30304]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:25:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:44 smithi143 ceph-mon[34337]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T05:25:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:44 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2924025493' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T05:25:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:44 smithi143 ceph-mon[34337]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:25:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:44 smithi096 ceph-mon[34253]: osdmap e86: 12 total, 12 up, 12 in 2024-05-08T05:25:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:44 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2924025493' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-05-08T05:25:44.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:44 smithi096 ceph-mon[34253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:25:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:45 smithi059 ceph-mon[30304]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:45 smithi059 ceph-mon[30304]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T05:25:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:45 smithi059 ceph-mon[30304]: Cluster is now healthy 2024-05-08T05:25:45.431 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:45 smithi096 ceph-mon[34253]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:45.431 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:45 smithi096 ceph-mon[34253]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T05:25:45.431 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:45 smithi096 ceph-mon[34253]: Cluster is now healthy 2024-05-08T05:25:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:45 smithi143 ceph-mon[34337]: pgmap v306: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:45 smithi143 ceph-mon[34337]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-05-08T05:25:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:45 smithi143 ceph-mon[34337]: Cluster is now healthy 2024-05-08T05:25:46.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.991+0000 7fd4923ea700 1 -- 172.21.15.59:0/1776724107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c103750 msgr2=0x7fd48c103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.991+0000 7fd4923ea700 1 --2- 172.21.15.59:0/1776724107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c103750 0x7fd48c103b30 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fd47c0099c0 tx=0x7fd47c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.991+0000 7fd4923ea700 1 -- 172.21.15.59:0/1776724107 shutdown_connections 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.991+0000 7fd4923ea700 1 --2- 172.21.15.59:0/1776724107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd48c108850 0x7fd48c1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.991+0000 7fd4923ea700 1 --2- 172.21.15.59:0/1776724107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c103750 0x7fd48c103b30 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.991+0000 7fd4923ea700 1 --2- 172.21.15.59:0/1776724107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd48c104100 0x7fd48c108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.991+0000 7fd4923ea700 1 -- 172.21.15.59:0/1776724107 >> 172.21.15.59:0/1776724107 conn(0x7fd48c07a600 msgr2=0x7fd48c07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.992+0000 7fd4923ea700 1 -- 172.21.15.59:0/1776724107 shutdown_connections 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.992+0000 7fd4923ea700 1 -- 172.21.15.59:0/1776724107 wait complete. 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.992+0000 7fd4923ea700 1 Processor -- start 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.992+0000 7fd4923ea700 1 -- start start 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.992+0000 7fd4923ea700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd48c103750 0x7fd48c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.992+0000 7fd4923ea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd48c104100 0x7fd48c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:46.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd4923ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c108850 0x7fd48c199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd4923ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c113640 con 0x7fd48c108850 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd4923ea700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1134c0 con 0x7fd48c104100 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd4923ea700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd48c1137c0 con 0x7fd48c103750 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd490987700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c108850 0x7fd48c199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd490987700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c108850 0x7fd48c199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56078/0 (socket says 172.21.15.59:56078) 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd490987700 1 -- 172.21.15.59:0/4151923250 learned_addr learned my addr 172.21.15.59:0/4151923250 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd483fff700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd48c104100 0x7fd48c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd48bfff700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd48c103750 0x7fd48c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd490987700 1 -- 172.21.15.59:0/4151923250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd48c103750 msgr2=0x7fd48c19f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd490987700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd48c103750 0x7fd48c19f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd490987700 1 -- 172.21.15.59:0/4151923250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd48c104100 msgr2=0x7fd48c19fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd490987700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd48c104100 0x7fd48c19fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd490987700 1 -- 172.21.15.59:0/4151923250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd48c19a0e0 con 0x7fd48c108850 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd490987700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c108850 0x7fd48c199980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd484007ee0 tx=0x7fd48400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:46.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd489ffb700 1 -- 172.21.15.59:0/4151923250 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd48400ddd0 con 0x7fd48c108850 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd489ffb700 1 -- 172.21.15.59:0/4151923250 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd484004020 con 0x7fd48c108850 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd489ffb700 1 -- 172.21.15.59:0/4151923250 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd484015660 con 0x7fd48c108850 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd48c19a3d0 con 0x7fd48c108850 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.993+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd48c1a6690 con 0x7fd48c108850 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.994+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd48c10ba90 con 0x7fd48c108850 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.996+0000 7fd489ffb700 1 -- 172.21.15.59:0/4151923250 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd4840041d0 con 0x7fd48c108850 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.996+0000 7fd489ffb700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd4780b2880 0x7fd4780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.997+0000 7fd48bfff700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd4780b2880 0x7fd4780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.997+0000 7fd489ffb700 1 -- 172.21.15.59:0/4151923250 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fd4840e1ee0 con 0x7fd48c108850 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:45.997+0000 7fd48bfff700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd4780b2880 0x7fd4780b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd47c000c00 tx=0x7fd47c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:46.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.001+0000 7fd489ffb700 1 -- 172.21.15.59:0/4151923250 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd484010070 con 0x7fd48c108850 2024-05-08T05:25:46.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.264+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd48c10b470 con 0x7fd48c108850 2024-05-08T05:25:46.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.265+0000 7fd489ffb700 1 -- 172.21.15.59:0/4151923250 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7fd4840a99c0 con 0x7fd48c108850 2024-05-08T05:25:46.266 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:25:46.266 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:25:42.142697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24473,"qdb_cluster":[24473]},"id":1}]} 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd4780b2880 msgr2=0x7fd4780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd4780b2880 0x7fd4780b4d40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fd47c000c00 tx=0x7fd47c005fb0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c108850 msgr2=0x7fd48c199980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c108850 0x7fd48c199980 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fd484007ee0 tx=0x7fd48400d3b0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 shutdown_connections 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd48c108850 0x7fd48c199980 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd48c103750 0x7fd48c19f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd4780b2880 0x7fd4780b4d40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 --2- 172.21.15.59:0/4151923250 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd48c104100 0x7fd48c19fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 >> 172.21.15.59:0/4151923250 conn(0x7fd48c07a600 msgr2=0x7fd48c1099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 shutdown_connections 2024-05-08T05:25:46.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:46.268+0000 7fd4923ea700 1 -- 172.21.15.59:0/4151923250 wait complete. 2024-05-08T05:25:46.270 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-05-08T05:25:46.979 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:25:47.234 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:47 smithi059 ceph-mon[30304]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:47 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/4151923250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:47 smithi143 ceph-mon[34337]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:47 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/4151923250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:47 smithi096 ceph-mon[34253]: pgmap v307: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:47 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/4151923250' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:48.870 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.866+0000 7f290432e700 1 -- 172.21.15.59:0/2004953770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc104100 msgr2=0x7f28fc108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.866+0000 7f290432e700 1 --2- 172.21.15.59:0/2004953770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc104100 0x7f28fc108120 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0099c0 tx=0x7f28ec009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.866+0000 7f290432e700 1 -- 172.21.15.59:0/2004953770 shutdown_connections 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.866+0000 7f290432e700 1 --2- 172.21.15.59:0/2004953770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28fc108850 0x7f28fc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.866+0000 7f290432e700 1 --2- 172.21.15.59:0/2004953770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28fc103750 0x7f28fc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.866+0000 7f290432e700 1 --2- 172.21.15.59:0/2004953770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc104100 0x7f28fc108120 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.866+0000 7f290432e700 1 -- 172.21.15.59:0/2004953770 >> 172.21.15.59:0/2004953770 conn(0x7f28fc07a600 msgr2=0x7f28fc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 -- 172.21.15.59:0/2004953770 shutdown_connections 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 -- 172.21.15.59:0/2004953770 wait complete. 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 Processor -- start 2024-05-08T05:25:48.871 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 -- start start 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28fc103750 0x7f28fc19f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28fc104100 0x7f28fc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc108850 0x7f28fc199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc113610 con 0x7f28fc108850 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc113490 con 0x7f28fc103750 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.867+0000 7f290432e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f28fc113790 con 0x7f28fc104100 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29028cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc108850 0x7f28fc199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29028cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc108850 0x7f28fc199890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57210/0 (socket says 172.21.15.59:57210) 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29028cb700 1 -- 172.21.15.59:0/70688113 learned_addr learned my addr 172.21.15.59:0/70688113 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:48.872 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29020ca700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28fc103750 0x7f28fc19f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29018c9700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28fc104100 0x7f28fc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29028cb700 1 -- 172.21.15.59:0/70688113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28fc104100 msgr2=0x7f28fc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29028cb700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28fc104100 0x7f28fc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29028cb700 1 -- 172.21.15.59:0/70688113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28fc103750 msgr2=0x7f28fc19f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29028cb700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28fc103750 0x7f28fc19f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29028cb700 1 -- 172.21.15.59:0/70688113 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28fc199ff0 con 0x7f28fc108850 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f29028cb700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc108850 0x7f28fc199890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f28f4007d70 tx=0x7f28f400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f28f37fe700 1 -- 172.21.15.59:0/70688113 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f4016020 con 0x7f28fc108850 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28fc19a2e0 con 0x7f28fc108850 2024-05-08T05:25:48.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f28f37fe700 1 -- 172.21.15.59:0/70688113 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28f400ac90 con 0x7f28fc108850 2024-05-08T05:25:48.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28fc1a65a0 con 0x7f28fc108850 2024-05-08T05:25:48.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.868+0000 7f28f37fe700 1 -- 172.21.15.59:0/70688113 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28f40115e0 con 0x7f28fc108850 2024-05-08T05:25:48.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.870+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28fc06a190 con 0x7f28fc108850 2024-05-08T05:25:48.874 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.871+0000 7f28f37fe700 1 -- 172.21.15.59:0/70688113 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f28f4019020 con 0x7f28fc108850 2024-05-08T05:25:48.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.872+0000 7f28f37fe700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28e80b2880 0x7f28e80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:48.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.872+0000 7f29020ca700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28e80b2880 0x7f28e80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:48.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.872+0000 7f28f37fe700 1 -- 172.21.15.59:0/70688113 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f28f40e28b0 con 0x7f28fc108850 2024-05-08T05:25:48.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.872+0000 7f29020ca700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28e80b2880 0x7f28e80b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f28f8005950 tx=0x7f28f800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:48.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:48.875+0000 7f28f37fe700 1 -- 172.21.15.59:0/70688113 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f28f4020020 con 0x7f28fc108850 2024-05-08T05:25:49.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.090+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f28fc19adf0 con 0x7f28fc108850 2024-05-08T05:25:49.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.091+0000 7f28f37fe700 1 -- 172.21.15.59:0/70688113 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24075 (secure 0 0 0) 0x7f28fc19adf0 con 0x7f28fc108850 2024-05-08T05:25:49.092 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:25:49.092 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:25:42.142557+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:25:49.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28e80b2880 msgr2=0x7f28e80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:49.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28e80b2880 0x7f28e80b4d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f28f8005950 tx=0x7f28f800a300 comp rx=0 tx=0).stop 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc108850 msgr2=0x7f28fc199890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc108850 0x7f28fc199890 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f28f4007d70 tx=0x7f28f400a340 comp rx=0 tx=0).stop 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 shutdown_connections 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f28fc108850 0x7f28fc199890 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28e80b2880 0x7f28e80b4d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f28fc103750 0x7f28fc19f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 --2- 172.21.15.59:0/70688113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f28fc104100 0x7f28fc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 >> 172.21.15.59:0/70688113 conn(0x7f28fc07a600 msgr2=0x7f28fc101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 shutdown_connections 2024-05-08T05:25:49.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:49.094+0000 7f290432e700 1 -- 172.21.15.59:0/70688113 wait complete. 2024-05-08T05:25:49.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:49 smithi059 ceph-mon[30304]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:49 smithi143 ceph-mon[34337]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:49 smithi096 ceph-mon[34253]: pgmap v308: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:49.849 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:25:50.105 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:50.368 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:50 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/70688113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:25:50.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:50 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/70688113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:25:50.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:50 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/70688113' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:25:51.267 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:51 smithi059 ceph-mon[30304]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:51 smithi143 ceph-mon[34337]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:51 smithi096 ceph-mon[34253]: pgmap v309: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:51.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.811+0000 7f81e6ad6700 1 -- 172.21.15.59:0/2545438768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0108850 msgr2=0x7f81e01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:51.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.811+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/2545438768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0108850 0x7f81e01103d0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f81d800b0d0 tx=0x7f81d800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:25:51.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.811+0000 7f81e6ad6700 1 -- 172.21.15.59:0/2545438768 shutdown_connections 2024-05-08T05:25:51.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.811+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/2545438768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0108850 0x7f81e01103d0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:51.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.811+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/2545438768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e0103750 0x7f81e0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:51.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.811+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/2545438768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81e0104100 0x7f81e0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.811+0000 7f81e6ad6700 1 -- 172.21.15.59:0/2545438768 >> 172.21.15.59:0/2545438768 conn(0x7f81e007a600 msgr2=0x7f81e007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.811+0000 7f81e6ad6700 1 -- 172.21.15.59:0/2545438768 shutdown_connections 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.811+0000 7f81e6ad6700 1 -- 172.21.15.59:0/2545438768 wait complete. 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e6ad6700 1 Processor -- start 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e6ad6700 1 -- start start 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e6ad6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0103750 0x7f81e00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e6ad6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e0104100 0x7f81e0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e6ad6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81e0108850 0x7f81e0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e6ad6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f81e0113670 con 0x7f81e0103750 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e6ad6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01134f0 con 0x7f81e0108850 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e6ad6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f81e01137f0 con 0x7f81e0104100 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e4872700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0103750 0x7f81e00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e4872700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0103750 0x7f81e00757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57226/0 (socket says 172.21.15.59:57226) 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.812+0000 7f81e4872700 1 -- 172.21.15.59:0/3959732058 learned_addr learned my addr 172.21.15.59:0/3959732058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:51.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81e5073700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81e0108850 0x7f81e0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81dffff700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e0104100 0x7f81e0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81e4872700 1 -- 172.21.15.59:0/3959732058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e0104100 msgr2=0x7f81e0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81e4872700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e0104100 0x7f81e0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81e4872700 1 -- 172.21.15.59:0/3959732058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81e0108850 msgr2=0x7f81e0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81e4872700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81e0108850 0x7f81e0076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81e4872700 1 -- 172.21.15.59:0/3959732058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81e01a7fe0 con 0x7f81e0103750 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81e4872700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0103750 0x7f81e00757a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f81c800d980 tx=0x7f81c800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81ddffb700 1 -- 172.21.15.59:0/3959732058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8016070 con 0x7f81e0103750 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81e01a82d0 con 0x7f81e0103750 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81ddffb700 1 -- 172.21.15.59:0/3959732058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81c8004830 con 0x7f81e0103750 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81ddffb700 1 -- 172.21.15.59:0/3959732058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81c8005540 con 0x7f81e0103750 2024-05-08T05:25:51.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.813+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81e01a8860 con 0x7f81e0103750 2024-05-08T05:25:51.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.814+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81e006a190 con 0x7f81e0103750 2024-05-08T05:25:51.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.816+0000 7f81ddffb700 1 -- 172.21.15.59:0/3959732058 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f81c8004220 con 0x7f81e0103750 2024-05-08T05:25:51.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.816+0000 7f81ddffb700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81cc0b2880 0x7f81cc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:51.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.817+0000 7f81dffff700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81cc0b2880 0x7f81cc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:51.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.817+0000 7f81ddffb700 1 -- 172.21.15.59:0/3959732058 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f81c80e2210 con 0x7f81e0103750 2024-05-08T05:25:51.817 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.817+0000 7f81dffff700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81cc0b2880 0x7f81cc0b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f81e0103030 tx=0x7f81d000b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:51.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:51.821+0000 7f81ddffb700 1 -- 172.21.15.59:0/3959732058 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f81c801f020 con 0x7f81e0103750 2024-05-08T05:25:52.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.040+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f81e002e6d0 con 0x7f81e0103750 2024-05-08T05:25:52.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.042+0000 7f81ddffb700 1 -- 172.21.15.59:0/3959732058 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24075 (secure 0 0 0) 0x7f81c80a9cf0 con 0x7f81e0103750 2024-05-08T05:25:52.043 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:25:52.043 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:25:42.142557+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:25:52.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81cc0b2880 msgr2=0x7f81cc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81cc0b2880 0x7f81cc0b4d40 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f81e0103030 tx=0x7f81d000b410 comp rx=0 tx=0).stop 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0103750 msgr2=0x7f81e00757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0103750 0x7f81e00757a0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f81c800d980 tx=0x7f81c800dc90 comp rx=0 tx=0).stop 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 shutdown_connections 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f81cc0b2880 0x7f81cc0b4d40 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f81e0108850 0x7f81e0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f81e0103750 0x7f81e00757a0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 --2- 172.21.15.59:0/3959732058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f81e0104100 0x7f81e0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 >> 172.21.15.59:0/3959732058 conn(0x7f81e007a600 msgr2=0x7f81e01016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 shutdown_connections 2024-05-08T05:25:52.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:52.044+0000 7f81e6ad6700 1 -- 172.21.15.59:0/3959732058 wait complete. 2024-05-08T05:25:52.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:52 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3959732058' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:25:52.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:52 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3959732058' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:25:52.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:52 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3959732058' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:25:52.826 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:25:53.089 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:53.438 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:53 smithi059 ceph-mon[30304]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:53 smithi143 ceph-mon[34337]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:53 smithi096 ceph-mon[34253]: pgmap v310: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:54.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.895+0000 7fb9927c2700 1 -- 172.21.15.59:0/1184421171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c108410 msgr2=0x7fb98c1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:54.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.895+0000 7fb9927c2700 1 --2- 172.21.15.59:0/1184421171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c108410 0x7fb98c1087f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009990 tx=0x7fb97c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:25:54.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.895+0000 7fb9927c2700 1 -- 172.21.15.59:0/1184421171 shutdown_connections 2024-05-08T05:25:54.898 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.895+0000 7fb9927c2700 1 --2- 172.21.15.59:0/1184421171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb98c102d60 0x7fb98c10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.895+0000 7fb9927c2700 1 --2- 172.21.15.59:0/1184421171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c108410 0x7fb98c1087f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.895+0000 7fb9927c2700 1 --2- 172.21.15.59:0/1184421171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb98c1023a0 0x7fb98c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.895+0000 7fb9927c2700 1 -- 172.21.15.59:0/1184421171 >> 172.21.15.59:0/1184421171 conn(0x7fb98c079350 msgr2=0x7fb98c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.895+0000 7fb9927c2700 1 -- 172.21.15.59:0/1184421171 shutdown_connections 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.895+0000 7fb9927c2700 1 -- 172.21.15.59:0/1184421171 wait complete. 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.896+0000 7fb9927c2700 1 Processor -- start 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.896+0000 7fb9927c2700 1 -- start start 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb9927c2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb98c1023a0 0x7fb98c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb9927c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c102d60 0x7fb98c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb9927c2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb98c108410 0x7fb98c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb9927c2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c112580 con 0x7fb98c102d60 2024-05-08T05:25:54.899 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb9927c2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c112400 con 0x7fb98c1023a0 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb9927c2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb98c112700 con 0x7fb98c108410 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb983fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c102d60 0x7fb98c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb983fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c102d60 0x7fb98c1045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57248/0 (socket says 172.21.15.59:57248) 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb983fff700 1 -- 172.21.15.59:0/3661898474 learned_addr learned my addr 172.21.15.59:0/3661898474 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb990d5f700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb98c108410 0x7fb98c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb98bfff700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb98c1023a0 0x7fb98c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb983fff700 1 -- 172.21.15.59:0/3661898474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb98c108410 msgr2=0x7fb98c104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb983fff700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb98c108410 0x7fb98c104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb983fff700 1 -- 172.21.15.59:0/3661898474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb98c1023a0 msgr2=0x7fb98c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb983fff700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb98c1023a0 0x7fb98c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb983fff700 1 -- 172.21.15.59:0/3661898474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb98c1a8080 con 0x7fb98c102d60 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb98bfff700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb98c1023a0 0x7fb98c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.897+0000 7fb983fff700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c102d60 0x7fb98c1045c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c920 tx=0x7fb97400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:54.900 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.898+0000 7fb989ffb700 1 -- 172.21.15.59:0/3661898474 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974007b00 con 0x7fb98c102d60 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.898+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb98c1a8370 con 0x7fb98c102d60 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.898+0000 7fb989ffb700 1 -- 172.21.15.59:0/3661898474 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb974004ce0 con 0x7fb98c102d60 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.898+0000 7fb989ffb700 1 -- 172.21.15.59:0/3661898474 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb974005550 con 0x7fb98c102d60 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.898+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb98c1a88b0 con 0x7fb98c102d60 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.899+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb98c06a190 con 0x7fb98c102d60 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.900+0000 7fb989ffb700 1 -- 172.21.15.59:0/3661898474 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb974007ca0 con 0x7fb98c102d60 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.901+0000 7fb989ffb700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb9780b2880 0x7fb9780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.901+0000 7fb98bfff700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb9780b2880 0x7fb9780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.901+0000 7fb989ffb700 1 -- 172.21.15.59:0/3661898474 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fb9740e2360 con 0x7fb98c102d60 2024-05-08T05:25:54.902 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.901+0000 7fb98bfff700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb9780b2880 0x7fb9780b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009750 tx=0x7fb97c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:54.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:54.905+0000 7fb989ffb700 1 -- 172.21.15.59:0/3661898474 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb974014030 con 0x7fb98c102d60 2024-05-08T05:25:55.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.167+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb98c1a8b70 con 0x7fb98c102d60 2024-05-08T05:25:55.167 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:55 smithi059 ceph-mon[30304]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:55.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.168+0000 7fb989ffb700 1 -- 172.21.15.59:0/3661898474 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11794 (secure 0 0 0) 0x7fb9740a9e40 con 0x7fb98c102d60 2024-05-08T05:25:55.168 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:25:55.168 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14565,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14571,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24385,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24401,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24415,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24419,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24421,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24433,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24443,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:25:42.142697+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24473},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24473,"qdb_cluster":[24473]},"id":1}]} 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb9780b2880 msgr2=0x7fb9780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb9780b2880 0x7fb9780b4d40 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb97c009750 tx=0x7fb97c005fd0 comp rx=0 tx=0).stop 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c102d60 msgr2=0x7fb98c1045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c102d60 0x7fb98c1045c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fb97400c920 tx=0x7fb97400cc30 comp rx=0 tx=0).stop 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 shutdown_connections 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb98c108410 0x7fb98c104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb9780b2880 0x7fb9780b4d40 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb98c1023a0 0x7fb98c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 --2- 172.21.15.59:0/3661898474 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb98c102d60 0x7fb98c1045c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.170+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 >> 172.21.15.59:0/3661898474 conn(0x7fb98c079350 msgr2=0x7fb98c1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.171+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 shutdown_connections 2024-05-08T05:25:55.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:55.171+0000 7fb9927c2700 1 -- 172.21.15.59:0/3661898474 wait complete. 2024-05-08T05:25:55.172 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 17 2024-05-08T05:25:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:55 smithi143 ceph-mon[34337]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:55 smithi096 ceph-mon[34253]: pgmap v311: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:55.839 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:25:56.098 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:56.447 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:56 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3661898474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:56.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:56 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3661898474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:56.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:56 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3661898474' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:25:57.195 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:57 smithi059 ceph-mon[30304]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:57 smithi143 ceph-mon[34337]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:57 smithi096 ceph-mon[34253]: pgmap v312: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:57.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.735+0000 7f87eb723700 1 -- 172.21.15.59:0/88769369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4104100 msgr2=0x7f87e4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.735+0000 7f87eb723700 1 --2- 172.21.15.59:0/88769369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4104100 0x7f87e4108120 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f87d80099c0 tx=0x7f87d8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.736+0000 7f87eb723700 1 -- 172.21.15.59:0/88769369 shutdown_connections 2024-05-08T05:25:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.736+0000 7f87eb723700 1 --2- 172.21.15.59:0/88769369 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e4108850 0x7f87e41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.736+0000 7f87eb723700 1 --2- 172.21.15.59:0/88769369 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87e4103750 0x7f87e4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.736+0000 7f87eb723700 1 --2- 172.21.15.59:0/88769369 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4104100 0x7f87e4108120 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.736+0000 7f87eb723700 1 -- 172.21.15.59:0/88769369 >> 172.21.15.59:0/88769369 conn(0x7f87e407a600 msgr2=0x7f87e407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.736+0000 7f87eb723700 1 -- 172.21.15.59:0/88769369 shutdown_connections 2024-05-08T05:25:57.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.736+0000 7f87eb723700 1 -- 172.21.15.59:0/88769369 wait complete. 2024-05-08T05:25:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.736+0000 7f87eb723700 1 Processor -- start 2024-05-08T05:25:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.736+0000 7f87eb723700 1 -- start start 2024-05-08T05:25:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87eb723700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e4103750 0x7f87e419f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:57.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87eb723700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87e4104100 0x7f87e419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87eb723700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4108850 0x7f87e41999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87eb723700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4113690 con 0x7f87e4108850 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87eb723700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4113510 con 0x7f87e4103750 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87eb723700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f87e4113810 con 0x7f87e4104100 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e9cc0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4108850 0x7f87e41999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e9cc0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4108850 0x7f87e41999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:57274/0 (socket says 172.21.15.59:57274) 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e9cc0700 1 -- 172.21.15.59:0/232447013 learned_addr learned my addr 172.21.15.59:0/232447013 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e94bf700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e4103750 0x7f87e419f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e8cbe700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87e4104100 0x7f87e419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e9cc0700 1 -- 172.21.15.59:0/232447013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87e4104100 msgr2=0x7f87e419fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e9cc0700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87e4104100 0x7f87e419fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e9cc0700 1 -- 172.21.15.59:0/232447013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e4103750 msgr2=0x7f87e419f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e9cc0700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e4103750 0x7f87e419f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e9cc0700 1 -- 172.21.15.59:0/232447013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87e419a110 con 0x7f87e4108850 2024-05-08T05:25:57.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.737+0000 7f87e9cc0700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4108850 0x7f87e41999b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87dc007d70 tx=0x7f87dc00a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:57.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.738+0000 7f87d67fc700 1 -- 172.21.15.59:0/232447013 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc016020 con 0x7f87e4108850 2024-05-08T05:25:57.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.738+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87e419a400 con 0x7f87e4108850 2024-05-08T05:25:57.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.738+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87e41a6670 con 0x7f87e4108850 2024-05-08T05:25:57.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.738+0000 7f87d67fc700 1 -- 172.21.15.59:0/232447013 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87dc00ac90 con 0x7f87e4108850 2024-05-08T05:25:57.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.738+0000 7f87d67fc700 1 -- 172.21.15.59:0/232447013 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f87dc0115e0 con 0x7f87e4108850 2024-05-08T05:25:57.741 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.739+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f87e406a190 con 0x7f87e4108850 2024-05-08T05:25:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.740+0000 7f87d67fc700 1 -- 172.21.15.59:0/232447013 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f87dc019020 con 0x7f87e4108850 2024-05-08T05:25:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.741+0000 7f87d67fc700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f87d00b2900 0x7f87d00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:25:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.741+0000 7f87e94bf700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f87d00b2900 0x7f87d00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:25:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.741+0000 7f87d67fc700 1 -- 172.21.15.59:0/232447013 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f87dc0e2350 con 0x7f87e4108850 2024-05-08T05:25:57.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.741+0000 7f87e94bf700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f87d00b2900 0x7f87d00b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87e4108260 tx=0x7f87e0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:25:57.746 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.746+0000 7f87d67fc700 1 -- 172.21.15.59:0/232447013 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f87dc020020 con 0x7f87e4108850 2024-05-08T05:25:57.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.967+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f87e402e510 con 0x7f87e4108850 2024-05-08T05:25:57.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.968+0000 7f87d67fc700 1 -- 172.21.15.59:0/232447013 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v86) ==== 74+0+24075 (secure 0 0 0) 0x7f87dc0a9e30 con 0x7f87e4108850 2024-05-08T05:25:57.969 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:25:57.969 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":86,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:25:42.142557+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:25:57.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f87d00b2900 msgr2=0x7f87d00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:57.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f87d00b2900 0x7f87d00b4dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f87e4108260 tx=0x7f87e0009190 comp rx=0 tx=0).stop 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4108850 msgr2=0x7f87e41999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4108850 0x7f87e41999b0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f87dc007d70 tx=0x7f87dc00a340 comp rx=0 tx=0).stop 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 shutdown_connections 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f87e4108850 0x7f87e41999b0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f87d00b2900 0x7f87d00b4dc0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f87e4103750 0x7f87e419f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 --2- 172.21.15.59:0/232447013 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f87e4104100 0x7f87e419fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 >> 172.21.15.59:0/232447013 conn(0x7f87e407a600 msgr2=0x7f87e41099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 shutdown_connections 2024-05-08T05:25:57.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:25:57.971+0000 7f87eb723700 1 -- 172.21.15.59:0/232447013 wait complete. 2024-05-08T05:25:58.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:58 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/232447013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:25:58.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:58 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/232447013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:25:58.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:58 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/232447013' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:25:58.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-05-08T05:25:27.601131+0000', 'modified': '2024-05-08T05:25:42.142697+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24473}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6839/126053836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 126053836}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 126053836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24473, 'qdb_cluster': [24473]} 2024-05-08T05:25:58.707 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-05-08T05:25:58.707 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-08T05:25:58.968 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:25:59.449 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:25:59 smithi059 ceph-mon[30304]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:25:59 smithi143 ceph-mon[34337]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:25:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:25:59 smithi096 ceph-mon[34253]: pgmap v313: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:00.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.599+0000 7f84c3fcc700 1 -- 172.21.15.59:0/485514209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc104130 msgr2=0x7f84bc108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:00.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.599+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/485514209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc104130 0x7f84bc108140 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84b00099c0 tx=0x7f84b0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.599+0000 7f84c3fcc700 1 -- 172.21.15.59:0/485514209 shutdown_connections 2024-05-08T05:26:00.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.599+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/485514209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc108870 0x7f84bc1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.599+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/485514209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc103780 0x7f84bc103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.599+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/485514209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc104130 0x7f84bc108140 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.599+0000 7f84c3fcc700 1 -- 172.21.15.59:0/485514209 >> 172.21.15.59:0/485514209 conn(0x7f84bc07a610 msgr2=0x7f84bc07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.599+0000 7f84c3fcc700 1 -- 172.21.15.59:0/485514209 shutdown_connections 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.599+0000 7f84c3fcc700 1 -- 172.21.15.59:0/485514209 wait complete. 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c3fcc700 1 Processor -- start 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c3fcc700 1 -- start start 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c3fcc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc103780 0x7f84bc19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c3fcc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc104130 0x7f84bc19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c3fcc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc108870 0x7f84bc1999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c3fcc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc1136a0 con 0x7f84bc108870 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c3fcc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc113520 con 0x7f84bc103780 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c3fcc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f84bc113820 con 0x7f84bc104130 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c2569700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc108870 0x7f84bc1999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c2569700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc108870 0x7f84bc1999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48604/0 (socket says 172.21.15.59:48604) 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c2569700 1 -- 172.21.15.59:0/2169072643 learned_addr learned my addr 172.21.15.59:0/2169072643 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:00.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c1567700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc104130 0x7f84bc19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.600+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc103780 0x7f84bc19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84c2569700 1 -- 172.21.15.59:0/2169072643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc104130 msgr2=0x7f84bc19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84c2569700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc104130 0x7f84bc19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84c2569700 1 -- 172.21.15.59:0/2169072643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc103780 msgr2=0x7f84bc19f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84c2569700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc103780 0x7f84bc19f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84c2569700 1 -- 172.21.15.59:0/2169072643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84bc19a120 con 0x7f84bc108870 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc103780 0x7f84bc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84c2569700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc108870 0x7f84bc1999c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f84b400d980 tx=0x7f84b400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84baffd700 1 -- 172.21.15.59:0/2169072643 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4017070 con 0x7f84bc108870 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84baffd700 1 -- 172.21.15.59:0/2169072643 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b4005020 con 0x7f84bc108870 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84c3fcc700 1 -- 172.21.15.59:0/2169072643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84bc19a410 con 0x7f84bc108870 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84baffd700 1 -- 172.21.15.59:0/2169072643 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40099b0 con 0x7f84bc108870 2024-05-08T05:26:00.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.601+0000 7f84c3fcc700 1 -- 172.21.15.59:0/2169072643 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84bc1a66d0 con 0x7f84bc108870 2024-05-08T05:26:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.603+0000 7f84baffd700 1 -- 172.21.15.59:0/2169072643 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f84b4004a30 con 0x7f84bc108870 2024-05-08T05:26:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.604+0000 7f84baffd700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84a40b2880 0x7f84a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.605+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84a40b2880 0x7f84a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.605+0000 7f84baffd700 1 -- 172.21.15.59:0/2169072643 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f84b40e22a0 con 0x7f84bc108870 2024-05-08T05:26:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.605+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84a40b2880 0x7f84a40b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f84ac007fd0 tx=0x7f84ac007830 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.605+0000 7f84ce85fb80 1 Processor -- start 2024-05-08T05:26:00.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.605+0000 7f84ce85fb80 1 -- start start 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07ac6480 0x556e07ac6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aca6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07acac30 0x556e07acb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556e07a18810 con 0x556e07ac6480 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556e079b7770 con 0x556e07acac30 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556e07a147d0 con 0x556e07ac6da0 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1d68700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07ac6480 0x556e07ac6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1d68700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07ac6480 0x556e07ac6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48610/0 (socket says 172.21.15.59:48610) 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1d68700 1 -- 172.21.15.59:0/3054764008 learned_addr learned my addr 172.21.15.59:0/3054764008 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1567700 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aca6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c2569700 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07acac30 0x556e07acb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1d68700 1 -- 172.21.15.59:0/3054764008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 msgr2=0x556e07aca6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1d68700 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aca6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1d68700 1 -- 172.21.15.59:0/3054764008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07acac30 msgr2=0x556e07acb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:26:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1d68700 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07acac30 0x556e07acb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1d68700 1 -- 172.21.15.59:0/3054764008 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e07965880 con 0x556e07ac6480 2024-05-08T05:26:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c2569700 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07acac30 0x556e07acb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:26:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1567700 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:26:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84c1d68700 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07ac6480 0x556e07ac6860 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f84ac018820 tx=0x7f84ac011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9201e60dad47c9b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84a3fff700 1 -- 172.21.15.59:0/3054764008 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac026070 con 0x556e07ac6480 2024-05-08T05:26:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84a3fff700 1 -- 172.21.15.59:0/3054764008 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ac0180d0 con 0x556e07ac6480 2024-05-08T05:26:00.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.606+0000 7f84a3fff700 1 -- 172.21.15.59:0/3054764008 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84ac016610 con 0x556e07ac6480 2024-05-08T05:26:00.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.607+0000 7f84ce85fb80 1 -- 172.21.15.59:0/3054764008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07ac6480 msgr2=0x556e07ac6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:00.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.607+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07ac6480 0x556e07ac6860 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f84ac018820 tx=0x7f84ac011000 comp rx=0 tx=0).stop 2024-05-08T05:26:00.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.607+0000 7f84ce85fb80 1 -- 172.21.15.59:0/3054764008 shutdown_connections 2024-05-08T05:26:00.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.607+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07acac30 0x556e07acb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.607+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07ac6480 0x556e07ac6860 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.607+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/3054764008 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aca6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.607+0000 7f84ce85fb80 1 -- 172.21.15.59:0/3054764008 >> 172.21.15.59:0/3054764008 conn(0x556e079f20b0 msgr2=0x556e07ac0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.607+0000 7f84ce85fb80 1 -- 172.21.15.59:0/3054764008 shutdown_connections 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.607+0000 7f84ce85fb80 1 -- 172.21.15.59:0/3054764008 wait complete. 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84ce85fb80 1 -- start start 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84ce85fb80 1 Processor -- start 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6480 0x556e07ad20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6da0 0x556e07acf470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07acf9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556e07ad2e40 con 0x556e07acac30 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556e0797ebe0 con 0x556e07ac6da0 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556e07ad2fc0 con 0x556e07ac6480 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84ce85fb80 10 client.0 _ll_register_callbacks cb 0x556e079b08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84c2569700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07acf9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84c2569700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07acf9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48614/0 (socket says 172.21.15.59:48614) 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.608+0000 7f84c2569700 1 -- 172.21.15.59:0/2111879269 learned_addr learned my addr 172.21.15.59:0/2111879269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6480 0x556e07ad20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84c1567700 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6da0 0x556e07acf470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84c2569700 1 -- 172.21.15.59:0/2111879269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6480 msgr2=0x556e07ad20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84c2569700 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6480 0x556e07ad20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84c2569700 1 -- 172.21.15.59:0/2111879269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6da0 msgr2=0x556e07acf470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84c2569700 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6da0 0x556e07acf470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84c2569700 1 -- 172.21.15.59:0/2111879269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e079b4640 con 0x556e07acac30 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84c1567700 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6da0 0x556e07acf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84c2569700 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07acf9b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f84b40a9380 tx=0x7f84b40a9e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac8dbc57e85c37a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84a17fa700 1 -- 172.21.15.59:0/2111879269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40b24e0 con 0x556e07acac30 2024-05-08T05:26:00.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2111879269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84ac0180d0 con 0x556e07acac30 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84a17fa700 1 -- 172.21.15.59:0/2111879269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f84b40a98e0 con 0x556e07acac30 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84a17fa700 1 -- 172.21.15.59:0/2111879269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f84b40a9ae0 con 0x556e07acac30 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84ce85fb80 10 client.14679 fetch_fsmap learned FSMap version 17 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2111879269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556e07add6a0 con 0x556e07acac30 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.609+0000 7f84a17fa700 1 -- 172.21.15.59:0/2111879269 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84ac0180d0 con 0x556e07acac30 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.610+0000 7f84a17fa700 1 -- 172.21.15.59:0/2111879269 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f84b400eb80 con 0x556e07acac30 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.610+0000 7f84ce85fb80 10 client.14679 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.610+0000 7f84ce85fb80 10 client.14679 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.610+0000 7f84ce85fb80 20 client.14679 populate_metadata read hostname 'smithi059' 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.610+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556e07adfba0 0x556e07ae1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.610+0000 7f84ce85fb80 4 client.14679 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.610+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2111879269 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e07a92600 con 0x556e07adfba0 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.610+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556e07adfba0 0x556e07ae1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.610+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556e07adfba0 0x556e07ae1010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ea960d4e98453db server_cookie=a144c66a4f6c4b40 in_seq=0 out_seq=0 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.611+0000 7f84a17fa700 10 client.14679 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.611+0000 7f84a17fa700 1 -- 172.21.15.59:0/2111879269 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556e07a92600 con 0x556e07adfba0 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.611+0000 7f84a17fa700 10 client.14679 handle_command_reply: tid=0 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.611+0000 7f84ce85fb80 1 client.14679 shutdown 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.611+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2111879269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 msgr2=0x556e07acf9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.611+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07acf9b0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f84b40a9380 tx=0x7f84b40a9e30 comp rx=0 tx=0).stop 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2111879269 shutdown_connections 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556e07adfba0 0x556e07ae1010 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07acf9b0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6480 0x556e07ad20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2111879269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6da0 0x556e07acf470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84a17fa700 0 client.14679 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2111879269 >> 172.21.15.59:0/2111879269 conn(0x556e079f20b0 msgr2=0x556e07abb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2111879269 shutdown_connections 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2111879269 wait complete. 2024-05-08T05:26:00.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.612+0000 7f84ce85fb80 20 client.14679 trim_cache size 0 max 16384 2024-05-08T05:26:00.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.633+0000 7f84ce85fb80 1 Processor -- start 2024-05-08T05:26:00.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.633+0000 7f84ce85fb80 1 -- start start 2024-05-08T05:26:00.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.633+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 0x556e07acfa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.633+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aea860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.633+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07abb550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.633+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556e07adf350 con 0x556e07acac30 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.633+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556e07adfa20 con 0x556e07ac6480 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.633+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556e07ae00f0 con 0x556e07ac6da0 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c2569700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07abb550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c2569700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07abb550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48616/0 (socket says 172.21.15.59:48616) 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c2569700 1 -- 172.21.15.59:0/2529945563 learned_addr learned my addr 172.21.15.59:0/2529945563 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 0x556e07acfa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c1567700 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aea860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c2569700 1 -- 172.21.15.59:0/2529945563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 msgr2=0x556e07aea860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c2569700 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aea860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c2569700 1 -- 172.21.15.59:0/2529945563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 msgr2=0x556e07acfa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c2569700 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 0x556e07acfa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c2569700 1 -- 172.21.15.59:0/2529945563 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e07adf180 con 0x556e07acac30 2024-05-08T05:26:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c1567700 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:26:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 0x556e07acfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:26:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84c2569700 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07abb550 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f84b40e25c0 tx=0x7f84b40a8c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2111bea723b1367e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84a37fe700 1 -- 172.21.15.59:0/2529945563 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40a5540 con 0x556e07acac30 2024-05-08T05:26:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84a37fe700 1 -- 172.21.15.59:0/2529945563 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b40a9680 con 0x556e07acac30 2024-05-08T05:26:00.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.634+0000 7f84a37fe700 1 -- 172.21.15.59:0/2529945563 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b40b2140 con 0x556e07acac30 2024-05-08T05:26:00.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2529945563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 msgr2=0x556e07abb550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:00.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07abb550 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f84b40e25c0 tx=0x7f84b40a8c90 comp rx=0 tx=0).stop 2024-05-08T05:26:00.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2529945563 shutdown_connections 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07abb550 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 0x556e07acfa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2529945563 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07aea860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2529945563 >> 172.21.15.59:0/2529945563 conn(0x556e079f20b0 msgr2=0x556e07abae40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2529945563 shutdown_connections 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2529945563 wait complete. 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 -- start start 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.635+0000 7f84ce85fb80 1 Processor -- start 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 0x556e07b431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07b436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84ce85fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07b472a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556e07ae0e90 con 0x556e07acac30 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556e07ae07c0 con 0x556e07ac6480 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84ce85fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556e07ae1560 con 0x556e07ac6da0 2024-05-08T05:26:00.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84ce85fb80 10 client.0 _ll_register_callbacks cb 0x556e079b08d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c2569700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07b472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c2569700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07b472a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48628/0 (socket says 172.21.15.59:48628) 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c2569700 1 -- 172.21.15.59:0/2050527317 learned_addr learned my addr 172.21.15.59:0/2050527317 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c1567700 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07b436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 0x556e07b431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c2569700 1 -- 172.21.15.59:0/2050527317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 msgr2=0x556e07b436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c2569700 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07b436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c2569700 1 -- 172.21.15.59:0/2050527317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 msgr2=0x556e07b431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c2569700 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 0x556e07b431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c2569700 1 -- 172.21.15.59:0/2050527317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e07b47980 con 0x556e07acac30 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84c2569700 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07b472a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84b4000c00 tx=0x7f84b40a8b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d165aa490d89fb61 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.636+0000 7f84a17fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84a17fa700 1 -- 172.21.15.59:0/2050527317 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84b4087020 con 0x556e07acac30 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84a17fa700 1 -- 172.21.15.59:0/2050527317 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7f84b40b2050 con 0x556e07acac30 2024-05-08T05:26:00.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2050527317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84b40a9680 con 0x556e07acac30 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84a17fa700 1 -- 172.21.15.59:0/2050527317 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7f84b400ed90 con 0x556e07acac30 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84ce85fb80 10 client.14685 fetch_fsmap learned FSMap version 17 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2050527317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x556e07b4fe60 con 0x556e07acac30 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84a17fa700 1 -- 172.21.15.59:0/2050527317 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84b40a9680 con 0x556e07acac30 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84a17fa700 1 -- 172.21.15.59:0/2050527317 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7f84b406e020 con 0x556e07acac30 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84ce85fb80 10 client.14685 fetch_fsmap finished waiting for FSMap version 17 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84ce85fb80 10 client.14685 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84ce85fb80 20 client.14685 populate_metadata read hostname 'smithi059' 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556e07b54400 0x556e07b53110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84ce85fb80 4 client.14685 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.637+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2050527317 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "status"}) -- 0x556e07a261f0 con 0x556e07b54400 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556e07b54400 0x556e07b53110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84a17fa700 10 client.14685 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84c1d68700 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556e07b54400 0x556e07b53110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b0e01885cd69e8b server_cookie=d7be92c86236753f in_seq=0 out_seq=0 2024-05-08T05:26:00.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84a17fa700 1 -- 172.21.15.59:0/2050527317 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x556e07a261f0 con 0x556e07b54400 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84a17fa700 10 client.14685 handle_command_reply: tid=0 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 client.14685 shutdown 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2050527317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 msgr2=0x556e07b472a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07b472a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f84b4000c00 tx=0x7f84b40a8b10 comp rx=0 tx=0).stop 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2050527317 shutdown_connections 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556e07b54400 0x556e07b53110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556e07acac30 0x556e07b472a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556e07ac6480 0x556e07b431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84a17fa700 0 client.14685 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 --2- 172.21.15.59:0/2050527317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556e07ac6da0 0x556e07b436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2050527317 >> 172.21.15.59:0/2050527317 conn(0x556e079f20b0 msgr2=0x556e07abb6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2050527317 shutdown_connections 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.638+0000 7f84ce85fb80 1 -- 172.21.15.59:0/2050527317 wait complete. 2024-05-08T05:26:00.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84ce85fb80 20 client.14685 trim_cache size 0 max 16384 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24473, 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 32.975568621000001, 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 17, 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 86, 2024-05-08T05:26:00.645 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 86, 2024-05-08T05:26:00.646 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 48.894585456000001 2024-05-08T05:26:00.646 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:26:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 -- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84a40b2880 msgr2=0x7f84a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84a40b2880 0x7f84a40b4d40 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f84ac007fd0 tx=0x7f84ac007830 comp rx=0 tx=0).stop 2024-05-08T05:26:00.646 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 -- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc108870 msgr2=0x7f84bc1999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc108870 0x7f84bc1999c0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f84b400d980 tx=0x7f84b400dc90 comp rx=0 tx=0).stop 2024-05-08T05:26:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 -- 172.21.15.59:0/2169072643 shutdown_connections 2024-05-08T05:26:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84bc108870 0x7f84bc1999c0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84a40b2880 0x7f84a40b4d40 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84bc103780 0x7f84bc19f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 --2- 172.21.15.59:0/2169072643 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84bc104130 0x7f84bc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 -- 172.21.15.59:0/2169072643 >> 172.21.15.59:0/2169072643 conn(0x7f84bc07a610 msgr2=0x7f84bc101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 -- 172.21.15.59:0/2169072643 shutdown_connections 2024-05-08T05:26:00.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:00.639+0000 7f84c3fcc700 1 -- 172.21.15.59:0/2169072643 wait complete. 2024-05-08T05:26:01.288 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:01 smithi059 ceph-mon[30304]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:01.316 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:26:01.316 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:26:01.316 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T05:26:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:01 smithi143 ceph-mon[34337]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:01 smithi096 ceph-mon[34253]: pgmap v314: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:01.569 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:03.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 -- 172.21.15.59:0/61482923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38806bbd0 msgr2=0x7fd38806c050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:03.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 --2- 172.21.15.59:0/61482923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38806bbd0 0x7fd38806c050 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fd374009990 tx=0x7fd374009ca0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 -- 172.21.15.59:0/61482923 shutdown_connections 2024-05-08T05:26:03.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 --2- 172.21.15.59:0/61482923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd38806c590 0x7fd388110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 --2- 172.21.15.59:0/61482923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd388109510 0x7fd3881098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 --2- 172.21.15.59:0/61482923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd38806bbd0 0x7fd38806c050 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 -- 172.21.15.59:0/61482923 >> 172.21.15.59:0/61482923 conn(0x7fd388079350 msgr2=0x7fd388079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 -- 172.21.15.59:0/61482923 shutdown_connections 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 -- 172.21.15.59:0/61482923 wait complete. 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 Processor -- start 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 -- start start 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.258+0000 7fd38eb08700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd38806bbd0 0x7fd38819f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38eb08700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd38806c590 0x7fd38819fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38eb08700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd388109510 0x7fd388199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38eb08700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd388113720 con 0x7fd388109510 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38eb08700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881135a0 con 0x7fd38806c590 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38eb08700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd3881138a0 con 0x7fd38806bbd0 2024-05-08T05:26:03.261 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38d0a5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd388109510 0x7fd388199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38d0a5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd388109510 0x7fd388199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48644/0 (socket says 172.21.15.59:48644) 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38d0a5700 1 -- 172.21.15.59:0/2763498347 learned_addr learned my addr 172.21.15.59:0/2763498347 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd387fff700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd38806c590 0x7fd38819fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38c8a4700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd38806bbd0 0x7fd38819f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38d0a5700 1 -- 172.21.15.59:0/2763498347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd38806bbd0 msgr2=0x7fd38819f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38d0a5700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd38806bbd0 0x7fd38819f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38d0a5700 1 -- 172.21.15.59:0/2763498347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd38806c590 msgr2=0x7fd38819fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38d0a5700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd38806c590 0x7fd38819fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38d0a5700 1 -- 172.21.15.59:0/2763498347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd38819a260 con 0x7fd388109510 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38d0a5700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd388109510 0x7fd388199b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd38000b970 tx=0x7fd38000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd385ffb700 1 -- 172.21.15.59:0/2763498347 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd380019070 con 0x7fd388109510 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd38819a550 con 0x7fd388109510 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd385ffb700 1 -- 172.21.15.59:0/2763498347 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd38000cde0 con 0x7fd388109510 2024-05-08T05:26:03.262 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.259+0000 7fd385ffb700 1 -- 172.21.15.59:0/2763498347 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd38001d3b0 con 0x7fd388109510 2024-05-08T05:26:03.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.260+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3881a6810 con 0x7fd388109510 2024-05-08T05:26:03.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.261+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd38810b9f0 con 0x7fd388109510 2024-05-08T05:26:03.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.262+0000 7fd385ffb700 1 -- 172.21.15.59:0/2763498347 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fd38001c070 con 0x7fd388109510 2024-05-08T05:26:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.263+0000 7fd385ffb700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd3780b2880 0x7fd3780b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.263+0000 7fd38c8a4700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd3780b2880 0x7fd3780b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.263+0000 7fd385ffb700 1 -- 172.21.15.59:0/2763498347 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8802+0+0 (secure 0 0 0) 0x7fd3800e1af0 con 0x7fd388109510 2024-05-08T05:26:03.264 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.264+0000 7fd38c8a4700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd3780b2880 0x7fd3780b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006fd0 tx=0x7fd37c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:03.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.267+0000 7fd385ffb700 1 -- 172.21.15.59:0/2763498347 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd3800a9570 con 0x7fd388109510 2024-05-08T05:26:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:03 smithi059 ceph-mon[30304]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:03 smithi143 ceph-mon[34337]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:03 smithi096 ceph-mon[34253]: pgmap v315: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:03.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.532+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fd38819aff0 con 0x7fd388109510 2024-05-08T05:26:03.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.532+0000 7fd385ffb700 1 -- 172.21.15.59:0/2763498347 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7fd3800a8e70 con 0x7fd388109510 2024-05-08T05:26:03.532 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd3780b2880 msgr2=0x7fd3780b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd3780b2880 0x7fd3780b4d40 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fd37c006fd0 tx=0x7fd37c008040 comp rx=0 tx=0).stop 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd388109510 msgr2=0x7fd388199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd388109510 0x7fd388199b00 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fd38000b970 tx=0x7fd38000bc80 comp rx=0 tx=0).stop 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 shutdown_connections 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd388109510 0x7fd388199b00 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd38806bbd0 0x7fd38819f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd3780b2880 0x7fd3780b4d40 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 --2- 172.21.15.59:0/2763498347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd38806c590 0x7fd38819fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:03.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 >> 172.21.15.59:0/2763498347 conn(0x7fd388079350 msgr2=0x7fd3881012a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:03.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.534+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 shutdown_connections 2024-05-08T05:26:03.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:03.535+0000 7fd38eb08700 1 -- 172.21.15.59:0/2763498347 wait complete. 2024-05-08T05:26:04.205 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2763498347' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:26:04.205 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-08T05:26:04.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-08T05:26:04.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T05:26:04.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-08T05:26:04.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-08T05:26:04.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-08T05:26:04.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-08T05:26:04.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-05-08T05:26:04.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T05:26:04.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-08T05:26:04.238 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-05-08T05:26:04.238 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:26:04.238 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-05-08T05:26:04.238 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-05-08T05:26:04.238 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:26:04.238 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-05-08T05:26:04.238 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:26:04.238 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:26:04.238 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T05:26:04.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:04.239 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-05-08T05:26:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 42130sec preferred_lft 42130sec 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-05-08T05:26:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-05-08T05:26:04.259 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T05:26:04.259 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:26:04.259 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:26:04.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2024-05-08T05:26:04.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2024-05-08T05:26:04.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2024-05-08T05:26:04.259 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T05:26:04.259 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:26:04.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:26:04.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:26:04.461 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:04.461 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:26:04.490 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-05-08T05:26:04.490 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:04.490 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-05-08T05:26:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2763498347' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:26:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-08T05:26:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-08T05:26:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T05:26:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-08T05:26:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-08T05:26:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-08T05:26:04.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-08T05:26:04.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-05-08T05:26:04.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T05:26:04.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-08T05:26:04.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2763498347' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]: dispatch 2024-05-08T05:26:04.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]: dispatch 2024-05-08T05:26:04.568 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-05-08T05:26:04.569 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:26:04.569 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:26:04.569 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:26:04.569 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:26:04.569 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:26:04.569 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:26:04.569 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:26:04.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:26:04.570 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:26:04.570 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:26:04.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:04 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:26:04.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:04 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:26:04.859 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:04.859 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-05-08T05:26:04.875 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:04.876 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-05-08T05:26:04.932 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:26:04.932 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:26:04.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:26:04.932 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-05-08T05:26:04.932 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:26:05.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:26:05.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:26:05.052 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-05-08T05:26:05.053 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:26:05.053 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:26:05.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-05-08T05:26:05.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T05:26:05.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-05-08T05:26:05.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-05-08T05:26:05.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-05-08T05:26:05.053 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:26:05.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:26:05.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:26:05.185 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:26:05.185 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:26:05.185 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-05-08T05:26:05.185 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T05:26:05.186 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:26:05.235 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:05 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:26:05.259 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:26:05.265 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-05-08T05:26:05.265 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:26:05.265 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:26:05.286 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-05-08T05:26:05.286 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:26:05.287 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-05-08T05:26:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T05:26:05.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-08T05:26:05.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:05 smithi059 ceph-mon[30304]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T05:26:05.343 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-05-08T05:26:05.409 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-05-08T05:26:05.467 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-05-08T05:26:05.468 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/25940d8187c8b978313634d4d6b17d69298615990826438dfcdd06160cf2ceb8/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/4e4a8f07ae480f6d462fa054eba1deb5233e4d3376c4b6f849f8adc0eed6887e/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/0 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/6f7eb7d29f5036d45d453b0c3e49c5d35cc2574f03396ea9c3200db311f97ad0/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/3d808424d71404adc2c214700856b271b6e2c9689867ce6fb88beb316433d71d/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/ffb6e483119d6ce0a25feb1cd6ee3a53298e6ac969b5063edf31c032801aab33/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/f58ec6f7eca477735d7060240ebb22fd1f3bd99454c69c7ae44551ee5fa6f4ab/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/5180cad2edabbfb14e4ed2da295919352509b4252d46d4cb24835794f7d20640/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/e71ba56b2748132e2a2071be175fd7bf376d95bb827468f9ce264e5942ff1222/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/f653dfe3c73292de0ca7ea97a40b0129b117d8b73d635378e73b0418cd807727/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/f5be2a56513d7ddbfeec3dddb80e8fbabcb10e7fadd40bff0b40ea60c664291e/merged 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:26:05.469 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:26:05.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:05.470 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-05-08T05:26:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-08T05:26:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-08T05:26:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T05:26:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-08T05:26:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-08T05:26:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-08T05:26:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-08T05:26:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-05-08T05:26:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T05:26:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-08T05:26:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:05 smithi143 ceph-mon[34337]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T05:26:05.523 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-05-08T05:26:05.523 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-05-08T05:26:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: pgmap v316: 129 pgs: 129 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.5", "id": [8, 11]}]': finished 2024-05-08T05:26:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 4]}]': finished 2024-05-08T05:26:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-05-08T05:26:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1d", "id": [4, 8]}]': finished 2024-05-08T05:26:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.21", "id": [5, 9]}]': finished 2024-05-08T05:26:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.2c", "id": [4, 9]}]': finished 2024-05-08T05:26:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.2", "id": [9, 1]}]': finished 2024-05-08T05:26:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.9", "id": [5, 8]}]': finished 2024-05-08T05:26:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.12", "id": [9, 10]}]': finished 2024-05-08T05:26:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.18", "id": [9, 8]}]': finished 2024-05-08T05:26:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:05 smithi096 ceph-mon[34253]: osdmap e87: 12 total, 12 up, 12 in 2024-05-08T05:26:05.565 DEBUG:teuthology.orchestra.run.smithi059:> sudo modprobe fuse 2024-05-08T05:26:05.591 DEBUG:teuthology.orchestra.run.smithi059:> cat /proc/self/mounts | awk '{print $2}' 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/proc 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/sys 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/dev 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/security 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/shm 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/pts 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/run 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/cgroup 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/pstore 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/bpf 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/ 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/selinux 2024-05-08T05:26:05.648 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/hugepages 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/dev/mqueue 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/debug 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/tracing 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/fs/fuse/connections 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/sys/kernel/config 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-sysctl.service 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/nfs/rpc_pipefs 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/1000 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/ceph 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/25940d8187c8b978313634d4d6b17d69298615990826438dfcdd06160cf2ceb8/merged 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/4e4a8f07ae480f6d462fa054eba1deb5233e4d3376c4b6f849f8adc0eed6887e/merged 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/run/user/0 2024-05-08T05:26:05.649 INFO:teuthology.orchestra.run.smithi059.stdout:/proc/sys/fs/binfmt_misc 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/6f7eb7d29f5036d45d453b0c3e49c5d35cc2574f03396ea9c3200db311f97ad0/merged 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/3d808424d71404adc2c214700856b271b6e2c9689867ce6fb88beb316433d71d/merged 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/ffb6e483119d6ce0a25feb1cd6ee3a53298e6ac969b5063edf31c032801aab33/merged 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/f58ec6f7eca477735d7060240ebb22fd1f3bd99454c69c7ae44551ee5fa6f4ab/merged 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/5180cad2edabbfb14e4ed2da295919352509b4252d46d4cb24835794f7d20640/merged 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/e71ba56b2748132e2a2071be175fd7bf376d95bb827468f9ce264e5942ff1222/merged 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/f653dfe3c73292de0ca7ea97a40b0129b117d8b73d635378e73b0418cd807727/merged 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/var/lib/containers/storage/overlay/f5be2a56513d7ddbfeec3dddb80e8fbabcb10e7fadd40bff0b40ea60c664291e/merged 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run.smithi059.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:26:05.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:05.651 DEBUG:teuthology.orchestra.run.smithi059:> ls /sys/fs/fuse/connections 2024-05-08T05:26:05.673 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:2024-05-08T05:26:05.672+0000 7fb244d24480 -1 init, newargv = 0x5573871ef950 newargc=15 2024-05-08T05:26:05.673 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:ceph-fuse[76505]: starting ceph client 2024-05-08T05:26:05.679 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:ceph-fuse[76505]: starting fuse 2024-05-08T05:26:05.689 INFO:teuthology.orchestra.run.smithi059.stdout:103 2024-05-08T05:26:05.690 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [103] 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> sudo stdin-killer -- python3 -c ' 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> import glob 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> import re 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> import os 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> import subprocess 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> def _find_admin_socket(client_name): 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> files = glob.glob(asok_path) 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> 2024-05-08T05:26:05.690 DEBUG:teuthology.orchestra.run.smithi059:> # Given a non-glob path, it better be there 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> if "*" not in asok_path: 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> assert(len(files) == 1) 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> return files[0] 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> for f in files: 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> if os.path.exists("/proc/{0}".format(pid)): 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> contents = proc_f.read() 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> if mountpoint in contents: 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> return f 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> print(_find_admin_socket("client.admin")) 2024-05-08T05:26:05.691 DEBUG:teuthology.orchestra.run.smithi059:> ' 2024-05-08T05:26:05.811 INFO:teuthology.orchestra.run.smithi059.stdout:/var/run/ceph/ceph-client.admin.76505.asok 2024-05-08T05:26:05.814 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:05 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:26:05.822 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.76505.asok 2024-05-08T05:26:05.822 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:05.822 DEBUG:teuthology.orchestra.run.smithi059:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.76505.asok status 2024-05-08T05:26:05.962 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "metadata": { 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_sha1": "bee299f4ac344e4c45e550d7a80abcadd7bc3d5a", 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_version": "ceph version 19.0.0-3538-gbee299f4 (bee299f4ac344e4c45e550d7a80abcadd7bc3d5a) squid (dev)", 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "entity_id": "admin", 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": "smithi059", 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "pid": "76505", 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "root": "/" 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_count": 0, 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "dentry_pinned_count": 0, 2024-05-08T05:26:05.963 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 14697, 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "inst": { 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "name": { 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "client", 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "num": 14697 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3666387376 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": { 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "192.168.0.1:0", 2024-05-08T05:26:05.964 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3666387376 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout: "inst_str": "client.14697 192.168.0.1:0/3666387376", 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout: "addr_str": "192.168.0.1:0/3666387376", 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout: "inode_count": 1, 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_epoch": 17, 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch": 88, 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_epoch_barrier": 0, 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout: "blocklisted": false, 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs" 2024-05-08T05:26:05.965 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:26:05.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:05.971 DEBUG:teuthology.orchestra.run.smithi059:> stat --file-system '--printf=%T 2024-05-08T05:26:05.971 DEBUG:teuthology.orchestra.run.smithi059:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:26:05.988 INFO:teuthology.orchestra.run.smithi059.stdout:fuseblk 2024-05-08T05:26:05.988 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:26:05.988 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:05.988 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:26:06.055 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-05-08T05:26:06.148 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:06 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:26:06.172 INFO:teuthology.orchestra.run.smithi059.stdout:# file: . 2024-05-08T05:26:06.172 INFO:teuthology.orchestra.run.smithi059.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-05-08T05:26:06.172 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:26:06.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:06 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:26:06.179 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:26:06.179 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:26:06.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-08T05:26:06.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:06 smithi059 ceph-mon[30304]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T05:26:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:06 smithi143 ceph-mon[34337]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T05:26:06.495 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:06.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:06 smithi096 ceph-mon[34253]: osdmap e88: 12 total, 12 up, 12 in 2024-05-08T05:26:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:07 smithi143 ceph-mon[34337]: pgmap v319: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:07 smithi096 ceph-mon[34253]: pgmap v319: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:07.540 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:07 smithi059 ceph-mon[30304]: pgmap v319: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:26:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.215+0000 7ff734ad1700 1 -- 172.21.15.59:0/1095403603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff730103eb0 msgr2=0x7ff730110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.215+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1095403603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff730103eb0 0x7ff730110390 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7ff72800b0a0 tx=0x7ff72800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.215+0000 7ff734ad1700 1 -- 172.21.15.59:0/1095403603 shutdown_connections 2024-05-08T05:26:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.215+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1095403603 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff730103eb0 0x7ff730110390 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.215+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1095403603 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff730109600 0x7ff7301099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.215+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1095403603 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7301034f0 0x7ff730103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.215+0000 7ff734ad1700 1 -- 172.21.15.59:0/1095403603 >> 172.21.15.59:0/1095403603 conn(0x7ff73007a650 msgr2=0x7ff73007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.215+0000 7ff734ad1700 1 -- 172.21.15.59:0/1095403603 shutdown_connections 2024-05-08T05:26:08.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.215+0000 7ff734ad1700 1 -- 172.21.15.59:0/1095403603 wait complete. 2024-05-08T05:26:08.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.216+0000 7ff734ad1700 1 Processor -- start 2024-05-08T05:26:08.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.216+0000 7ff734ad1700 1 -- start start 2024-05-08T05:26:08.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff734ad1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7301034f0 0x7ff7301a18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff734ad1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff730103eb0 0x7ff7301a1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff734ad1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff730109600 0x7ff73019bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff734ad1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff730113670 con 0x7ff7301034f0 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff734ad1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7301134f0 con 0x7ff730103eb0 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff734ad1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff7301137f0 con 0x7ff730109600 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7301034f0 0x7ff7301a18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7301034f0 0x7ff7301a18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48662/0 (socket says 172.21.15.59:48662) 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72e59c700 1 -- 172.21.15.59:0/1862307749 learned_addr learned my addr 172.21.15.59:0/1862307749 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72ed9d700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff730109600 0x7ff73019bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff725d9b700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff730103eb0 0x7ff7301a1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72e59c700 1 -- 172.21.15.59:0/1862307749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff730109600 msgr2=0x7ff73019bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72e59c700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff730109600 0x7ff73019bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72e59c700 1 -- 172.21.15.59:0/1862307749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff730103eb0 msgr2=0x7ff7301a1e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:08.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72e59c700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff730103eb0 0x7ff7301a1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72e59c700 1 -- 172.21.15.59:0/1862307749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff73019c210 con 0x7ff7301034f0 2024-05-08T05:26:08.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff72e59c700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7301034f0 0x7ff7301a18f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff72000e9a0 tx=0x7ff72000ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:08.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.217+0000 7ff727fff700 1 -- 172.21.15.59:0/1862307749 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72000cbd0 con 0x7ff7301034f0 2024-05-08T05:26:08.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.218+0000 7ff727fff700 1 -- 172.21.15.59:0/1862307749 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff72000cd70 con 0x7ff7301034f0 2024-05-08T05:26:08.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.218+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff73019c500 con 0x7ff7301034f0 2024-05-08T05:26:08.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.218+0000 7ff727fff700 1 -- 172.21.15.59:0/1862307749 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720018680 con 0x7ff7301034f0 2024-05-08T05:26:08.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.218+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7301a87c0 con 0x7ff7301034f0 2024-05-08T05:26:08.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.219+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff73006a190 con 0x7ff7301034f0 2024-05-08T05:26:08.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.220+0000 7ff727fff700 1 -- 172.21.15.59:0/1862307749 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7ff720010430 con 0x7ff7301034f0 2024-05-08T05:26:08.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.221+0000 7ff727fff700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff71c0b2900 0x7ff71c0b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:08.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.221+0000 7ff725d9b700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff71c0b2900 0x7ff71c0b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:08.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.221+0000 7ff727fff700 1 -- 172.21.15.59:0/1862307749 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7ff7200e2c80 con 0x7ff7301034f0 2024-05-08T05:26:08.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.222+0000 7ff725d9b700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff71c0b2900 0x7ff71c0b4dc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff73007af00 tx=0x7ff718005cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:08.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.224+0000 7ff727fff700 1 -- 172.21.15.59:0/1862307749 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7200aa5e0 con 0x7ff7301034f0 2024-05-08T05:26:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.459+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7ff73019cfe0 con 0x7ff7301034f0 2024-05-08T05:26:08.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.459+0000 7ff727fff700 1 -- 172.21.15.59:0/1862307749 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+585 (secure 0 0 0) 0x7ff7200a9ee0 con 0x7ff7301034f0 2024-05-08T05:26:08.461 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1451301612 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:08.461 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3771264184 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:08.461 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/95297313 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:08.461 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2518610768 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:08.461 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2478322707 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:08.461 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/95297313 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:08.462 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/817743540 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:08.462 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/817743540 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:08.462 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3865726924 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:08.462 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/362628113 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:08.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff71c0b2900 msgr2=0x7ff71c0b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:08.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff71c0b2900 0x7ff71c0b4dc0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff73007af00 tx=0x7ff718005cf0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7301034f0 msgr2=0x7ff7301a18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:08.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7301034f0 0x7ff7301a18f0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7ff72000e9a0 tx=0x7ff72000ecb0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 shutdown_connections 2024-05-08T05:26:08.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff730109600 0x7ff73019bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7301034f0 0x7ff7301a18f0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff71c0b2900 0x7ff71c0b4dc0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 --2- 172.21.15.59:0/1862307749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff730103eb0 0x7ff7301a1e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:08.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 >> 172.21.15.59:0/1862307749 conn(0x7ff73007a650 msgr2=0x7ff73010e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:08.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 shutdown_connections 2024-05-08T05:26:08.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:08.462+0000 7ff734ad1700 1 -- 172.21.15.59:0/1862307749 wait complete. 2024-05-08T05:26:08.464 INFO:teuthology.orchestra.run.smithi059.stderr:listed 10 entries 2024-05-08T05:26:09.214 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:26:09.214 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:26:09.233 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-08T05:26:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:09 smithi143 ceph-mon[34337]: pgmap v320: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:09 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1862307749' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:26:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:09 smithi096 ceph-mon[34253]: pgmap v320: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:09.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:09 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1862307749' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:26:09.530 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:09.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:09 smithi059 ceph-mon[30304]: pgmap v320: 129 pgs: 3 peering, 126 active+clean; 579 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:09.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:09 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1862307749' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:26:11.363 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:11 smithi059 ceph-mon[30304]: pgmap v321: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 -- 172.21.15.59:0/1378996876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006bc10 msgr2=0x7fbe6006bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 --2- 172.21.15.59:0/1378996876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006bc10 0x7fbe6006bff0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c009990 tx=0x7fbe5c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 -- 172.21.15.59:0/1378996876 shutdown_connections 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 --2- 172.21.15.59:0/1378996876 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe6010ffe0 0x7fbe601123d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 --2- 172.21.15.59:0/1378996876 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006bc10 0x7fbe6006bff0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 --2- 172.21.15.59:0/1378996876 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe6006c530 0x7fbe6010faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 -- 172.21.15.59:0/1378996876 >> 172.21.15.59:0/1378996876 conn(0x7fbe600783f0 msgr2=0x7fbe60078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 -- 172.21.15.59:0/1378996876 shutdown_connections 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 -- 172.21.15.59:0/1378996876 wait complete. 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.372+0000 7fbe67879700 1 Processor -- start 2024-05-08T05:26:11.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe67879700 1 -- start start 2024-05-08T05:26:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe67879700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe6006bc10 0x7fbe6019bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe67879700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006c530 0x7fbe6019c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe67879700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe6010ffe0 0x7fbe601a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe67879700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe601158e0 con 0x7fbe6006c530 2024-05-08T05:26:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe67879700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60115760 con 0x7fbe6006bc10 2024-05-08T05:26:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe67879700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe60115a60 con 0x7fbe6010ffe0 2024-05-08T05:26:11.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe64e14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006c530 0x7fbe6019c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe64e14700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006c530 0x7fbe6019c2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:60998/0 (socket says 172.21.15.59:60998) 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe64e14700 1 -- 172.21.15.59:0/3187436416 learned_addr learned my addr 172.21.15.59:0/3187436416 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe65615700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe6006bc10 0x7fbe6019bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe65e16700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe6010ffe0 0x7fbe601a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe64e14700 1 -- 172.21.15.59:0/3187436416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe6010ffe0 msgr2=0x7fbe601a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe64e14700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe6010ffe0 0x7fbe601a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe64e14700 1 -- 172.21.15.59:0/3187436416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe6006bc10 msgr2=0x7fbe6019bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe64e14700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe6006bc10 0x7fbe6019bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe64e14700 1 -- 172.21.15.59:0/3187436416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe601a0da0 con 0x7fbe6006c530 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe65e16700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe6010ffe0 0x7fbe601a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe65615700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe6006bc10 0x7fbe6019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.373+0000 7fbe64e14700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006c530 0x7fbe6019c2d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d950 tx=0x7fbe5000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.374+0000 7fbe567fc700 1 -- 172.21.15.59:0/3187436416 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe50016070 con 0x7fbe6006c530 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.374+0000 7fbe567fc700 1 -- 172.21.15.59:0/3187436416 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe50010460 con 0x7fbe6006c530 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.374+0000 7fbe567fc700 1 -- 172.21.15.59:0/3187436416 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe5000f620 con 0x7fbe6006c530 2024-05-08T05:26:11.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.374+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe601a1090 con 0x7fbe6006c530 2024-05-08T05:26:11.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.374+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe601a8910 con 0x7fbe6006c530 2024-05-08T05:26:11.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.375+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbe6006a190 con 0x7fbe6006c530 2024-05-08T05:26:11.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.376+0000 7fbe567fc700 1 -- 172.21.15.59:0/3187436416 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fbe50019070 con 0x7fbe6006c530 2024-05-08T05:26:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.377+0000 7fbe567fc700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbe4c0b2880 0x7fbe4c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.377+0000 7fbe65615700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbe4c0b2880 0x7fbe4c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.377+0000 7fbe567fc700 1 -- 172.21.15.59:0/3187436416 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fbe500e2290 con 0x7fbe6006c530 2024-05-08T05:26:11.379 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.378+0000 7fbe65615700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbe4c0b2880 0x7fbe4c0b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c006010 tx=0x7fbe5c00b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:11.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.382+0000 7fbe567fc700 1 -- 172.21.15.59:0/3187436416 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fbe500a9bf0 con 0x7fbe6006c530 2024-05-08T05:26:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:11 smithi143 ceph-mon[34337]: pgmap v321: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:11.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:11 smithi096 ceph-mon[34253]: pgmap v321: 129 pgs: 3 peering, 126 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:11.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.611+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fbe6002e510 con 0x7fbe6006c530 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.611+0000 7fbe567fc700 1 -- 172.21.15.59:0/3187436416 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v88) ==== 81+0+585 (secure 0 0 0) 0x7fbe500a94f0 con 0x7fbe6006c530 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1451301612 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3771264184 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/95297313 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2518610768 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2478322707 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/95297313 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/817743540 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/817743540 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:11.613 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3865726924 2024-05-09T05:17:35.412203+0000 2024-05-08T05:26:11.614 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/362628113 2024-05-09T05:18:03.888613+0000 2024-05-08T05:26:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbe4c0b2880 msgr2=0x7fbe4c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbe4c0b2880 0x7fbe4c0b4d40 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbe5c006010 tx=0x7fbe5c00b540 comp rx=0 tx=0).stop 2024-05-08T05:26:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006c530 msgr2=0x7fbe6019c2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006c530 0x7fbe6019c2d0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fbe5000d950 tx=0x7fbe5000dd10 comp rx=0 tx=0).stop 2024-05-08T05:26:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 shutdown_connections 2024-05-08T05:26:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe6010ffe0 0x7fbe601a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fbe4c0b2880 0x7fbe4c0b4d40 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe6006bc10 0x7fbe6019bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 --2- 172.21.15.59:0/3187436416 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe6006c530 0x7fbe6019c2d0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:11.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.614+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 >> 172.21.15.59:0/3187436416 conn(0x7fbe600783f0 msgr2=0x7fbe60111680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:11.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.615+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 shutdown_connections 2024-05-08T05:26:11.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:11.615+0000 7fbe67879700 1 -- 172.21.15.59:0/3187436416 wait complete. 2024-05-08T05:26:11.617 INFO:teuthology.orchestra.run.smithi059.stderr:listed 10 entries 2024-05-08T05:26:12.300 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:12 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3187436416' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:26:12.330 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi059.front.sepia.ceph.com... 2024-05-08T05:26:12.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:12.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-05-08T05:26:12.358 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi059.stderr:ceph-fuse[76505]: fuse finished with error 0 and tester_r 0 2024-05-08T05:26:12.360 INFO:teuthology.orchestra.run:waiting for 300 2024-05-08T05:26:12.459 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:26:12.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:26:12.460 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-05-08T05:26:12.483 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:26:12.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:12 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3187436416' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:26:12.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:12 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3187436416' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:26:12.775 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:13 smithi143 ceph-mon[34337]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:13.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:13 smithi096 ceph-mon[34253]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:13.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:13 smithi059 ceph-mon[30304]: pgmap v322: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:14.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.460+0000 7f882c4ea700 1 -- 172.21.15.59:0/1153904385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824104130 msgr2=0x7f8824108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.460+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1153904385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824104130 0x7f8824108140 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f88200099c0 tx=0x7f8820009cd0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.460+0000 7f882c4ea700 1 -- 172.21.15.59:0/1153904385 shutdown_connections 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.460+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1153904385 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8824108870 0x7f88241103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.460+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1153904385 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824103780 0x7f8824103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.460+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1153904385 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824104130 0x7f8824108140 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.460+0000 7f882c4ea700 1 -- 172.21.15.59:0/1153904385 >> 172.21.15.59:0/1153904385 conn(0x7f882407a610 msgr2=0x7f882407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.460+0000 7f882c4ea700 1 -- 172.21.15.59:0/1153904385 shutdown_connections 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.460+0000 7f882c4ea700 1 -- 172.21.15.59:0/1153904385 wait complete. 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882c4ea700 1 Processor -- start 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882c4ea700 1 -- start start 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882c4ea700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8824103780 0x7f882419f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882c4ea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824104130 0x7f882419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:14.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882c4ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824108870 0x7f8824199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882c4ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88241136b0 con 0x7f8824108870 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882c4ea700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8824113530 con 0x7f8824104130 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882c4ea700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8824113830 con 0x7f8824103780 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882aa87700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824108870 0x7f8824199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f8829a85700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824104130 0x7f882419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882aa87700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824108870 0x7f8824199970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32780/0 (socket says 172.21.15.59:32780) 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882aa87700 1 -- 172.21.15.59:0/1372957127 learned_addr learned my addr 172.21.15.59:0/1372957127 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.461+0000 7f882a286700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8824103780 0x7f882419f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f8829a85700 1 -- 172.21.15.59:0/1372957127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8824103780 msgr2=0x7f882419f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f8829a85700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8824103780 0x7f882419f7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f8829a85700 1 -- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824108870 msgr2=0x7f8824199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:14.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f8829a85700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824108870 0x7f8824199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f8829a85700 1 -- 172.21.15.59:0/1372957127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f882419a0d0 con 0x7f8824104130 2024-05-08T05:26:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f882aa87700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824108870 0x7f8824199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:26:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f8829a85700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824104130 0x7f882419fcf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8820009f50 tx=0x7f88200047b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f88177fe700 1 -- 172.21.15.59:0/1372957127 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820027070 con 0x7f8824104130 2024-05-08T05:26:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f882419a360 con 0x7f8824104130 2024-05-08T05:26:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f88177fe700 1 -- 172.21.15.59:0/1372957127 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8820003f20 con 0x7f8824104130 2024-05-08T05:26:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f88177fe700 1 -- 172.21.15.59:0/1372957127 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8820022bd0 con 0x7f8824104130 2024-05-08T05:26:14.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.462+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88241a6680 con 0x7f8824104130 2024-05-08T05:26:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.463+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8824104c00 con 0x7f8824104130 2024-05-08T05:26:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.465+0000 7f88177fe700 1 -- 172.21.15.59:0/1372957127 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f882002a070 con 0x7f8824104130 2024-05-08T05:26:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.465+0000 7f88177fe700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88100b2880 0x7f88100b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.466+0000 7f882a286700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88100b2880 0x7f88100b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.466+0000 7f88177fe700 1 -- 172.21.15.59:0/1372957127 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f88200e38b0 con 0x7f8824104130 2024-05-08T05:26:14.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.466+0000 7f882a286700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88100b2880 0x7f88100b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f882419aab0 tx=0x7f881800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:14.470 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.469+0000 7f88177fe700 1 -- 172.21.15.59:0/1372957127 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8820031070 con 0x7f8824104130 2024-05-08T05:26:14.686 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.685+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f882402e510 con 0x7f8824104130 2024-05-08T05:26:14.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.686+0000 7f88177fe700 1 -- 172.21.15.59:0/1372957127 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24601 (secure 0 0 0) 0x7f8820013070 con 0x7f8824104130 2024-05-08T05:26:14.688 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:26:14.689 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:26:05.137852+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:26:14.690 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88100b2880 msgr2=0x7f88100b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88100b2880 0x7f88100b4d40 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f882419aab0 tx=0x7f881800b410 comp rx=0 tx=0).stop 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824104130 msgr2=0x7f882419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824104130 0x7f882419fcf0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8820009f50 tx=0x7f88200047b0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 shutdown_connections 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8824108870 0x7f8824199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8824103780 0x7f882419f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88100b2880 0x7f88100b4d40 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 --2- 172.21.15.59:0/1372957127 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8824104130 0x7f882419fcf0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 >> 172.21.15.59:0/1372957127 conn(0x7f882407a610 msgr2=0x7f88241017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 shutdown_connections 2024-05-08T05:26:14.691 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:14.689+0000 7f882c4ea700 1 -- 172.21.15.59:0/1372957127 wait complete. 2024-05-08T05:26:15.362 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:15 smithi059 ceph-mon[30304]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 519 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:15.362 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:15 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1372957127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:26:15.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:26:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:15 smithi143 ceph-mon[34337]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 519 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:15 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1372957127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:26:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:15 smithi096 ceph-mon[34253]: pgmap v323: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 519 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:15 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1372957127' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:26:15.617 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:17.305 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:17 smithi059 ceph-mon[30304]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 471 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:17.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.316+0000 7fac0d436700 1 -- 172.21.15.59:0/2395302998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806bb40 msgr2=0x7fac08110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:17.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.316+0000 7fac0d436700 1 --2- 172.21.15.59:0/2395302998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806bb40 0x7fac08110330 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fac0000b0a0 tx=0x7fac0000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.316+0000 7fac0d436700 1 -- 172.21.15.59:0/2395302998 shutdown_connections 2024-05-08T05:26:17.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.316+0000 7fac0d436700 1 --2- 172.21.15.59:0/2395302998 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806bb40 0x7fac08110330 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.316+0000 7fac0d436700 1 --2- 172.21.15.59:0/2395302998 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac08109510 0x7fac081098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.316+0000 7fac0d436700 1 --2- 172.21.15.59:0/2395302998 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0806b180 0x7fac0806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.316+0000 7fac0d436700 1 -- 172.21.15.59:0/2395302998 >> 172.21.15.59:0/2395302998 conn(0x7fac08079350 msgr2=0x7fac08079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:17.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.316+0000 7fac0d436700 1 -- 172.21.15.59:0/2395302998 shutdown_connections 2024-05-08T05:26:17.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.316+0000 7fac0d436700 1 -- 172.21.15.59:0/2395302998 wait complete. 2024-05-08T05:26:17.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.317+0000 7fac0d436700 1 Processor -- start 2024-05-08T05:26:17.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.317+0000 7fac0d436700 1 -- start start 2024-05-08T05:26:17.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac0d436700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806b180 0x7fac0819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:17.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac0d436700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0806bb40 0x7fac0819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac0d436700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac08109510 0x7fac08199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac0d436700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fac08113670 con 0x7fac0806b180 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac0d436700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fac081134f0 con 0x7fac0806bb40 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac0d436700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fac081137f0 con 0x7fac08109510 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac06ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806b180 0x7fac0819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac06ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806b180 0x7fac0819f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:32796/0 (socket says 172.21.15.59:32796) 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac06ffd700 1 -- 172.21.15.59:0/481979515 learned_addr learned my addr 172.21.15.59:0/481979515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac067fc700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0806bb40 0x7fac0819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac077fe700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac08109510 0x7fac08199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac06ffd700 1 -- 172.21.15.59:0/481979515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac08109510 msgr2=0x7fac08199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac06ffd700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac08109510 0x7fac08199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac06ffd700 1 -- 172.21.15.59:0/481979515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0806bb40 msgr2=0x7fac0819fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac06ffd700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0806bb40 0x7fac0819fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac06ffd700 1 -- 172.21.15.59:0/481979515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac0819a0d0 con 0x7fac0806b180 2024-05-08T05:26:17.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac06ffd700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806b180 0x7fac0819f760 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fabf800d950 tx=0x7fabf800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:17.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fabeffff700 1 -- 172.21.15.59:0/481979515 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf8016070 con 0x7fac0806b180 2024-05-08T05:26:17.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.318+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac0819a3c0 con 0x7fac0806b180 2024-05-08T05:26:17.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.319+0000 7fabeffff700 1 -- 172.21.15.59:0/481979515 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabf8009df0 con 0x7fac0806b180 2024-05-08T05:26:17.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.319+0000 7fabeffff700 1 -- 172.21.15.59:0/481979515 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabf800f3c0 con 0x7fac0806b180 2024-05-08T05:26:17.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.319+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac081a6630 con 0x7fac0806b180 2024-05-08T05:26:17.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.319+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fac0806a190 con 0x7fac0806b180 2024-05-08T05:26:17.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.321+0000 7fabeffff700 1 -- 172.21.15.59:0/481979515 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fabf8019070 con 0x7fac0806b180 2024-05-08T05:26:17.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.322+0000 7fabeffff700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fabf00b2900 0x7fabf00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:17.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.322+0000 7fac067fc700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fabf00b2900 0x7fabf00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:17.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.322+0000 7fabeffff700 1 -- 172.21.15.59:0/481979515 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fabf80e2120 con 0x7fac0806b180 2024-05-08T05:26:17.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.322+0000 7fac067fc700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fabf00b2900 0x7fabf00b4dc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fac08079950 tx=0x7fabfc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:17.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.325+0000 7fabeffff700 1 -- 172.21.15.59:0/481979515 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fabf80a9a80 con 0x7fac0806b180 2024-05-08T05:26:17.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:17 smithi143 ceph-mon[34337]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 471 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:17 smithi096 ceph-mon[34253]: pgmap v324: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 471 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:17.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.541+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fac0802e510 con 0x7fac0806b180 2024-05-08T05:26:17.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.543+0000 7fabeffff700 1 -- 172.21.15.59:0/481979515 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24601 (secure 0 0 0) 0x7fabf80a9380 con 0x7fac0806b180 2024-05-08T05:26:17.545 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:26:17.545 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:26:05.137852+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:26:17.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.545+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fabf00b2900 msgr2=0x7fabf00b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:17.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.545+0000 7fac0d436700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fabf00b2900 0x7fabf00b4dc0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fac08079950 tx=0x7fabfc009260 comp rx=0 tx=0).stop 2024-05-08T05:26:17.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806b180 msgr2=0x7fac0819f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:17.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806b180 0x7fac0819f760 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fabf800d950 tx=0x7fabf800dc60 comp rx=0 tx=0).stop 2024-05-08T05:26:17.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 shutdown_connections 2024-05-08T05:26:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fac08109510 0x7fac08199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fac0806b180 0x7fac0819f760 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fabf00b2900 0x7fabf00b4dc0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 --2- 172.21.15.59:0/481979515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fac0806bb40 0x7fac0819fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 >> 172.21.15.59:0/481979515 conn(0x7fac08079350 msgr2=0x7fac081010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 shutdown_connections 2024-05-08T05:26:17.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:17.546+0000 7fac0d436700 1 -- 172.21.15.59:0/481979515 wait complete. 2024-05-08T05:26:18.200 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs set cephfs standby_count_wanted 0 2024-05-08T05:26:18.451 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:18.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:18 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/481979515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:26:18.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:18 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/481979515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:26:18.573 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:18 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/481979515' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:26:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:19 smithi143 ceph-mon[34337]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:19 smithi096 ceph-mon[34253]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:19 smithi059 ceph-mon[30304]: pgmap v325: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:26:20.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.216+0000 7f339e4bc700 1 -- 172.21.15.59:0/3914847556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398108850 msgr2=0x7f33981103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:20.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.216+0000 7f339e4bc700 1 --2- 172.21.15.59:0/3914847556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398108850 0x7f33981103d0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f339000b0d0 tx=0x7f339000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:26:20.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.216+0000 7f339e4bc700 1 -- 172.21.15.59:0/3914847556 shutdown_connections 2024-05-08T05:26:20.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.216+0000 7f339e4bc700 1 --2- 172.21.15.59:0/3914847556 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398108850 0x7f33981103d0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:20.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.216+0000 7f339e4bc700 1 --2- 172.21.15.59:0/3914847556 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3398103750 0x7f3398103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:20.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.216+0000 7f339e4bc700 1 --2- 172.21.15.59:0/3914847556 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398104100 0x7f3398108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:20.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.216+0000 7f339e4bc700 1 -- 172.21.15.59:0/3914847556 >> 172.21.15.59:0/3914847556 conn(0x7f339807a600 msgr2=0x7f339807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:20.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.217+0000 7f339e4bc700 1 -- 172.21.15.59:0/3914847556 shutdown_connections 2024-05-08T05:26:20.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.217+0000 7f339e4bc700 1 -- 172.21.15.59:0/3914847556 wait complete. 2024-05-08T05:26:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.217+0000 7f339e4bc700 1 Processor -- start 2024-05-08T05:26:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.217+0000 7f339e4bc700 1 -- start start 2024-05-08T05:26:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f339e4bc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398103750 0x7f339819f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f339e4bc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3398104100 0x7f339819fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f339e4bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398108850 0x7f3398199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f339e4bc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3398113620 con 0x7f3398103750 2024-05-08T05:26:20.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f339e4bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f33981134a0 con 0x7f3398108850 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f339e4bc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f33981137a0 con 0x7f3398104100 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f3397fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398103750 0x7f339819f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f3397fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398103750 0x7f339819f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47752/0 (socket says 172.21.15.59:47752) 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f3397fff700 1 -- 172.21.15.59:0/943739244 learned_addr learned my addr 172.21.15.59:0/943739244 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f33977fe700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3398104100 0x7f339819fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f3397fff700 1 -- 172.21.15.59:0/943739244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3398104100 msgr2=0x7f339819fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f3397fff700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3398104100 0x7f339819fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f3397fff700 1 -- 172.21.15.59:0/943739244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398108850 msgr2=0x7f3398199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f3397fff700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398108850 0x7f3398199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f3397fff700 1 -- 172.21.15.59:0/943739244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339819a020 con 0x7f3398103750 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f3397fff700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398103750 0x7f339819f6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f338800c9b0 tx=0x7f338800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f33957fa700 1 -- 172.21.15.59:0/943739244 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388007b00 con 0x7f3398103750 2024-05-08T05:26:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f33957fa700 1 -- 172.21.15.59:0/943739244 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3388007ca0 con 0x7f3398103750 2024-05-08T05:26:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339819a310 con 0x7f3398103750 2024-05-08T05:26:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.218+0000 7f33957fa700 1 -- 172.21.15.59:0/943739244 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3388018730 con 0x7f3398103750 2024-05-08T05:26:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.219+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33981a6520 con 0x7f3398103750 2024-05-08T05:26:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.220+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3398104be0 con 0x7f3398103750 2024-05-08T05:26:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.221+0000 7f33957fa700 1 -- 172.21.15.59:0/943739244 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f338800f450 con 0x7f3398103750 2024-05-08T05:26:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.222+0000 7f33957fa700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33800b2880 0x7f33800b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.222+0000 7f33977fe700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33800b2880 0x7f33800b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.222+0000 7f33957fa700 1 -- 172.21.15.59:0/943739244 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f33880e2900 con 0x7f3398103750 2024-05-08T05:26:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.223+0000 7f33977fe700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33800b2880 0x7f33800b4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f338c005fd0 tx=0x7f338c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:20.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.226+0000 7f33957fa700 1 -- 172.21.15.59:0/943739244 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3388014030 con 0x7f3398103750 2024-05-08T05:26:20.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:20.486+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f339802e690 con 0x7f3398103750 2024-05-08T05:26:21.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.186+0000 7f33957fa700 1 -- 172.21.15.59:0/943739244 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f33880aa350 con 0x7f3398103750 2024-05-08T05:26:21.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.189+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33800b2880 msgr2=0x7f33800b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:21.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.189+0000 7f339e4bc700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33800b2880 0x7f33800b4d40 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f338c005fd0 tx=0x7f338c005db0 comp rx=0 tx=0).stop 2024-05-08T05:26:21.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.189+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398103750 msgr2=0x7f339819f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:21.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.189+0000 7f339e4bc700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398103750 0x7f339819f6c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f338800c9b0 tx=0x7f338800cd70 comp rx=0 tx=0).stop 2024-05-08T05:26:21.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.190+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 shutdown_connections 2024-05-08T05:26:21.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.190+0000 7f339e4bc700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f33800b2880 0x7f33800b4d40 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:21.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.190+0000 7f339e4bc700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3398108850 0x7f3398199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:21.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.190+0000 7f339e4bc700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3398103750 0x7f339819f6c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:21.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.190+0000 7f339e4bc700 1 --2- 172.21.15.59:0/943739244 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3398104100 0x7f339819fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:21.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.190+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 >> 172.21.15.59:0/943739244 conn(0x7f339807a600 msgr2=0x7f33981014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:21.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.190+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 shutdown_connections 2024-05-08T05:26:21.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:21.190+0000 7f339e4bc700 1 -- 172.21.15.59:0/943739244 wait complete. 2024-05-08T05:26:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:21 smithi143 ceph-mon[34337]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T05:26:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:21 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/943739244' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:26:21.511 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:21 smithi059 ceph-mon[30304]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T05:26:21.512 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:21 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/943739244' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:26:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:21 smithi096 ceph-mon[34253]: pgmap v326: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 426 B/s wr, 0 op/s 2024-05-08T05:26:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:21 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/943739244' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-05-08T05:26:21.857 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'}} 2024-05-08T05:26:21.858 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs set cephfs max_mds 5 2024-05-08T05:26:22.122 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:22.474 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:22 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/943739244' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T05:26:22.474 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:22 smithi143 ceph-mon[34337]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:26:22.474 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:22 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:26:22.476 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:22 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/943739244' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T05:26:22.476 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:22 smithi096 ceph-mon[34253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:26:22.476 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:22 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:26:22.508 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:22 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/943739244' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-05-08T05:26:22.508 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:22 smithi059 ceph-mon[30304]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:26:22.508 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:22 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:26:23.312 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:23 smithi059 ceph-mon[30304]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:26:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:23 smithi143 ceph-mon[34337]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:26:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:23 smithi096 ceph-mon[34253]: pgmap v327: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:26:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/3820001479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 msgr2=0x7f9f28103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/3820001479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 0x7f9f28103b30 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c0099c0 tx=0x7f9f1c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:26:23.875 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/3820001479 shutdown_connections 2024-05-08T05:26:23.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/3820001479 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28108850 0x7f9f281103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:23.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/3820001479 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 0x7f9f28103b30 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:23.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/3820001479 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f28104100 0x7f9f28108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:23.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/3820001479 >> 172.21.15.59:0/3820001479 conn(0x7f9f2807a600 msgr2=0x7f9f2807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:23.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/3820001479 shutdown_connections 2024-05-08T05:26:23.876 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/3820001479 wait complete. 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.874+0000 7f9f2eb5a700 1 Processor -- start 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2eb5a700 1 -- start start 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2eb5a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 0x7f9f2819bda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2eb5a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28104100 0x7f9f2819c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2eb5a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f28108850 0x7f9f281a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2eb5a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f28113640 con 0x7f9f28103750 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2eb5a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281134c0 con 0x7f9f28104100 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2eb5a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9f281137c0 con 0x7f9f28108850 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2c8f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 0x7f9f2819bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2c8f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 0x7f9f2819bda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47774/0 (socket says 172.21.15.59:47774) 2024-05-08T05:26:23.878 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2c8f6700 1 -- 172.21.15.59:0/2856466617 learned_addr learned my addr 172.21.15.59:0/2856466617 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2d0f7700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f28108850 0x7f9f281a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f27fff700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28104100 0x7f9f2819c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2c8f6700 1 -- 172.21.15.59:0/2856466617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f28108850 msgr2=0x7f9f281a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2c8f6700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f28108850 0x7f9f281a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2c8f6700 1 -- 172.21.15.59:0/2856466617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28104100 msgr2=0x7f9f2819c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2c8f6700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28104100 0x7f9f2819c2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2c8f6700 1 -- 172.21.15.59:0/2856466617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f281a0d60 con 0x7f9f28103750 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.875+0000 7f9f2d0f7700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f28108850 0x7f9f281a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.876+0000 7f9f2c8f6700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 0x7f9f2819bda0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c006010 tx=0x7f9f1c005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.876+0000 7f9f25ffb700 1 -- 172.21.15.59:0/2856466617 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c026070 con 0x7f9f28103750 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.876+0000 7f9f25ffb700 1 -- 172.21.15.59:0/2856466617 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f1c003c70 con 0x7f9f28103750 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.876+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f281a0ff0 con 0x7f9f28103750 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.876+0000 7f9f25ffb700 1 -- 172.21.15.59:0/2856466617 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f1c00fbb0 con 0x7f9f28103750 2024-05-08T05:26:23.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.876+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f281a88d0 con 0x7f9f28103750 2024-05-08T05:26:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.877+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f2806a190 con 0x7f9f28103750 2024-05-08T05:26:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.879+0000 7f9f25ffb700 1 -- 172.21.15.59:0/2856466617 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7f9f1c029070 con 0x7f9f28103750 2024-05-08T05:26:23.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.879+0000 7f9f25ffb700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f180b2900 0x7f9f180b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.879+0000 7f9f27fff700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f180b2900 0x7f9f180b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.879+0000 7f9f25ffb700 1 -- 172.21.15.59:0/2856466617 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f9f1c0e32f0 con 0x7f9f28103750 2024-05-08T05:26:23.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.880+0000 7f9f27fff700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f180b2900 0x7f9f180b4dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f28102280 tx=0x7f9f14009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:23.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:23.883+0000 7f9f25ffb700 1 -- 172.21.15.59:0/2856466617 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f1c030020 con 0x7f9f28103750 2024-05-08T05:26:24.152 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.150+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f9f281a8aa0 con 0x7f9f28103750 2024-05-08T05:26:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.379+0000 7f9f25ffb700 1 -- 172.21.15.59:0/2856466617 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f9f1c013070 con 0x7f9f28103750 2024-05-08T05:26:24.383 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f180b2900 msgr2=0x7f9f180b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f180b2900 0x7f9f180b4dc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9f28102280 tx=0x7f9f14009260 comp rx=0 tx=0).stop 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 msgr2=0x7f9f2819bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 0x7f9f2819bda0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f9f1c006010 tx=0x7f9f1c005280 comp rx=0 tx=0).stop 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 shutdown_connections 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f28108850 0x7f9f281a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f28103750 0x7f9f2819bda0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f180b2900 0x7f9f180b4dc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 --2- 172.21.15.59:0/2856466617 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f28104100 0x7f9f2819c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 >> 172.21.15.59:0/2856466617 conn(0x7f9f2807a600 msgr2=0x7f9f281099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 shutdown_connections 2024-05-08T05:26:24.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:24.382+0000 7f9f2eb5a700 1 -- 172.21.15.59:0/2856466617 wait complete. 2024-05-08T05:26:24.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:24 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.628 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:24 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2856466617' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T05:26:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:24 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:24 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2856466617' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T05:26:24.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:24 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:24.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:24 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2856466617' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-05-08T05:26:25.330 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs set cephfs allow_standby_replay true 2024-05-08T05:26:25.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:25 smithi059 ceph-mon[30304]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:26:25.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:25 smithi059 ceph-mon[30304]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T05:26:25.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:25 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2856466617' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T05:26:25.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:25 smithi059 ceph-mon[30304]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T05:26:25.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:25 smithi059 ceph-mon[30304]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:26:25.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:25 smithi059 ceph-mon[30304]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-08T05:26:25.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:25 smithi059 ceph-mon[30304]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-08T05:26:25.625 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:25 smithi143 ceph-mon[34337]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:26:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:25 smithi143 ceph-mon[34337]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T05:26:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:25 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2856466617' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T05:26:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:25 smithi143 ceph-mon[34337]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T05:26:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:25 smithi143 ceph-mon[34337]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:26:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:25 smithi143 ceph-mon[34337]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-08T05:26:25.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:25 smithi143 ceph-mon[34337]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-08T05:26:25.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:25 smithi096 ceph-mon[34253]: pgmap v328: 129 pgs: 129 active+clean; 581 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 85 B/s wr, 0 op/s 2024-05-08T05:26:25.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:25 smithi096 ceph-mon[34253]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-05-08T05:26:25.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:25 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2856466617' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-05-08T05:26:25.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:25 smithi096 ceph-mon[34253]: daemon mds.h assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-05-08T05:26:25.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:25 smithi096 ceph-mon[34253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-05-08T05:26:25.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:25 smithi096 ceph-mon[34253]: fsmap cephfs:2 {0=b=up:active,1=h=up:creating} 10 up:standby 2024-05-08T05:26:25.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:25 smithi096 ceph-mon[34253]: daemon mds.h is now active in filesystem cephfs as rank 1 2024-05-08T05:26:26.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:26 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] up:active 2024-05-08T05:26:26.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:26 smithi143 ceph-mon[34337]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T05:26:26.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:26 smithi143 ceph-mon[34337]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-08T05:26:26.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:26 smithi143 ceph-mon[34337]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-08T05:26:26.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:26 smithi143 ceph-mon[34337]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T05:26:26.759 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:26 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] up:active 2024-05-08T05:26:26.760 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:26 smithi059 ceph-mon[30304]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T05:26:26.760 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:26 smithi059 ceph-mon[30304]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-08T05:26:26.760 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:26 smithi059 ceph-mon[30304]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-08T05:26:26.760 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:26 smithi059 ceph-mon[30304]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T05:26:26.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:26 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] up:active 2024-05-08T05:26:26.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:26 smithi096 ceph-mon[34253]: daemon mds.i assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-05-08T05:26:26.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:26 smithi096 ceph-mon[34253]: fsmap cephfs:2 {0=b=up:active,1=h=up:active} 10 up:standby 2024-05-08T05:26:26.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:26 smithi096 ceph-mon[34253]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:creating} 9 up:standby 2024-05-08T05:26:26.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:26 smithi096 ceph-mon[34253]: daemon mds.i is now active in filesystem cephfs as rank 2 2024-05-08T05:26:27.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.315+0000 7fea0f69c700 1 -- 172.21.15.59:0/1162585081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea081029e0 msgr2=0x7fea08102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:27.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.315+0000 7fea0f69c700 1 --2- 172.21.15.59:0/1162585081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea081029e0 0x7fea08102dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fea040099c0 tx=0x7fea04009cd0 comp rx=0 tx=0).stop 2024-05-08T05:26:27.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.316+0000 7fea0f69c700 1 -- 172.21.15.59:0/1162585081 shutdown_connections 2024-05-08T05:26:27.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.316+0000 7fea0f69c700 1 --2- 172.21.15.59:0/1162585081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea08107730 0x7fea0810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:27.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.316+0000 7fea0f69c700 1 --2- 172.21.15.59:0/1162585081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea081029e0 0x7fea08102dc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.316+0000 7fea0f69c700 1 --2- 172.21.15.59:0/1162585081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea08103390 0x7fea08107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.316+0000 7fea0f69c700 1 -- 172.21.15.59:0/1162585081 >> 172.21.15.59:0/1162585081 conn(0x7fea08079350 msgr2=0x7fea08079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.316+0000 7fea0f69c700 1 -- 172.21.15.59:0/1162585081 shutdown_connections 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.316+0000 7fea0f69c700 1 -- 172.21.15.59:0/1162585081 wait complete. 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.316+0000 7fea0f69c700 1 Processor -- start 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.316+0000 7fea0f69c700 1 -- start start 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0f69c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea081029e0 0x7fea0819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0f69c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea08103390 0x7fea0819af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0f69c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea08107730 0x7fea081a5a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0f69c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fea081125b0 con 0x7fea08103390 2024-05-08T05:26:27.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0f69c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fea08112430 con 0x7fea081029e0 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0f69c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fea08112730 con 0x7fea08107730 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0cc37700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea08103390 0x7fea0819af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0cc37700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea08103390 0x7fea0819af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47782/0 (socket says 172.21.15.59:47782) 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0cc37700 1 -- 172.21.15.59:0/326612044 learned_addr learned my addr 172.21.15.59:0/326612044 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0d438700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea081029e0 0x7fea0819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0dc39700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea08107730 0x7fea081a5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0d438700 1 -- 172.21.15.59:0/326612044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea08107730 msgr2=0x7fea081a5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0d438700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea08107730 0x7fea081a5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0d438700 1 -- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea08103390 msgr2=0x7fea0819af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0d438700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea08103390 0x7fea0819af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0d438700 1 -- 172.21.15.59:0/326612044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea081a5fb0 con 0x7fea081029e0 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0cc37700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea08103390 0x7fea0819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0dc39700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea08107730 0x7fea081a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:26:27.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fea0d438700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea081029e0 0x7fea0819a9f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea040098e0 tx=0x7fea04009550 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:27.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fe9fe7fc700 1 -- 172.21.15.59:0/326612044 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea04009e60 con 0x7fea081029e0 2024-05-08T05:26:27.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.317+0000 7fe9fe7fc700 1 -- 172.21.15.59:0/326612044 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea04023070 con 0x7fea081029e0 2024-05-08T05:26:27.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.318+0000 7fe9fe7fc700 1 -- 172.21.15.59:0/326612044 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea0401fce0 con 0x7fea081029e0 2024-05-08T05:26:27.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.318+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea081987e0 con 0x7fea081029e0 2024-05-08T05:26:27.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.318+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea08198c70 con 0x7fea081029e0 2024-05-08T05:26:27.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.319+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea0810a900 con 0x7fea081029e0 2024-05-08T05:26:27.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.320+0000 7fe9fe7fc700 1 -- 172.21.15.59:0/326612044 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fea0401f4d0 con 0x7fea081029e0 2024-05-08T05:26:27.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.320+0000 7fe9fe7fc700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe9f40b2900 0x7fe9f40b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:27.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.321+0000 7fea0cc37700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe9f40b2900 0x7fe9f40b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:27.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.321+0000 7fe9fe7fc700 1 -- 172.21.15.59:0/326612044 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fea040e7760 con 0x7fea081029e0 2024-05-08T05:26:27.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.321+0000 7fea0cc37700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe9f40b2900 0x7fe9f40b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea08102380 tx=0x7fe9f800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:27.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.326+0000 7fe9fe7fc700 1 -- 172.21.15.59:0/326612044 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fea04013070 con 0x7fea081029e0 2024-05-08T05:26:27.529 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b[64457]: 2024-05-08T05:26:27.251+0000 7f00205c7700 -1 mds.pinger is_rank_lagging: rank=1 was never sent ping request. 2024-05-08T05:26:27.529 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b[64457]: 2024-05-08T05:26:27.251+0000 7f00205c7700 -1 mds.pinger is_rank_lagging: rank=2 was never sent ping request. 2024-05-08T05:26:27.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-mon[34253]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T05:26:27.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] up:active 2024-05-08T05:26:27.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-mon[34253]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T05:26:27.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-mon[34253]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-08T05:26:27.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-mon[34253]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-08T05:26:27.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-mon[34253]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-08T05:26:27.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:27.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:27 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:27.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:27.597+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fea0806c160 con 0x7fea081029e0 2024-05-08T05:26:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:27 smithi143 ceph-mon[34337]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T05:26:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:27 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] up:active 2024-05-08T05:26:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:27 smithi143 ceph-mon[34337]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T05:26:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:27 smithi143 ceph-mon[34337]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-08T05:26:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:27 smithi143 ceph-mon[34337]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-08T05:26:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:27 smithi143 ceph-mon[34337]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-08T05:26:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:27 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:27.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:27 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:27 smithi059 ceph-mon[30304]: pgmap v329: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s wr, 3 op/s 2024-05-08T05:26:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:27 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] up:active 2024-05-08T05:26:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:27 smithi059 ceph-mon[30304]: daemon mds.c assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-05-08T05:26:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:27 smithi059 ceph-mon[30304]: fsmap cephfs:3 {0=b=up:active,1=h=up:active,2=i=up:active} 9 up:standby 2024-05-08T05:26:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:27 smithi059 ceph-mon[30304]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:creating} 8 up:standby 2024-05-08T05:26:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:27 smithi059 ceph-mon[30304]: daemon mds.c is now active in filesystem cephfs as rank 3 2024-05-08T05:26:27.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:27 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:27.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:27 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:28.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.412+0000 7fe9fe7fc700 1 -- 172.21.15.59:0/326612044 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v27) ==== 121+0+0 (secure 0 0 0) 0x7fea040af1b0 con 0x7fea081029e0 2024-05-08T05:26:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe9f40b2900 msgr2=0x7fe9f40b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe9f40b2900 0x7fe9f40b4dc0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fea08102380 tx=0x7fe9f800a380 comp rx=0 tx=0).stop 2024-05-08T05:26:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea081029e0 msgr2=0x7fea0819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea081029e0 0x7fea0819a9f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fea040098e0 tx=0x7fea04009550 comp rx=0 tx=0).stop 2024-05-08T05:26:28.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 shutdown_connections 2024-05-08T05:26:28.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fea08107730 0x7fea081a5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:28.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe9f40b2900 0x7fe9f40b4dc0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:28.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fea081029e0 0x7fea0819a9f0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:28.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 --2- 172.21.15.59:0/326612044 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fea08103390 0x7fea0819af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:28.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 >> 172.21.15.59:0/326612044 conn(0x7fea08079350 msgr2=0x7fea08100ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:28.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.416+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 shutdown_connections 2024-05-08T05:26:28.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:28.417+0000 7fea0f69c700 1 -- 172.21.15.59:0/326612044 wait complete. 2024-05-08T05:26:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] up:active 2024-05-08T05:26:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T05:26:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T05:26:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: Cluster is now healthy 2024-05-08T05:26:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-08T05:26:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-08T05:26:28.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-08T05:26:28.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/326612044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:26:28.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:26:28.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:26:28.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:26:28.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:28 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:28.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] up:active 2024-05-08T05:26:28.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T05:26:28.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T05:26:28.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: Cluster is now healthy 2024-05-08T05:26:28.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-08T05:26:28.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-08T05:26:28.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-08T05:26:28.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/326612044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:26:28.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:26:28.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:26:28.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:26:28.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:28 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:28.826 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] up:active 2024-05-08T05:26:28.826 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: daemon mds.e assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-05-08T05:26:28.826 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-05-08T05:26:28.826 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: Cluster is now healthy 2024-05-08T05:26:28.827 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: fsmap cephfs:4 {0=b=up:active,1=h=up:active,2=i=up:active,3=c=up:active} 8 up:standby 2024-05-08T05:26:28.827 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: fsmap cephfs:5 1 up:creating 7 up:standby 4 up:active 2024-05-08T05:26:28.827 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: daemon mds.e is now active in filesystem cephfs as rank 4 2024-05-08T05:26:28.827 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/326612044' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:26:28.827 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: from='client.? ' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-05-08T05:26:28.827 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:26:28.827 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:26:28.827 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:28 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:26:29.146 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs set cephfs session_timeout 300 2024-05-08T05:26:29.409 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:29.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:29 smithi059 ceph-mon[30304]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-08T05:26:29.662 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:29 smithi059 ceph-mon[30304]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T05:26:29.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:29 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] up:active 2024-05-08T05:26:29.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:29 smithi059 ceph-mon[30304]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-08T05:26:29.663 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:29 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:26:29.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:29 smithi143 ceph-mon[34337]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-08T05:26:29.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:29 smithi143 ceph-mon[34337]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T05:26:29.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:29 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] up:active 2024-05-08T05:26:29.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:29 smithi143 ceph-mon[34337]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-08T05:26:29.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:29 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:26:29.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:29 smithi096 ceph-mon[34253]: pgmap v330: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s wr, 5 op/s 2024-05-08T05:26:29.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:29 smithi096 ceph-mon[34253]: from='client.? ' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-05-08T05:26:29.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:29 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] up:active 2024-05-08T05:26:29.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:29 smithi096 ceph-mon[34253]: fsmap cephfs:5 7 up:standby 5 up:active 2024-05-08T05:26:29.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:29 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:26:31.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.038+0000 7fb7d7c89700 1 -- 172.21.15.59:0/2776083918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 msgr2=0x7fb7d01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.038+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/2776083918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 0x7fb7d01103d0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fb7c800b0d0 tx=0x7fb7c800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.038+0000 7fb7d7c89700 1 -- 172.21.15.59:0/2776083918 shutdown_connections 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.038+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/2776083918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 0x7fb7d01103d0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.038+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/2776083918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d0103750 0x7fb7d0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.038+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/2776083918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d0104100 0x7fb7d0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.038+0000 7fb7d7c89700 1 -- 172.21.15.59:0/2776083918 >> 172.21.15.59:0/2776083918 conn(0x7fb7d007a600 msgr2=0x7fb7d007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.038+0000 7fb7d7c89700 1 -- 172.21.15.59:0/2776083918 shutdown_connections 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.038+0000 7fb7d7c89700 1 -- 172.21.15.59:0/2776083918 wait complete. 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d7c89700 1 Processor -- start 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d7c89700 1 -- start start 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d7c89700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d0103750 0x7fb7d019f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d7c89700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d0104100 0x7fb7d019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d7c89700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 0x7fb7d01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d7c89700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d0113620 con 0x7fb7d0108850 2024-05-08T05:26:31.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d7c89700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d01134a0 con 0x7fb7d0104100 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d7c89700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d01137a0 con 0x7fb7d0103750 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d6226700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 0x7fb7d01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d6226700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 0x7fb7d01998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42688/0 (socket says 172.21.15.59:42688) 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d6226700 1 -- 172.21.15.59:0/1647567220 learned_addr learned my addr 172.21.15.59:0/1647567220 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d5224700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d0104100 0x7fb7d019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.039+0000 7fb7d5a25700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d0103750 0x7fb7d019f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7d6226700 1 -- 172.21.15.59:0/1647567220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d0103750 msgr2=0x7fb7d019f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7d6226700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d0103750 0x7fb7d019f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7d6226700 1 -- 172.21.15.59:0/1647567220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d0104100 msgr2=0x7fb7d019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7d6226700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d0104100 0x7fb7d019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7d6226700 1 -- 172.21.15.59:0/1647567220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d019a040 con 0x7fb7d0108850 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7d6226700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 0x7fb7d01998e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80095c0 tx=0x7fb7c8007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7c6ffd700 1 -- 172.21.15.59:0/1647567220 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c801f070 con 0x7fb7d0108850 2024-05-08T05:26:31.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7c6ffd700 1 -- 172.21.15.59:0/1647567220 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7c8007e20 con 0x7fb7d0108850 2024-05-08T05:26:31.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d019a2d0 con 0x7fb7d0108850 2024-05-08T05:26:31.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7c6ffd700 1 -- 172.21.15.59:0/1647567220 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7c801b4a0 con 0x7fb7d0108850 2024-05-08T05:26:31.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.040+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d01a65a0 con 0x7fb7d0108850 2024-05-08T05:26:31.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.041+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d0104be0 con 0x7fb7d0108850 2024-05-08T05:26:31.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.042+0000 7fb7c6ffd700 1 -- 172.21.15.59:0/1647567220 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fb7c8022070 con 0x7fb7d0108850 2024-05-08T05:26:31.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.043+0000 7fb7c6ffd700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7bc0b2880 0x7fb7bc0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:31.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.043+0000 7fb7d5a25700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7bc0b2880 0x7fb7bc0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:31.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.043+0000 7fb7c6ffd700 1 -- 172.21.15.59:0/1647567220 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fb7c800f020 con 0x7fb7d0108850 2024-05-08T05:26:31.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.044+0000 7fb7d5a25700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7bc0b2880 0x7fb7bc0b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0108350 tx=0x7fb7cc00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:31.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.047+0000 7fb7c6ffd700 1 -- 172.21.15.59:0/1647567220 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7c8029020 con 0x7fb7d0108850 2024-05-08T05:26:31.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.315+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fb7d002e510 con 0x7fb7d0108850 2024-05-08T05:26:31.420 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.419+0000 7fb7c6ffd700 1 -- 172.21.15.59:0/1647567220 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v29) ==== 115+0+0 (secure 0 0 0) 0x7fb7c80ab560 con 0x7fb7d0108850 2024-05-08T05:26:31.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7bc0b2880 msgr2=0x7fb7bc0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:31.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7bc0b2880 0x7fb7bc0b4d40 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb7d0108350 tx=0x7fb7cc00b410 comp rx=0 tx=0).stop 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 msgr2=0x7fb7d01998e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 0x7fb7d01998e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb7c80095c0 tx=0x7fb7c8007a20 comp rx=0 tx=0).stop 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 shutdown_connections 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d0108850 0x7fb7d01998e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d0103750 0x7fb7d019f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7bc0b2880 0x7fb7bc0b4d40 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 --2- 172.21.15.59:0/1647567220 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d0104100 0x7fb7d019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 >> 172.21.15.59:0/1647567220 conn(0x7fb7d007a600 msgr2=0x7fb7d0101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 shutdown_connections 2024-05-08T05:26:31.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:31.422+0000 7fb7d7c89700 1 -- 172.21.15.59:0/1647567220 wait complete. 2024-05-08T05:26:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:31 smithi059 ceph-mon[30304]: pgmap v331: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.2 KiB/s wr, 9 op/s 2024-05-08T05:26:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:31 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1647567220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:26:31.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:31 smithi143 ceph-mon[34337]: pgmap v331: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.2 KiB/s wr, 9 op/s 2024-05-08T05:26:31.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:31 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1647567220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:26:31.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:31 smithi096 ceph-mon[34253]: pgmap v331: 129 pgs: 129 active+clean; 584 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 170 B/s rd, 2.2 KiB/s wr, 9 op/s 2024-05-08T05:26:31.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:31 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1647567220' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-05-08T05:26:32.064 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--namespace-isolated --size 25000000000'} subvols for filesystem 'cephfs' 2024-05-08T05:26:32.064 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs subvolumegroup create cephfs qa 2024-05-08T05:26:32.318 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:32.731 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:32 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1647567220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T05:26:32.731 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:32 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:26:32.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:32 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1647567220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T05:26:32.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:32 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:26:32.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:32 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1647567220' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-05-08T05:26:32.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:32 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:26:33.494 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:33 smithi059 ceph-mon[30304]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-05-08T05:26:33.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:33 smithi143 ceph-mon[34337]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-05-08T05:26:33.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:33 smithi096 ceph-mon[34253]: pgmap v332: 129 pgs: 129 active+clean; 586 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 KiB/s rd, 3.9 KiB/s wr, 15 op/s 2024-05-08T05:26:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.042+0000 7fa05d7fb700 1 -- 172.21.15.59:0/2070687660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 msgr2=0x7fa058102a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.042+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/2070687660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 0x7fa058102a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fa0480099c0 tx=0x7fa048009cd0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.042+0000 7fa05d7fb700 1 -- 172.21.15.59:0/2070687660 shutdown_connections 2024-05-08T05:26:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.042+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/2070687660 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0581076e0 0x7fa05810f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.042+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/2070687660 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 0x7fa058102a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.042+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/2070687660 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa058103040 0x7fa058107000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.042+0000 7fa05d7fb700 1 -- 172.21.15.59:0/2070687660 >> 172.21.15.59:0/2070687660 conn(0x7fa0580783f0 msgr2=0x7fa058078800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:34.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.042+0000 7fa05d7fb700 1 -- 172.21.15.59:0/2070687660 shutdown_connections 2024-05-08T05:26:34.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.042+0000 7fa05d7fb700 1 -- 172.21.15.59:0/2070687660 wait complete. 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.043+0000 7fa05d7fb700 1 Processor -- start 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.043+0000 7fa05d7fb700 1 -- start start 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.043+0000 7fa05d7fb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 0x7fa05819e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.043+0000 7fa05d7fb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa058103040 0x7fa05819ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.043+0000 7fa05d7fb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0581076e0 0x7fa0581989b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.043+0000 7fa05d7fb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa058112670 con 0x7fa058102690 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.043+0000 7fa05d7fb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581124f0 con 0x7fa0581076e0 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.043+0000 7fa05d7fb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa0581127f0 con 0x7fa058103040 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa056ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 0x7fa05819e7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa056ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 0x7fa05819e7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42704/0 (socket says 172.21.15.59:42704) 2024-05-08T05:26:34.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa056ffd700 1 -- 172.21.15.59:0/4173690737 learned_addr learned my addr 172.21.15.59:0/4173690737 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa0567fc700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa058103040 0x7fa05819ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa056ffd700 1 -- 172.21.15.59:0/4173690737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa058103040 msgr2=0x7fa05819ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa056ffd700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa058103040 0x7fa05819ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa056ffd700 1 -- 172.21.15.59:0/4173690737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0581076e0 msgr2=0x7fa0581989b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa0577fe700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0581076e0 0x7fa0581989b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa056ffd700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0581076e0 0x7fa0581989b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa056ffd700 1 -- 172.21.15.59:0/4173690737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa058199110 con 0x7fa058102690 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa0567fc700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa058103040 0x7fa05819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa0577fe700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0581076e0 0x7fa0581989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa056ffd700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 0x7fa05819e7f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098e0 tx=0x7fa048003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa03ffff700 1 -- 172.21.15.59:0/4173690737 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048027070 con 0x7fa058102690 2024-05-08T05:26:34.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0581993a0 con 0x7fa058102690 2024-05-08T05:26:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa03ffff700 1 -- 172.21.15.59:0/4173690737 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0480040a0 con 0x7fa058102690 2024-05-08T05:26:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa03ffff700 1 -- 172.21.15.59:0/4173690737 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04800f460 con 0x7fa058102690 2024-05-08T05:26:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.044+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa05806bc10 con 0x7fa058102690 2024-05-08T05:26:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.045+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa05806a190 con 0x7fa058102690 2024-05-08T05:26:34.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.047+0000 7fa03ffff700 1 -- 172.21.15.59:0/4173690737 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 16) ==== 149652+0+0 (secure 0 0 0) 0x7fa04802a070 con 0x7fa058102690 2024-05-08T05:26:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.048+0000 7fa03ffff700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0400b2900 0x7fa0400b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.048+0000 7fa0567fc700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0400b2900 0x7fa0400b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.048+0000 7fa03ffff700 1 -- 172.21.15.59:0/4173690737 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fa0480e3e90 con 0x7fa058102690 2024-05-08T05:26:34.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.048+0000 7fa0567fc700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0400b2900 0x7fa0400b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa058102060 tx=0x7fa04c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:34.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.051+0000 7fa03ffff700 1 -- 172.21.15.59:0/4173690737 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa048013070 con 0x7fa058102690 2024-05-08T05:26:34.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.299+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fa05806bf30 con 0x7fa0400b2900 2024-05-08T05:26:34.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.333+0000 7fa03ffff700 1 -- 172.21.15.59:0/4173690737 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fa05806bf30 con 0x7fa0400b2900 2024-05-08T05:26:34.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0400b2900 msgr2=0x7fa0400b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:34.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0400b2900 0x7fa0400b4dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa058102060 tx=0x7fa04c008040 comp rx=0 tx=0).stop 2024-05-08T05:26:34.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 msgr2=0x7fa05819e7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:34.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 0x7fa05819e7f0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa0480098e0 tx=0x7fa048003680 comp rx=0 tx=0).stop 2024-05-08T05:26:34.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 shutdown_connections 2024-05-08T05:26:34.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0400b2900 0x7fa0400b4dc0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa0581076e0 0x7fa0581989b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa058102690 0x7fa05819e7f0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 --2- 172.21.15.59:0/4173690737 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa058103040 0x7fa05819ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:34.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 >> 172.21.15.59:0/4173690737 conn(0x7fa0580783f0 msgr2=0x7fa0581062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:34.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 shutdown_connections 2024-05-08T05:26:34.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:34.336+0000 7fa05d7fb700 1 -- 172.21.15.59:0/4173690737 wait complete. 2024-05-08T05:26:34.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:26:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:26:34.305+0000 7f4c6ced9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:26:34.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:26:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:26:34.305+0000 7f4c6ced9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:26:34.587 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:26:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:26:34.305+0000 7f4c6ced9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:26:34.587 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:26:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:26:34.305+0000 7f4c6ced9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:26:34.587 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:26:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:26:34.305+0000 7f4c6ced9700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:26:35.019 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-05-08T05:26:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:35.732 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:35 smithi059 ceph-mon[30304]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-08T05:26:35.732 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:35 smithi059 ceph-mon[30304]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:35.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:35 smithi143 ceph-mon[34337]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-08T05:26:35.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:35 smithi143 ceph-mon[34337]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:35.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:35 smithi096 ceph-mon[34253]: pgmap v333: 129 pgs: 129 active+clean; 587 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 4.0 KiB/s wr, 19 op/s 2024-05-08T05:26:35.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:35 smithi096 ceph-mon[34253]: from='client.14751 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.042+0000 7ff327588700 1 -- 172.21.15.59:0/3468622057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102fd0 msgr2=0x7ff320107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.042+0000 7ff327588700 1 --2- 172.21.15.59:0/3468622057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102fd0 0x7ff320107020 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7ff3100099c0 tx=0x7ff310009cd0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.042+0000 7ff327588700 1 -- 172.21.15.59:0/3468622057 shutdown_connections 2024-05-08T05:26:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.042+0000 7ff327588700 1 --2- 172.21.15.59:0/3468622057 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff320107700 0x7ff32010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.042+0000 7ff327588700 1 --2- 172.21.15.59:0/3468622057 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff320102620 0x7ff320102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.042+0000 7ff327588700 1 --2- 172.21.15.59:0/3468622057 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102fd0 0x7ff320107020 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.042+0000 7ff327588700 1 -- 172.21.15.59:0/3468622057 >> 172.21.15.59:0/3468622057 conn(0x7ff320079440 msgr2=0x7ff320079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.042+0000 7ff327588700 1 -- 172.21.15.59:0/3468622057 shutdown_connections 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.042+0000 7ff327588700 1 -- 172.21.15.59:0/3468622057 wait complete. 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff327588700 1 Processor -- start 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff327588700 1 -- start start 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff327588700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102620 0x7ff32019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff327588700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff320102fd0 0x7ff32019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff327588700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff320107700 0x7ff320199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff327588700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff320112570 con 0x7ff320102620 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff327588700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff3201123f0 con 0x7ff320107700 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff327588700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff3201126f0 con 0x7ff320102fd0 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff325324700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102620 0x7ff32019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff325324700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102620 0x7ff32019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42726/0 (socket says 172.21.15.59:42726) 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff325324700 1 -- 172.21.15.59:0/3592026890 learned_addr learned my addr 172.21.15.59:0/3592026890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:37.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff325b25700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff320107700 0x7ff320199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.043+0000 7ff324b23700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff320102fd0 0x7ff32019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff325324700 1 -- 172.21.15.59:0/3592026890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff320102fd0 msgr2=0x7ff32019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff325324700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff320102fd0 0x7ff32019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff325324700 1 -- 172.21.15.59:0/3592026890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff320107700 msgr2=0x7ff320199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff325324700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff320107700 0x7ff320199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff325324700 1 -- 172.21.15.59:0/3592026890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff32019a070 con 0x7ff320102620 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff324b23700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff320102fd0 0x7ff32019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff325324700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102620 0x7ff32019f750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00ef10 tx=0x7ff31c00c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff3167fc700 1 -- 172.21.15.59:0/3592026890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c00ca70 con 0x7ff320102620 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff32019a300 con 0x7ff320102620 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff3167fc700 1 -- 172.21.15.59:0/3592026890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff31c019070 con 0x7ff320102620 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff3167fc700 1 -- 172.21.15.59:0/3592026890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff31c014480 con 0x7ff320102620 2024-05-08T05:26:37.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.044+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3201a6620 con 0x7ff320102620 2024-05-08T05:26:37.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.045+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff32006a190 con 0x7ff320102620 2024-05-08T05:26:37.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.047+0000 7ff3167fc700 1 -- 172.21.15.59:0/3592026890 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ff31c012050 con 0x7ff320102620 2024-05-08T05:26:37.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.047+0000 7ff3167fc700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff30c0b2910 0x7ff30c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:37.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.047+0000 7ff324b23700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff30c0b2910 0x7ff30c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:37.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.047+0000 7ff3167fc700 1 -- 172.21.15.59:0/3592026890 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7ff31c0e2e30 con 0x7ff320102620 2024-05-08T05:26:37.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.048+0000 7ff324b23700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff30c0b2910 0x7ff30c0b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff310006010 tx=0x7ff310005ea0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:37.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.051+0000 7ff3167fc700 1 -- 172.21.15.59:0/3592026890 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff31c010040 con 0x7ff320102620 2024-05-08T05:26:37.300 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.300+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7ff3201001b0 con 0x7ff30c0b2910 2024-05-08T05:26:37.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.326+0000 7ff3167fc700 1 -- 172.21.15.59:0/3592026890 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7ff3201001b0 con 0x7ff30c0b2910 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff30c0b2910 msgr2=0x7ff30c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff30c0b2910 0x7ff30c0b4dd0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ff310006010 tx=0x7ff310005ea0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102620 msgr2=0x7ff32019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102620 0x7ff32019f750 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7ff31c00ef10 tx=0x7ff31c00c3a0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 shutdown_connections 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff30c0b2910 0x7ff30c0b4dd0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff320107700 0x7ff320199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff320102620 0x7ff32019f750 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 --2- 172.21.15.59:0/3592026890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff320102fd0 0x7ff32019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.328+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 >> 172.21.15.59:0/3592026890 conn(0x7ff320079440 msgr2=0x7ff320106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.329+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 shutdown_connections 2024-05-08T05:26:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:37.329+0000 7ff327588700 1 -- 172.21.15.59:0/3592026890 wait complete. 2024-05-08T05:26:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:37 smithi059 ceph-mon[30304]: pgmap v334: 129 pgs: 129 active+clean; 588 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.3 KiB/s wr, 21 op/s 2024-05-08T05:26:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:37 smithi059 ceph-mon[30304]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T05:26:37.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:37 smithi143 ceph-mon[34337]: pgmap v334: 129 pgs: 129 active+clean; 588 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.3 KiB/s wr, 21 op/s 2024-05-08T05:26:37.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:37 smithi143 ceph-mon[34337]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T05:26:37.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:37 smithi096 ceph-mon[34253]: pgmap v334: 129 pgs: 129 active+clean; 588 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 4.3 KiB/s wr, 21 op/s 2024-05-08T05:26:37.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:37 smithi096 ceph-mon[34253]: mgrmap e17: x(active, since 8m), standbys: z, y 2024-05-08T05:26:38.034 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-05-08T05:26:38.288 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:38.733 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:38 smithi059 ceph-mon[30304]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:38.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:38 smithi143 ceph-mon[34337]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:38.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:38 smithi096 ceph-mon[34253]: from='client.14763 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:39.713 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:39 smithi059 ceph-mon[30304]: pgmap v335: 129 pgs: 129 active+clean; 588 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 19 op/s 2024-05-08T05:26:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:39 smithi143 ceph-mon[34337]: pgmap v335: 129 pgs: 129 active+clean; 588 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 19 op/s 2024-05-08T05:26:39.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:39 smithi096 ceph-mon[34253]: pgmap v335: 129 pgs: 129 active+clean; 588 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 3.2 KiB/s wr, 19 op/s 2024-05-08T05:26:39.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.974+0000 7f29fb813700 1 -- 172.21.15.59:0/2639765367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f4109530 msgr2=0x7f29f4109910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:39.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.974+0000 7f29fb813700 1 --2- 172.21.15.59:0/2639765367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f4109530 0x7f29f4109910 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f29f000b0a0 tx=0x7f29f000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:26:39.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.974+0000 7f29fb813700 1 -- 172.21.15.59:0/2639765367 shutdown_connections 2024-05-08T05:26:39.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.974+0000 7f29fb813700 1 --2- 172.21.15.59:0/2639765367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29f406c530 0x7f29f4110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:39.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.974+0000 7f29fb813700 1 --2- 172.21.15.59:0/2639765367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f4109530 0x7f29f4109910 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:39.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.974+0000 7f29fb813700 1 --2- 172.21.15.59:0/2639765367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f406bb70 0x7f29f406bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:39.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.974+0000 7f29fb813700 1 -- 172.21.15.59:0/2639765367 >> 172.21.15.59:0/2639765367 conn(0x7f29f4079350 msgr2=0x7f29f4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:39.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.974+0000 7f29fb813700 1 -- 172.21.15.59:0/2639765367 shutdown_connections 2024-05-08T05:26:39.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.974+0000 7f29fb813700 1 -- 172.21.15.59:0/2639765367 wait complete. 2024-05-08T05:26:39.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29fb813700 1 Processor -- start 2024-05-08T05:26:39.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29fb813700 1 -- start start 2024-05-08T05:26:39.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29fb813700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f406bb70 0x7f29f419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:39.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29fb813700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29f406c530 0x7f29f419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:39.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29fb813700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f4109530 0x7f29f4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:39.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29fb813700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f29f4113640 con 0x7f29f406bb70 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29fb813700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41134c0 con 0x7f29f406c530 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29fb813700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f29f41137c0 con 0x7f29f4109530 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29f95af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f406bb70 0x7f29f419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29f95af700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f406bb70 0x7f29f419f740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37316/0 (socket says 172.21.15.59:37316) 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29f95af700 1 -- 172.21.15.59:0/2828762746 learned_addr learned my addr 172.21.15.59:0/2828762746 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.975+0000 7f29f8dae700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29f406c530 0x7f29f419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29f9db0700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f4109530 0x7f29f4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29f95af700 1 -- 172.21.15.59:0/2828762746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f4109530 msgr2=0x7f29f4199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29f95af700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f4109530 0x7f29f4199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29f95af700 1 -- 172.21.15.59:0/2828762746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29f406c530 msgr2=0x7f29f419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29f95af700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29f406c530 0x7f29f419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29f95af700 1 -- 172.21.15.59:0/2828762746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29f419a060 con 0x7f29f406bb70 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29f95af700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f406bb70 0x7f29f419f740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f29f000b070 tx=0x7f29f0004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:39.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29ea7fc700 1 -- 172.21.15.59:0/2828762746 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0021070 con 0x7f29f406bb70 2024-05-08T05:26:39.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29ea7fc700 1 -- 172.21.15.59:0/2828762746 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29f0007e30 con 0x7f29f406bb70 2024-05-08T05:26:39.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29ea7fc700 1 -- 172.21.15.59:0/2828762746 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29f0004840 con 0x7f29f406bb70 2024-05-08T05:26:39.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29f419a2f0 con 0x7f29f406bb70 2024-05-08T05:26:39.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.976+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29f41a6610 con 0x7f29f406bb70 2024-05-08T05:26:39.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.977+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29f410ba10 con 0x7f29f406bb70 2024-05-08T05:26:39.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.979+0000 7f29ea7fc700 1 -- 172.21.15.59:0/2828762746 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f29f0007980 con 0x7f29f406bb70 2024-05-08T05:26:39.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.979+0000 7f29ea7fc700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f29e00b2910 0x7f29e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:39.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.980+0000 7f29f8dae700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f29e00b2910 0x7f29e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:39.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.980+0000 7f29ea7fc700 1 -- 172.21.15.59:0/2828762746 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f29f000f020 con 0x7f29f406bb70 2024-05-08T05:26:39.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.980+0000 7f29f8dae700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f29e00b2910 0x7f29e00b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29f40799c0 tx=0x7f29e400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:39.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:39.983+0000 7f29ea7fc700 1 -- 172.21.15.59:0/2828762746 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29f002a020 con 0x7f29f406bb70 2024-05-08T05:26:40.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.236+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7f29f40638e0 con 0x7f29e00b2910 2024-05-08T05:26:40.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.242+0000 7f29ea7fc700 1 -- 172.21.15.59:0/2828762746 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f29f40638e0 con 0x7f29e00b2910 2024-05-08T05:26:40.242 INFO:teuthology.orchestra.run.smithi059.stdout:/volumes/qa/sv_0/1934bf9d-7739-4db3-b0e2-9ba41c8ec602 2024-05-08T05:26:40.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f29e00b2910 msgr2=0x7f29e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f29e00b2910 0x7f29e00b4dd0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f29f40799c0 tx=0x7f29e400a380 comp rx=0 tx=0).stop 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f406bb70 msgr2=0x7f29f419f740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f406bb70 0x7f29f419f740 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f29f000b070 tx=0x7f29f0004d00 comp rx=0 tx=0).stop 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 shutdown_connections 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f29f4109530 0x7f29f4199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f29f406bb70 0x7f29f419f740 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f29e00b2910 0x7f29e00b4dd0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 --2- 172.21.15.59:0/2828762746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f29f406c530 0x7f29f419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 >> 172.21.15.59:0/2828762746 conn(0x7f29f4079350 msgr2=0x7f29f4101400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 shutdown_connections 2024-05-08T05:26:40.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:40.244+0000 7f29fb813700 1 -- 172.21.15.59:0/2828762746 wait complete. 2024-05-08T05:26:40.909 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-05-08T05:26:41.166 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:41 smithi059 ceph-mon[30304]: pgmap v336: 129 pgs: 129 active+clean; 608 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.6 KiB/s wr, 20 op/s 2024-05-08T05:26:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:41 smithi059 ceph-mon[30304]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:41.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:41 smithi143 ceph-mon[34337]: pgmap v336: 129 pgs: 129 active+clean; 608 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.6 KiB/s wr, 20 op/s 2024-05-08T05:26:41.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:41 smithi143 ceph-mon[34337]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:41.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:41 smithi096 ceph-mon[34253]: pgmap v336: 129 pgs: 129 active+clean; 608 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.6 KiB/s wr, 20 op/s 2024-05-08T05:26:41.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:41 smithi096 ceph-mon[34253]: from='client.14769 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.901+0000 7f9f48a7a700 1 -- 172.21.15.59:0/1718411317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 msgr2=0x7f9f4406b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.901+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/1718411317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 0x7f9f4406b600 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f9f38009990 tx=0x7f9f38009ca0 comp rx=0 tx=0).stop 2024-05-08T05:26:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.901+0000 7f9f48a7a700 1 -- 172.21.15.59:0/1718411317 shutdown_connections 2024-05-08T05:26:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.901+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/1718411317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f4406bb40 0x7f9f44110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.901+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/1718411317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f44109510 0x7f9f441098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.901+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/1718411317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 0x7f9f4406b600 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.901+0000 7f9f48a7a700 1 -- 172.21.15.59:0/1718411317 >> 172.21.15.59:0/1718411317 conn(0x7f9f44079350 msgr2=0x7f9f44079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.901+0000 7f9f48a7a700 1 -- 172.21.15.59:0/1718411317 shutdown_connections 2024-05-08T05:26:42.905 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.901+0000 7f9f48a7a700 1 -- 172.21.15.59:0/1718411317 wait complete. 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.902+0000 7f9f48a7a700 1 Processor -- start 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.902+0000 7f9f48a7a700 1 -- start start 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.902+0000 7f9f48a7a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 0x7f9f4419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.902+0000 7f9f48a7a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4406bb40 0x7f9f4419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.902+0000 7f9f48a7a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f44109510 0x7f9f44199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.902+0000 7f9f48a7a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9f44113640 con 0x7f9f4406b180 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.902+0000 7f9f48a7a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441134c0 con 0x7f9f4406bb40 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.902+0000 7f9f48a7a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9f441137c0 con 0x7f9f44109510 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f4259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 0x7f9f4419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f4259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 0x7f9f4419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37328/0 (socket says 172.21.15.59:37328) 2024-05-08T05:26:42.906 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f4259c700 1 -- 172.21.15.59:0/2768800398 learned_addr learned my addr 172.21.15.59:0/2768800398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f42d9d700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f44109510 0x7f9f44199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f41d9b700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4406bb40 0x7f9f4419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f4259c700 1 -- 172.21.15.59:0/2768800398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f44109510 msgr2=0x7f9f44199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f4259c700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f44109510 0x7f9f44199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f4259c700 1 -- 172.21.15.59:0/2768800398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4406bb40 msgr2=0x7f9f4419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f4259c700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4406bb40 0x7f9f4419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f4259c700 1 -- 172.21.15.59:0/2768800398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f4419a0e0 con 0x7f9f4406b180 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f4259c700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 0x7f9f4419f7c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400d950 tx=0x7f9f3400dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f337fe700 1 -- 172.21.15.59:0/2768800398 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f34016070 con 0x7f9f4406b180 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f4419a3d0 con 0x7f9f4406b180 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f337fe700 1 -- 172.21.15.59:0/2768800398 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f34009df0 con 0x7f9f4406b180 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f441a6690 con 0x7f9f4406b180 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.903+0000 7f9f337fe700 1 -- 172.21.15.59:0/2768800398 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f3400f3c0 con 0x7f9f4406b180 2024-05-08T05:26:42.907 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.904+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9f4406a190 con 0x7f9f4406b180 2024-05-08T05:26:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.906+0000 7f9f337fe700 1 -- 172.21.15.59:0/2768800398 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9f34019070 con 0x7f9f4406b180 2024-05-08T05:26:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.906+0000 7f9f337fe700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f2c0b2910 0x7f9f2c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.907+0000 7f9f41d9b700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f2c0b2910 0x7f9f2c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:42.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.907+0000 7f9f337fe700 1 -- 172.21.15.59:0/2768800398 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f9f340e2290 con 0x7f9f4406b180 2024-05-08T05:26:42.909 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.907+0000 7f9f41d9b700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f2c0b2910 0x7f9f2c0b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f38006010 tx=0x7f9f3801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:42.910 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:42.910+0000 7f9f337fe700 1 -- 172.21.15.59:0/2768800398 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f9f340a9af0 con 0x7f9f4406b180 2024-05-08T05:26:43.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.160+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f9f440638e0 con 0x7f9f2c0b2910 2024-05-08T05:26:43.187 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.187+0000 7f9f337fe700 1 -- 172.21.15.59:0/2768800398 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f9f440638e0 con 0x7f9f2c0b2910 2024-05-08T05:26:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f2c0b2910 msgr2=0x7f9f2c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f2c0b2910 0x7f9f2c0b4dd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f9f38006010 tx=0x7f9f3801a040 comp rx=0 tx=0).stop 2024-05-08T05:26:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 msgr2=0x7f9f4419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 0x7f9f4419f7c0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f9f3400d950 tx=0x7f9f3400dc60 comp rx=0 tx=0).stop 2024-05-08T05:26:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 shutdown_connections 2024-05-08T05:26:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9f44109510 0x7f9f44199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9f4406b180 0x7f9f4419f7c0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9f2c0b2910 0x7f9f2c0b4dd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:43.190 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 --2- 172.21.15.59:0/2768800398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9f4406bb40 0x7f9f4419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 >> 172.21.15.59:0/2768800398 conn(0x7f9f44079350 msgr2=0x7f9f441010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 shutdown_connections 2024-05-08T05:26:43.191 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:43.189+0000 7f9f48a7a700 1 -- 172.21.15.59:0/2768800398 wait complete. 2024-05-08T05:26:43.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:43 smithi143 ceph-mon[34337]: pgmap v337: 129 pgs: 129 active+clean; 608 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.4 KiB/s wr, 18 op/s 2024-05-08T05:26:43.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:43 smithi096 ceph-mon[34253]: pgmap v337: 129 pgs: 129 active+clean; 608 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.4 KiB/s wr, 18 op/s 2024-05-08T05:26:43.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:43 smithi059 ceph-mon[30304]: pgmap v337: 129 pgs: 129 active+clean; 608 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.4 KiB/s wr, 18 op/s 2024-05-08T05:26:43.995 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-05-08T05:26:44.251 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:44.626 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:44 smithi059 ceph-mon[30304]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:44.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:44 smithi143 ceph-mon[34337]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:44.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:44 smithi096 ceph-mon[34253]: from='client.14775 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:45.711 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:45 smithi059 ceph-mon[30304]: pgmap v338: 129 pgs: 129 active+clean; 628 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.7 KiB/s wr, 15 op/s 2024-05-08T05:26:45.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:45 smithi143 ceph-mon[34337]: pgmap v338: 129 pgs: 129 active+clean; 628 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.7 KiB/s wr, 15 op/s 2024-05-08T05:26:45.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:45 smithi096 ceph-mon[34253]: pgmap v338: 129 pgs: 129 active+clean; 628 KiB data, 348 MiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 4.7 KiB/s wr, 15 op/s 2024-05-08T05:26:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.097+0000 7f88d57ae700 1 -- 172.21.15.59:0/3482128725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d0104130 msgr2=0x7f88d0108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.097+0000 7f88d57ae700 1 --2- 172.21.15.59:0/3482128725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d0104130 0x7f88d0108140 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f88c40099c0 tx=0x7f88c4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.097+0000 7f88d57ae700 1 -- 172.21.15.59:0/3482128725 shutdown_connections 2024-05-08T05:26:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.097+0000 7f88d57ae700 1 --2- 172.21.15.59:0/3482128725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88d0108870 0x7f88d01103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.097+0000 7f88d57ae700 1 --2- 172.21.15.59:0/3482128725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d0103780 0x7f88d0103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.097+0000 7f88d57ae700 1 --2- 172.21.15.59:0/3482128725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d0104130 0x7f88d0108140 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.097+0000 7f88d57ae700 1 -- 172.21.15.59:0/3482128725 >> 172.21.15.59:0/3482128725 conn(0x7f88d007a610 msgr2=0x7f88d007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:46.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.097+0000 7f88d57ae700 1 -- 172.21.15.59:0/3482128725 shutdown_connections 2024-05-08T05:26:46.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.097+0000 7f88d57ae700 1 -- 172.21.15.59:0/3482128725 wait complete. 2024-05-08T05:26:46.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88d57ae700 1 Processor -- start 2024-05-08T05:26:46.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88d57ae700 1 -- start start 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88d57ae700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88d0103780 0x7f88d00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88d57ae700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d0104130 0x7f88d0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88d57ae700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d0108870 0x7f88d0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88d57ae700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f88d01136d0 con 0x7f88d0108870 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88d57ae700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0113550 con 0x7f88d0103780 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88d57ae700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f88d0113850 con 0x7f88d0104130 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88cf7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d0108870 0x7f88d0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88ceffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88d0103780 0x7f88d00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88cf7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d0108870 0x7f88d0076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37344/0 (socket says 172.21.15.59:37344) 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88ce7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d0104130 0x7f88d0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88ceffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88d0103780 0x7f88d00757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:45332/0 (socket says 172.21.15.59:45332) 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.098+0000 7f88cf7fe700 1 -- 172.21.15.59:0/1287911964 learned_addr learned my addr 172.21.15.59:0/1287911964 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:46.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88ce7fc700 1 -- 172.21.15.59:0/1287911964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88d0103780 msgr2=0x7f88d00757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88ce7fc700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88d0103780 0x7f88d00757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88ce7fc700 1 -- 172.21.15.59:0/1287911964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d0108870 msgr2=0x7f88d0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88ce7fc700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d0108870 0x7f88d0076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88ce7fc700 1 -- 172.21.15.59:0/1287911964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88d01a80b0 con 0x7f88d0104130 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88ce7fc700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d0104130 0x7f88d0075ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009f50 tx=0x7f88c40047b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88b7fff700 1 -- 172.21.15.59:0/1287911964 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4027070 con 0x7f88d0104130 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88d01a8340 con 0x7f88d0104130 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88b7fff700 1 -- 172.21.15.59:0/1287911964 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88c4003f20 con 0x7f88d0104130 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88b7fff700 1 -- 172.21.15.59:0/1287911964 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88c4022bd0 con 0x7f88d0104130 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.099+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88d01a8790 con 0x7f88d0104130 2024-05-08T05:26:46.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.100+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88d006a190 con 0x7f88d0104130 2024-05-08T05:26:46.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.101+0000 7f88b7fff700 1 -- 172.21.15.59:0/1287911964 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f88c402a070 con 0x7f88d0104130 2024-05-08T05:26:46.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.102+0000 7f88b7fff700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88b80b2910 0x7f88b80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:46.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.102+0000 7f88ceffd700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88b80b2910 0x7f88b80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:46.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.102+0000 7f88b7fff700 1 -- 172.21.15.59:0/1287911964 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f88c40e38f0 con 0x7f88d0104130 2024-05-08T05:26:46.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.103+0000 7f88ceffd700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88b80b2910 0x7f88b80b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88c0004630 tx=0x7f88c000b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:46.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.106+0000 7f88b7fff700 1 -- 172.21.15.59:0/1287911964 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f88c4031070 con 0x7f88d0104130 2024-05-08T05:26:46.360 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.359+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f88d002e5f0 con 0x7f88b80b2910 2024-05-08T05:26:46.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.364+0000 7f88b7fff700 1 -- 172.21.15.59:0/1287911964 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f88d002e5f0 con 0x7f88b80b2910 2024-05-08T05:26:46.364 INFO:teuthology.orchestra.run.smithi059.stdout:/volumes/qa/sv_1/5ff8d981-f338-4914-9e21-f8b83e78f038 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88b80b2910 msgr2=0x7f88b80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88b80b2910 0x7f88b80b4dd0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f88c0004630 tx=0x7f88c000b3c0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d0104130 msgr2=0x7f88d0075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d0104130 0x7f88d0075ce0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f88c4009f50 tx=0x7f88c40047b0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 shutdown_connections 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f88d0108870 0x7f88d0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f88b80b2910 0x7f88b80b4dd0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f88d0103780 0x7f88d00757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 --2- 172.21.15.59:0/1287911964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f88d0104130 0x7f88d0075ce0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:46.367 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.366+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 >> 172.21.15.59:0/1287911964 conn(0x7f88d007a610 msgr2=0x7f88d01099c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:46.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.367+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 shutdown_connections 2024-05-08T05:26:46.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:46.367+0000 7f88d57ae700 1 -- 172.21.15.59:0/1287911964 wait complete. 2024-05-08T05:26:47.122 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:26:47.379 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:47.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:47 smithi143 ceph-mon[34337]: pgmap v339: 129 pgs: 129 active+clean; 628 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-05-08T05:26:47.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:47 smithi143 ceph-mon[34337]: from='client.24580 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:47.770 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:47 smithi059 ceph-mon[30304]: pgmap v339: 129 pgs: 129 active+clean; 628 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-05-08T05:26:47.770 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:47 smithi059 ceph-mon[30304]: from='client.24580 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:47.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:47 smithi096 ceph-mon[34253]: pgmap v339: 129 pgs: 129 active+clean; 628 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 4.7 KiB/s wr, 12 op/s 2024-05-08T05:26:47.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:47 smithi096 ceph-mon[34253]: from='client.24580 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:26:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.168+0000 7f3dd4935700 1 -- 172.21.15.59:0/1123077582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd006c590 msgr2=0x7f3dd0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.168+0000 7f3dd4935700 1 --2- 172.21.15.59:0/1123077582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd006c590 0x7f3dd0110330 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800b0a0 tx=0x7f3dc800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.168+0000 7f3dd4935700 1 -- 172.21.15.59:0/1123077582 shutdown_connections 2024-05-08T05:26:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.168+0000 7f3dd4935700 1 --2- 172.21.15.59:0/1123077582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd006c590 0x7f3dd0110330 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.168+0000 7f3dd4935700 1 --2- 172.21.15.59:0/1123077582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dd0109530 0x7f3dd0109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.168+0000 7f3dd4935700 1 --2- 172.21.15.59:0/1123077582 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd006bbd0 0x7f3dd006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.168+0000 7f3dd4935700 1 -- 172.21.15.59:0/1123077582 >> 172.21.15.59:0/1123077582 conn(0x7f3dd0079340 msgr2=0x7f3dd0079750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.168+0000 7f3dd4935700 1 -- 172.21.15.59:0/1123077582 shutdown_connections 2024-05-08T05:26:49.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.168+0000 7f3dd4935700 1 -- 172.21.15.59:0/1123077582 wait complete. 2024-05-08T05:26:49.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dd4935700 1 Processor -- start 2024-05-08T05:26:49.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dd4935700 1 -- start start 2024-05-08T05:26:49.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dd4935700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd006bbd0 0x7f3dd01024a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dd4935700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dd006c590 0x7f3dd01029e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dd4935700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd0109530 0x7f3dd0102f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dd4935700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd0113650 con 0x7f3dd0109530 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dd4935700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd01134d0 con 0x7f3dd006c590 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dd4935700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3dd01137d0 con 0x7f3dd006bbd0 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dced9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd0109530 0x7f3dd0102f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dced9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd0109530 0x7f3dd0102f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35424/0 (socket says 172.21.15.59:35424) 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.169+0000 7f3dced9d700 1 -- 172.21.15.59:0/3620481789 learned_addr learned my addr 172.21.15.59:0/3620481789 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dcdd9b700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dd006c590 0x7f3dd01029e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dce59c700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd006bbd0 0x7f3dd01024a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dced9d700 1 -- 172.21.15.59:0/3620481789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd006bbd0 msgr2=0x7f3dd01024a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dced9d700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd006bbd0 0x7f3dd01024a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dced9d700 1 -- 172.21.15.59:0/3620481789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dd006c590 msgr2=0x7f3dd01029e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dced9d700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dd006c590 0x7f3dd01029e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dced9d700 1 -- 172.21.15.59:0/3620481789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3dd01a8130 con 0x7f3dd0109530 2024-05-08T05:26:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dced9d700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd0109530 0x7f3dd0102f20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800b830 tx=0x7f3dc8004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dbf7fe700 1 -- 172.21.15.59:0/3620481789 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8021070 con 0x7f3dd0109530 2024-05-08T05:26:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3dd01a83c0 con 0x7f3dd0109530 2024-05-08T05:26:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dbf7fe700 1 -- 172.21.15.59:0/3620481789 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3dc8007b80 con 0x7f3dd0109530 2024-05-08T05:26:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dbf7fe700 1 -- 172.21.15.59:0/3620481789 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3dc80052a0 con 0x7f3dd0109530 2024-05-08T05:26:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.170+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3dd01a8870 con 0x7f3dd0109530 2024-05-08T05:26:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.171+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3dd0104bd0 con 0x7f3dd0109530 2024-05-08T05:26:49.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.173+0000 7f3dbf7fe700 1 -- 172.21.15.59:0/3620481789 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f3dc8024070 con 0x7f3dd0109530 2024-05-08T05:26:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.173+0000 7f3dbf7fe700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3db80b2910 0x7f3db80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.173+0000 7f3dbf7fe700 1 -- 172.21.15.59:0/3620481789 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f3dc800f020 con 0x7f3dd0109530 2024-05-08T05:26:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.173+0000 7f3dce59c700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3db80b2910 0x7f3db80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:49.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.174+0000 7f3dce59c700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3db80b2910 0x7f3db80b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0005950 tx=0x7f3dc000a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:49.177 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.177+0000 7f3dbf7fe700 1 -- 172.21.15.59:0/3620481789 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3dc802b020 con 0x7f3dd0109530 2024-05-08T05:26:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.438+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3dd01039d0 con 0x7f3dd0109530 2024-05-08T05:26:49.439 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.439+0000 7f3dbf7fe700 1 -- 172.21.15.59:0/3620481789 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7f3dc80ab950 con 0x7f3dd0109530 2024-05-08T05:26:49.440 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:26:49.440 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:26:31.316568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:26:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3db80b2910 msgr2=0x7f3db80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3db80b2910 0x7f3db80b4dd0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f3dc0005950 tx=0x7f3dc000a300 comp rx=0 tx=0).stop 2024-05-08T05:26:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd0109530 msgr2=0x7f3dd0102f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:49.442 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd0109530 0x7f3dd0102f20 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800b830 tx=0x7f3dc8004d00 comp rx=0 tx=0).stop 2024-05-08T05:26:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 shutdown_connections 2024-05-08T05:26:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3dd0109530 0x7f3dd0102f20 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3dd006bbd0 0x7f3dd01024a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3db80b2910 0x7f3db80b4dd0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 --2- 172.21.15.59:0/3620481789 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3dd006c590 0x7f3dd01029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 >> 172.21.15.59:0/3620481789 conn(0x7f3dd0079340 msgr2=0x7f3dd01011c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 shutdown_connections 2024-05-08T05:26:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:49.442+0000 7f3dd4935700 1 -- 172.21.15.59:0/3620481789 wait complete. 2024-05-08T05:26:49.443 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 29 2024-05-08T05:26:49.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:49 smithi143 ceph-mon[34337]: pgmap v340: 129 pgs: 129 active+clean; 628 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-05-08T05:26:49.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:49 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3620481789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:26:49.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:49 smithi096 ceph-mon[34253]: pgmap v340: 129 pgs: 129 active+clean; 628 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-05-08T05:26:49.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:49 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3620481789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:26:49.820 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:49 smithi059 ceph-mon[30304]: pgmap v340: 129 pgs: 129 active+clean; 628 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 4.3 KiB/s wr, 11 op/s 2024-05-08T05:26:49.821 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:49 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3620481789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:26:50.139 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:26:50.396 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:51.726 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:51 smithi059 ceph-mon[30304]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-05-08T05:26:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:51 smithi143 ceph-mon[34337]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-05-08T05:26:51.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:51 smithi096 ceph-mon[34253]: pgmap v341: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 5.7 KiB/s wr, 13 op/s 2024-05-08T05:26:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 -- 172.21.15.59:0/3611362686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328102820 msgr2=0x7fe328110290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:52.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3611362686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328102820 0x7fe328110290 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fe32000b0a0 tx=0x7fe32000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 -- 172.21.15.59:0/3611362686 shutdown_connections 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3611362686 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328102820 0x7fe328110290 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3611362686 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328108580 0x7fe328108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3611362686 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe328101e60 0x7fe3281022e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 -- 172.21.15.59:0/3611362686 >> 172.21.15.59:0/3611362686 conn(0x7fe3280776c0 msgr2=0x7fe328077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 -- 172.21.15.59:0/3611362686 shutdown_connections 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 -- 172.21.15.59:0/3611362686 wait complete. 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 Processor -- start 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.068+0000 7fe32f12d700 1 -- start start 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32f12d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe328101e60 0x7fe32819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32f12d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328102820 0x7fe32819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32f12d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328108580 0x7fe328199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32f12d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe328113710 con 0x7fe328102820 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32f12d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe328113590 con 0x7fe328108580 2024-05-08T05:26:52.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32f12d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe328113890 con 0x7fe328101e60 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32cec9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe328101e60 0x7fe32819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32cec9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe328101e60 0x7fe32819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:38120/0 (socket says 172.21.15.59:38120) 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32cec9700 1 -- 172.21.15.59:0/3974749025 learned_addr learned my addr 172.21.15.59:0/3974749025 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe327fff700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328102820 0x7fe32819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32d6ca700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328108580 0x7fe328199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe327fff700 1 -- 172.21.15.59:0/3974749025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe328101e60 msgr2=0x7fe32819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe327fff700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe328101e60 0x7fe32819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe327fff700 1 -- 172.21.15.59:0/3974749025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328108580 msgr2=0x7fe328199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe327fff700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328108580 0x7fe328199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe327fff700 1 -- 172.21.15.59:0/3974749025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe32819a070 con 0x7fe328102820 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.069+0000 7fe32cec9700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe328101e60 0x7fe32819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.070+0000 7fe327fff700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328102820 0x7fe32819fc90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe31400b450 tx=0x7fe31400b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.070+0000 7fe325ffb700 1 -- 172.21.15.59:0/3974749025 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314015020 con 0x7fe328102820 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.070+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe32819a360 con 0x7fe328102820 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.070+0000 7fe325ffb700 1 -- 172.21.15.59:0/3974749025 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe314011ce0 con 0x7fe328102820 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.070+0000 7fe325ffb700 1 -- 172.21.15.59:0/3974749025 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe314010be0 con 0x7fe328102820 2024-05-08T05:26:52.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.070+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3281a6620 con 0x7fe328102820 2024-05-08T05:26:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.071+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe32806a190 con 0x7fe328102820 2024-05-08T05:26:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.072+0000 7fe325ffb700 1 -- 172.21.15.59:0/3974749025 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe314018020 con 0x7fe328102820 2024-05-08T05:26:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.073+0000 7fe325ffb700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3180b2910 0x7fe3180b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:52.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.073+0000 7fe32cec9700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3180b2910 0x7fe3180b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:52.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.073+0000 7fe325ffb700 1 -- 172.21.15.59:0/3974749025 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7fe3140e1c80 con 0x7fe328102820 2024-05-08T05:26:52.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.074+0000 7fe32cec9700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3180b2910 0x7fe3180b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe32819aa50 tx=0x7fe31c00b4a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:52.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.078+0000 7fe325ffb700 1 -- 172.21.15.59:0/3974749025 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe31401f020 con 0x7fe328102820 2024-05-08T05:26:52.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.294+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe32819ae40 con 0x7fe328102820 2024-05-08T05:26:52.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.296+0000 7fe325ffb700 1 -- 172.21.15.59:0/3974749025 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v88) ==== 74+0+24601 (secure 0 0 0) 0x7fe3140a9590 con 0x7fe328102820 2024-05-08T05:26:52.296 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:26:52.297 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":88,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:26:05.137852+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3180b2910 msgr2=0x7fe3180b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3180b2910 0x7fe3180b4dd0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fe32819aa50 tx=0x7fe31c00b4a0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328102820 msgr2=0x7fe32819fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328102820 0x7fe32819fc90 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe31400b450 tx=0x7fe31400b760 comp rx=0 tx=0).stop 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 shutdown_connections 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3180b2910 0x7fe3180b4dd0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe328108580 0x7fe328199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe328101e60 0x7fe32819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 --2- 172.21.15.59:0/3974749025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe328102820 0x7fe32819fc90 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 >> 172.21.15.59:0/3974749025 conn(0x7fe3280776c0 msgr2=0x7fe3281009e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 shutdown_connections 2024-05-08T05:26:52.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:52.298+0000 7fe32f12d700 1 -- 172.21.15.59:0/3974749025 wait complete. 2024-05-08T05:26:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:52 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3974749025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:26:52.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:52 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3974749025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:26:52.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:52 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3974749025' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:26:52.988 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T05:26:52.989 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:26:53.244 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:53.697 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:53 smithi059 ceph-mon[30304]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-05-08T05:26:53.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:53 smithi143 ceph-mon[34337]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-05-08T05:26:53.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:53 smithi096 ceph-mon[34253]: pgmap v342: 129 pgs: 129 active+clean; 642 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 3.6 KiB/s wr, 12 op/s 2024-05-08T05:26:54.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.962+0000 7f8954f2a700 1 -- 172.21.15.59:0/3709873415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89501076c0 msgr2=0x7f895010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:54.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.962+0000 7f8954f2a700 1 --2- 172.21.15.59:0/3709873415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89501076c0 0x7f895010f290 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f894800b0d0 tx=0x7f894800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:26:54.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.962+0000 7f8954f2a700 1 -- 172.21.15.59:0/3709873415 shutdown_connections 2024-05-08T05:26:54.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.962+0000 7f8954f2a700 1 --2- 172.21.15.59:0/3709873415 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f89501076c0 0x7f895010f290 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:54.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.962+0000 7f8954f2a700 1 --2- 172.21.15.59:0/3709873415 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8950102630 0x7f8950102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:54.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.962+0000 7f8954f2a700 1 --2- 172.21.15.59:0/3709873415 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8950102fe0 0x7f8950106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:54.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.962+0000 7f8954f2a700 1 -- 172.21.15.59:0/3709873415 >> 172.21.15.59:0/3709873415 conn(0x7f8950079350 msgr2=0x7f8950079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:54.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.962+0000 7f8954f2a700 1 -- 172.21.15.59:0/3709873415 shutdown_connections 2024-05-08T05:26:54.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.962+0000 7f8954f2a700 1 -- 172.21.15.59:0/3709873415 wait complete. 2024-05-08T05:26:54.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.963+0000 7f8954f2a700 1 Processor -- start 2024-05-08T05:26:54.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.963+0000 7f8954f2a700 1 -- start start 2024-05-08T05:26:54.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.963+0000 7f8954f2a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950102630 0x7f895019f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.963+0000 7f8954f2a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8950102fe0 0x7f895019fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.963+0000 7f8954f2a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89501076c0 0x7f8950199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.963+0000 7f8954f2a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8950112650 con 0x7f8950102630 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.963+0000 7f8954f2a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89501124d0 con 0x7f8950102fe0 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.963+0000 7f8954f2a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f89501127d0 con 0x7f89501076c0 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950102630 0x7f895019f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950102630 0x7f895019f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35458/0 (socket says 172.21.15.59:35458) 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894e59c700 1 -- 172.21.15.59:0/999414646 learned_addr learned my addr 172.21.15.59:0/999414646 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894ed9d700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89501076c0 0x7f8950199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f8945d9b700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8950102fe0 0x7f895019fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:54.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894e59c700 1 -- 172.21.15.59:0/999414646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89501076c0 msgr2=0x7f8950199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894e59c700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89501076c0 0x7f8950199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894e59c700 1 -- 172.21.15.59:0/999414646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8950102fe0 msgr2=0x7f895019fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894e59c700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8950102fe0 0x7f895019fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894e59c700 1 -- 172.21.15.59:0/999414646 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895019a2a0 con 0x7f8950102630 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f894e59c700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950102630 0x7f895019f930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f894000b530 tx=0x7f894000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f8947fff700 1 -- 172.21.15.59:0/999414646 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940015020 con 0x7f8950102630 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.964+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895019a590 con 0x7f8950102630 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.965+0000 7f8947fff700 1 -- 172.21.15.59:0/999414646 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8940011ce0 con 0x7f8950102630 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.965+0000 7f8947fff700 1 -- 172.21.15.59:0/999414646 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8940010dd0 con 0x7f8950102630 2024-05-08T05:26:54.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.965+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89501a6800 con 0x7f8950102630 2024-05-08T05:26:54.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.966+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895006a190 con 0x7f8950102630 2024-05-08T05:26:54.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.967+0000 7f8947fff700 1 -- 172.21.15.59:0/999414646 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8940018020 con 0x7f8950102630 2024-05-08T05:26:54.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.968+0000 7f8947fff700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f893c0b6d20 0x7f893c0b91e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:54.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.968+0000 7f8945d9b700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f893c0b6d20 0x7f893c0b91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:54.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.968+0000 7f8947fff700 1 -- 172.21.15.59:0/999414646 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f89400e1d40 con 0x7f8950102630 2024-05-08T05:26:54.969 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.968+0000 7f8945d9b700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f893c0b6d20 0x7f893c0b91e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8950108d60 tx=0x7f8938009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:54.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:54.972+0000 7f8947fff700 1 -- 172.21.15.59:0/999414646 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f894001f020 con 0x7f8950102630 2024-05-08T05:26:55.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.181+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f895019afe0 con 0x7f893c0b6d20 2024-05-08T05:26:55.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.191+0000 7f8947fff700 1 -- 172.21.15.59:0/999414646 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340641 (secure 0 0 0) 0x7f895019afe0 con 0x7f893c0b6d20 2024-05-08T05:26:55.197 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:26:55.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f893c0b6d20 msgr2=0x7f893c0b91e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:55.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f893c0b6d20 0x7f893c0b91e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f8950108d60 tx=0x7f8938009190 comp rx=0 tx=0).stop 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950102630 msgr2=0x7f895019f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950102630 0x7f895019f930 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f894000b530 tx=0x7f894000b840 comp rx=0 tx=0).stop 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 shutdown_connections 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f89501076c0 0x7f8950199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8950102630 0x7f895019f930 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f893c0b6d20 0x7f893c0b91e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 --2- 172.21.15.59:0/999414646 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8950102fe0 0x7f895019fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.195+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 >> 172.21.15.59:0/999414646 conn(0x7f8950079350 msgr2=0x7f895006bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.196+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 shutdown_connections 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:55.196+0000 7f8954f2a700 1 -- 172.21.15.59:0/999414646 wait complete. 2024-05-08T05:26:55.201 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T05:26:55.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:55 smithi143 ceph-mon[34337]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-05-08T05:26:55.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:55 smithi096 ceph-mon[34253]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-05-08T05:26:55.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:55 smithi059 ceph-mon[30304]: pgmap v343: 129 pgs: 129 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 3.7 KiB/s wr, 12 op/s 2024-05-08T05:26:56.024 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":343,"stamp":"2024-05-08T05:26:54.001678+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658968,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":364,"num_read_kb":291,"num_write":375,"num_write_kb":2293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356952,"kb_used_data":11960,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584872,"statfs":{"total":1151940427776,"available":1151574908928,"internally_reserved":0,"allocated":12247040,"data_stored":7970190,"data_compressed":425568,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":35921,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":129,"num_read_kb":107,"num_write":26,"num_write_kb":45,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003383"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142305+0000","last_change":"2024-05-08T05:25:25.667895+0000","last_active":"2024-05-08T05:26:05.142305+0000","last_peered":"2024-05-08T05:26:05.142305+0000","last_clean":"2024-05-08T05:26:05.142305+0000","last_became_active":"2024-05-08T05:25:25.667563+0000","last_became_peered":"2024-05-08T05:25:25.667563+0000","last_unstale":"2024-05-08T05:26:05.142305+0000","last_undegraded":"2024-05-08T05:26:05.142305+0000","last_fullsized":"2024-05-08T05:26:05.142305+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:47.677082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147060+0000","last_change":"2024-05-08T05:25:25.673346+0000","last_active":"2024-05-08T05:26:05.147060+0000","last_peered":"2024-05-08T05:26:05.147060+0000","last_clean":"2024-05-08T05:26:05.147060+0000","last_became_active":"2024-05-08T05:25:25.673011+0000","last_became_peered":"2024-05-08T05:25:25.673011+0000","last_unstale":"2024-05-08T05:26:05.147060+0000","last_undegraded":"2024-05-08T05:26:05.147060+0000","last_fullsized":"2024-05-08T05:26:05.147060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:21:05.091177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146886+0000","last_change":"2024-05-08T05:25:25.670469+0000","last_active":"2024-05-08T05:26:05.146886+0000","last_peered":"2024-05-08T05:26:05.146886+0000","last_clean":"2024-05-08T05:26:05.146886+0000","last_became_active":"2024-05-08T05:25:25.669700+0000","last_became_peered":"2024-05-08T05:25:25.669700+0000","last_unstale":"2024-05-08T05:26:05.146886+0000","last_undegraded":"2024-05-08T05:26:05.146886+0000","last_fullsized":"2024-05-08T05:26:05.146886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:06.046633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.145008+0000","last_change":"2024-05-08T05:25:25.659937+0000","last_active":"2024-05-08T05:26:05.145008+0000","last_peered":"2024-05-08T05:26:05.145008+0000","last_clean":"2024-05-08T05:26:05.145008+0000","last_became_active":"2024-05-08T05:25:25.658892+0000","last_became_peered":"2024-05-08T05:25:25.658892+0000","last_unstale":"2024-05-08T05:26:05.145008+0000","last_undegraded":"2024-05-08T05:26:05.145008+0000","last_fullsized":"2024-05-08T05:26:05.145008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:10.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143347+0000","last_change":"2024-05-08T05:25:25.669132+0000","last_active":"2024-05-08T05:26:05.143347+0000","last_peered":"2024-05-08T05:26:05.143347+0000","last_clean":"2024-05-08T05:26:05.143347+0000","last_became_active":"2024-05-08T05:25:25.668556+0000","last_became_peered":"2024-05-08T05:25:25.668556+0000","last_unstale":"2024-05-08T05:26:05.143347+0000","last_undegraded":"2024-05-08T05:26:05.143347+0000","last_fullsized":"2024-05-08T05:26:05.143347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:28.443763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143577+0000","last_change":"2024-05-08T05:25:25.668978+0000","last_active":"2024-05-08T05:26:05.143577+0000","last_peered":"2024-05-08T05:26:05.143577+0000","last_clean":"2024-05-08T05:26:05.143577+0000","last_became_active":"2024-05-08T05:25:25.668647+0000","last_became_peered":"2024-05-08T05:25:25.668647+0000","last_unstale":"2024-05-08T05:26:05.143577+0000","last_undegraded":"2024-05-08T05:26:05.143577+0000","last_fullsized":"2024-05-08T05:26:05.143577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:07:00.962620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143778+0000","last_change":"2024-05-08T05:25:25.664784+0000","last_active":"2024-05-08T05:26:05.143778+0000","last_peered":"2024-05-08T05:26:05.143778+0000","last_clean":"2024-05-08T05:26:05.143778+0000","last_became_active":"2024-05-08T05:25:25.664390+0000","last_became_peered":"2024-05-08T05:25:25.664390+0000","last_unstale":"2024-05-08T05:26:05.143778+0000","last_undegraded":"2024-05-08T05:26:05.143778+0000","last_fullsized":"2024-05-08T05:26:05.143778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:08:35.358796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306345+0000","last_change":"2024-05-08T05:25:25.677139+0000","last_active":"2024-05-08T05:26:05.306345+0000","last_peered":"2024-05-08T05:26:05.306345+0000","last_clean":"2024-05-08T05:26:05.306345+0000","last_became_active":"2024-05-08T05:25:25.676444+0000","last_became_peered":"2024-05-08T05:25:25.676444+0000","last_unstale":"2024-05-08T05:26:05.306345+0000","last_undegraded":"2024-05-08T05:26:05.306345+0000","last_fullsized":"2024-05-08T05:26:05.306345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:56.704152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.307286+0000","last_change":"2024-05-08T05:25:25.675593+0000","last_active":"2024-05-08T05:26:05.307286+0000","last_peered":"2024-05-08T05:26:05.307286+0000","last_clean":"2024-05-08T05:26:05.307286+0000","last_became_active":"2024-05-08T05:25:25.675136+0000","last_became_peered":"2024-05-08T05:25:25.675136+0000","last_unstale":"2024-05-08T05:26:05.307286+0000","last_undegraded":"2024-05-08T05:26:05.307286+0000","last_fullsized":"2024-05-08T05:26:05.307286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:49.042956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143328+0000","last_change":"2024-05-08T05:25:25.669384+0000","last_active":"2024-05-08T05:26:05.143328+0000","last_peered":"2024-05-08T05:26:05.143328+0000","last_clean":"2024-05-08T05:26:05.143328+0000","last_became_active":"2024-05-08T05:25:25.668861+0000","last_became_peered":"2024-05-08T05:25:25.668861+0000","last_unstale":"2024-05-08T05:26:05.143328+0000","last_undegraded":"2024-05-08T05:26:05.143328+0000","last_fullsized":"2024-05-08T05:26:05.143328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:57.491242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144002+0000","last_change":"2024-05-08T05:25:25.669463+0000","last_active":"2024-05-08T05:26:05.144002+0000","last_peered":"2024-05-08T05:26:05.144002+0000","last_clean":"2024-05-08T05:26:05.144002+0000","last_became_active":"2024-05-08T05:25:25.668672+0000","last_became_peered":"2024-05-08T05:25:25.668672+0000","last_unstale":"2024-05-08T05:26:05.144002+0000","last_undegraded":"2024-05-08T05:26:05.144002+0000","last_fullsized":"2024-05-08T05:26:05.144002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:30.779473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144902+0000","last_change":"2024-05-08T05:25:25.670378+0000","last_active":"2024-05-08T05:26:05.144902+0000","last_peered":"2024-05-08T05:26:05.144902+0000","last_clean":"2024-05-08T05:26:05.144902+0000","last_became_active":"2024-05-08T05:25:25.669801+0000","last_became_peered":"2024-05-08T05:25:25.669801+0000","last_unstale":"2024-05-08T05:26:05.144902+0000","last_undegraded":"2024-05-08T05:26:05.144902+0000","last_fullsized":"2024-05-08T05:26:05.144902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:49.614163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306176+0000","last_change":"2024-05-08T05:25:25.670874+0000","last_active":"2024-05-08T05:26:05.306176+0000","last_peered":"2024-05-08T05:26:05.306176+0000","last_clean":"2024-05-08T05:26:05.306176+0000","last_became_active":"2024-05-08T05:25:25.670557+0000","last_became_peered":"2024-05-08T05:25:25.670557+0000","last_unstale":"2024-05-08T05:26:05.306176+0000","last_undegraded":"2024-05-08T05:26:05.306176+0000","last_fullsized":"2024-05-08T05:26:05.306176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:58.594223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146585+0000","last_change":"2024-05-08T05:25:25.670015+0000","last_active":"2024-05-08T05:26:05.146585+0000","last_peered":"2024-05-08T05:26:05.146585+0000","last_clean":"2024-05-08T05:26:05.146585+0000","last_became_active":"2024-05-08T05:25:25.669493+0000","last_became_peered":"2024-05-08T05:25:25.669493+0000","last_unstale":"2024-05-08T05:26:05.146585+0000","last_undegraded":"2024-05-08T05:26:05.146585+0000","last_fullsized":"2024-05-08T05:26:05.146585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:49.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146559+0000","last_change":"2024-05-08T05:25:25.659651+0000","last_active":"2024-05-08T05:26:05.146559+0000","last_peered":"2024-05-08T05:26:05.146559+0000","last_clean":"2024-05-08T05:26:05.146559+0000","last_became_active":"2024-05-08T05:25:25.659308+0000","last_became_peered":"2024-05-08T05:25:25.659308+0000","last_unstale":"2024-05-08T05:26:05.146559+0000","last_undegraded":"2024-05-08T05:26:05.146559+0000","last_fullsized":"2024-05-08T05:26:05.146559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:49.945474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306249+0000","last_change":"2024-05-08T05:25:25.663696+0000","last_active":"2024-05-08T05:26:05.306249+0000","last_peered":"2024-05-08T05:26:05.306249+0000","last_clean":"2024-05-08T05:26:05.306249+0000","last_became_active":"2024-05-08T05:25:25.662590+0000","last_became_peered":"2024-05-08T05:25:25.662590+0000","last_unstale":"2024-05-08T05:26:05.306249+0000","last_undegraded":"2024-05-08T05:26:05.306249+0000","last_fullsized":"2024-05-08T05:26:05.306249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:29:21.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142073+0000","last_change":"2024-05-08T05:25:25.668552+0000","last_active":"2024-05-08T05:26:05.142073+0000","last_peered":"2024-05-08T05:26:05.142073+0000","last_clean":"2024-05-08T05:26:05.142073+0000","last_became_active":"2024-05-08T05:25:25.668185+0000","last_became_peered":"2024-05-08T05:25:25.668185+0000","last_unstale":"2024-05-08T05:26:05.142073+0000","last_undegraded":"2024-05-08T05:26:05.142073+0000","last_fullsized":"2024-05-08T05:26:05.142073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:25.762909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141657+0000","last_change":"2024-05-08T05:25:25.668259+0000","last_active":"2024-05-08T05:26:05.141657+0000","last_peered":"2024-05-08T05:26:05.141657+0000","last_clean":"2024-05-08T05:26:05.141657+0000","last_became_active":"2024-05-08T05:25:25.667767+0000","last_became_peered":"2024-05-08T05:25:25.667767+0000","last_unstale":"2024-05-08T05:26:05.141657+0000","last_undegraded":"2024-05-08T05:26:05.141657+0000","last_fullsized":"2024-05-08T05:26:05.141657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:36:34.984139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147777+0000","last_change":"2024-05-08T05:25:25.663381+0000","last_active":"2024-05-08T05:26:05.147777+0000","last_peered":"2024-05-08T05:26:05.147777+0000","last_clean":"2024-05-08T05:26:05.147777+0000","last_became_active":"2024-05-08T05:25:25.663019+0000","last_became_peered":"2024-05-08T05:25:25.663019+0000","last_unstale":"2024-05-08T05:26:05.147777+0000","last_undegraded":"2024-05-08T05:26:05.147777+0000","last_fullsized":"2024-05-08T05:26:05.147777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:58.440446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147242+0000","last_change":"2024-05-08T05:25:37.052656+0000","last_active":"2024-05-08T05:26:05.147242+0000","last_peered":"2024-05-08T05:26:05.147242+0000","last_clean":"2024-05-08T05:26:05.147242+0000","last_became_active":"2024-05-08T05:25:37.052237+0000","last_became_peered":"2024-05-08T05:25:37.052237+0000","last_unstale":"2024-05-08T05:26:05.147242+0000","last_undegraded":"2024-05-08T05:26:05.147242+0000","last_fullsized":"2024-05-08T05:26:05.147242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:54.354488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":79,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:52.454150+0000","last_change":"2024-05-08T05:25:22.649492+0000","last_active":"2024-05-08T05:26:52.454150+0000","last_peered":"2024-05-08T05:26:52.454150+0000","last_clean":"2024-05-08T05:26:52.454150+0000","last_became_active":"2024-05-08T05:25:22.649132+0000","last_became_peered":"2024-05-08T05:25:22.649132+0000","last_unstale":"2024-05-08T05:26:52.454150+0000","last_undegraded":"2024-05-08T05:26:52.454150+0000","last_fullsized":"2024-05-08T05:26:52.454150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:24.474848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306757+0000","last_change":"2024-05-08T05:25:25.667811+0000","last_active":"2024-05-08T05:26:05.306757+0000","last_peered":"2024-05-08T05:26:05.306757+0000","last_clean":"2024-05-08T05:26:05.306757+0000","last_became_active":"2024-05-08T05:25:25.667449+0000","last_became_peered":"2024-05-08T05:25:25.667449+0000","last_unstale":"2024-05-08T05:26:05.306757+0000","last_undegraded":"2024-05-08T05:26:05.306757+0000","last_fullsized":"2024-05-08T05:26:05.306757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:29.293702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.308251+0000","last_change":"2024-05-08T05:26:05.308251+0000","last_active":"2024-05-08T05:26:05.308251+0000","last_peered":"2024-05-08T05:26:05.308251+0000","last_clean":"2024-05-08T05:26:05.308251+0000","last_became_active":"2024-05-08T05:26:05.307622+0000","last_became_peered":"2024-05-08T05:26:05.307622+0000","last_unstale":"2024-05-08T05:26:05.308251+0000","last_undegraded":"2024-05-08T05:26:05.308251+0000","last_fullsized":"2024-05-08T05:26:05.308251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:51.116094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306902+0000","last_change":"2024-05-08T05:25:22.748130+0000","last_active":"2024-05-08T05:26:05.306902+0000","last_peered":"2024-05-08T05:26:05.306902+0000","last_clean":"2024-05-08T05:26:05.306902+0000","last_became_active":"2024-05-08T05:25:22.747769+0000","last_became_peered":"2024-05-08T05:25:22.747769+0000","last_unstale":"2024-05-08T05:26:05.306902+0000","last_undegraded":"2024-05-08T05:26:05.306902+0000","last_fullsized":"2024-05-08T05:26:05.306902+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:04.086851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147250+0000","last_change":"2024-05-08T05:25:25.672114+0000","last_active":"2024-05-08T05:26:05.147250+0000","last_peered":"2024-05-08T05:26:05.147250+0000","last_clean":"2024-05-08T05:26:05.147250+0000","last_became_active":"2024-05-08T05:25:25.671761+0000","last_became_peered":"2024-05-08T05:25:25.671761+0000","last_unstale":"2024-05-08T05:26:05.147250+0000","last_undegraded":"2024-05-08T05:26:05.147250+0000","last_fullsized":"2024-05-08T05:26:05.147250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:40.684565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143884+0000","last_change":"2024-05-08T05:25:37.058375+0000","last_active":"2024-05-08T05:26:05.143884+0000","last_peered":"2024-05-08T05:26:05.143884+0000","last_clean":"2024-05-08T05:26:05.143884+0000","last_became_active":"2024-05-08T05:25:37.057844+0000","last_became_peered":"2024-05-08T05:25:37.057844+0000","last_unstale":"2024-05-08T05:26:05.143884+0000","last_undegraded":"2024-05-08T05:26:05.143884+0000","last_fullsized":"2024-05-08T05:26:05.143884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:51:42.881830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:28.420331+0000","last_change":"2024-05-08T05:25:22.646177+0000","last_active":"2024-05-08T05:26:28.420331+0000","last_peered":"2024-05-08T05:26:28.420331+0000","last_clean":"2024-05-08T05:26:28.420331+0000","last_became_active":"2024-05-08T05:25:22.645755+0000","last_became_peered":"2024-05-08T05:25:22.645755+0000","last_unstale":"2024-05-08T05:26:28.420331+0000","last_undegraded":"2024-05-08T05:26:28.420331+0000","last_fullsized":"2024-05-08T05:26:28.420331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:08:27.891850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306826+0000","last_change":"2024-05-08T05:25:25.673300+0000","last_active":"2024-05-08T05:26:05.306826+0000","last_peered":"2024-05-08T05:26:05.306826+0000","last_clean":"2024-05-08T05:26:05.306826+0000","last_became_active":"2024-05-08T05:25:25.672881+0000","last_became_peered":"2024-05-08T05:25:25.672881+0000","last_unstale":"2024-05-08T05:26:05.306826+0000","last_undegraded":"2024-05-08T05:26:05.306826+0000","last_fullsized":"2024-05-08T05:26:05.306826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:16.111775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143086+0000","last_change":"2024-05-08T05:25:37.058903+0000","last_active":"2024-05-08T05:26:05.143086+0000","last_peered":"2024-05-08T05:26:05.143086+0000","last_clean":"2024-05-08T05:26:05.143086+0000","last_became_active":"2024-05-08T05:25:37.058508+0000","last_became_peered":"2024-05-08T05:25:37.058508+0000","last_unstale":"2024-05-08T05:26:05.143086+0000","last_undegraded":"2024-05-08T05:26:05.143086+0000","last_fullsized":"2024-05-08T05:26:05.143086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:15:16.605748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.418635+0000","last_change":"2024-05-08T05:25:22.646191+0000","last_active":"2024-05-08T05:26:27.418635+0000","last_peered":"2024-05-08T05:26:27.418635+0000","last_clean":"2024-05-08T05:26:27.418635+0000","last_became_active":"2024-05-08T05:25:22.645187+0000","last_became_peered":"2024-05-08T05:25:22.645187+0000","last_unstale":"2024-05-08T05:26:27.418635+0000","last_undegraded":"2024-05-08T05:26:27.418635+0000","last_fullsized":"2024-05-08T05:26:27.418635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:13.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.308149+0000","last_change":"2024-05-08T05:26:05.308149+0000","last_active":"2024-05-08T05:26:05.308149+0000","last_peered":"2024-05-08T05:26:05.308149+0000","last_clean":"2024-05-08T05:26:05.308149+0000","last_became_active":"2024-05-08T05:26:05.307602+0000","last_became_peered":"2024-05-08T05:26:05.307602+0000","last_unstale":"2024-05-08T05:26:05.308149+0000","last_undegraded":"2024-05-08T05:26:05.308149+0000","last_fullsized":"2024-05-08T05:26:05.308149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:02.791894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146867+0000","last_change":"2024-05-08T05:25:37.059052+0000","last_active":"2024-05-08T05:26:05.146867+0000","last_peered":"2024-05-08T05:26:05.146867+0000","last_clean":"2024-05-08T05:26:05.146867+0000","last_became_active":"2024-05-08T05:25:37.058545+0000","last_became_peered":"2024-05-08T05:25:37.058545+0000","last_unstale":"2024-05-08T05:26:05.146867+0000","last_undegraded":"2024-05-08T05:26:05.146867+0000","last_fullsized":"2024-05-08T05:26:05.146867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:31.521284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:48.449844+0000","last_change":"2024-05-08T05:25:22.648874+0000","last_active":"2024-05-08T05:26:48.449844+0000","last_peered":"2024-05-08T05:26:48.449844+0000","last_clean":"2024-05-08T05:26:48.449844+0000","last_became_active":"2024-05-08T05:25:22.648477+0000","last_became_peered":"2024-05-08T05:25:22.648477+0000","last_unstale":"2024-05-08T05:26:48.449844+0000","last_undegraded":"2024-05-08T05:26:48.449844+0000","last_fullsized":"2024-05-08T05:26:48.449844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:08.075904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143228+0000","last_change":"2024-05-08T05:25:25.662449+0000","last_active":"2024-05-08T05:26:05.143228+0000","last_peered":"2024-05-08T05:26:05.143228+0000","last_clean":"2024-05-08T05:26:05.143228+0000","last_became_active":"2024-05-08T05:25:25.662089+0000","last_became_peered":"2024-05-08T05:25:25.662089+0000","last_unstale":"2024-05-08T05:26:05.143228+0000","last_undegraded":"2024-05-08T05:26:05.143228+0000","last_fullsized":"2024-05-08T05:26:05.143228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:31.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147434+0000","last_change":"2024-05-08T05:25:37.055548+0000","last_active":"2024-05-08T05:26:05.147434+0000","last_peered":"2024-05-08T05:26:05.147434+0000","last_clean":"2024-05-08T05:26:05.147434+0000","last_became_active":"2024-05-08T05:25:37.055138+0000","last_became_peered":"2024-05-08T05:25:37.055138+0000","last_unstale":"2024-05-08T05:26:05.147434+0000","last_undegraded":"2024-05-08T05:26:05.147434+0000","last_fullsized":"2024-05-08T05:26:05.147434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:45.697831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:26.411198+0000","last_change":"2024-05-08T05:25:22.648072+0000","last_active":"2024-05-08T05:26:26.411198+0000","last_peered":"2024-05-08T05:26:26.411198+0000","last_clean":"2024-05-08T05:26:26.411198+0000","last_became_active":"2024-05-08T05:25:22.647457+0000","last_became_peered":"2024-05-08T05:25:22.647457+0000","last_unstale":"2024-05-08T05:26:26.411198+0000","last_undegraded":"2024-05-08T05:26:26.411198+0000","last_fullsized":"2024-05-08T05:26:26.411198+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:17.476479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.148392+0000","last_change":"2024-05-08T05:25:25.662698+0000","last_active":"2024-05-08T05:26:05.148392+0000","last_peered":"2024-05-08T05:26:05.148392+0000","last_clean":"2024-05-08T05:26:05.148392+0000","last_became_active":"2024-05-08T05:25:25.662368+0000","last_became_peered":"2024-05-08T05:25:25.662368+0000","last_unstale":"2024-05-08T05:26:05.148392+0000","last_undegraded":"2024-05-08T05:26:05.148392+0000","last_fullsized":"2024-05-08T05:26:05.148392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:38.511037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142093+0000","last_change":"2024-05-08T05:25:37.054407+0000","last_active":"2024-05-08T05:26:05.142093+0000","last_peered":"2024-05-08T05:26:05.142093+0000","last_clean":"2024-05-08T05:26:05.142093+0000","last_became_active":"2024-05-08T05:25:37.053999+0000","last_became_peered":"2024-05-08T05:25:37.053999+0000","last_unstale":"2024-05-08T05:26:05.142093+0000","last_undegraded":"2024-05-08T05:26:05.142093+0000","last_fullsized":"2024-05-08T05:26:05.142093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:16.942601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:25.403902+0000","last_change":"2024-05-08T05:25:22.643654+0000","last_active":"2024-05-08T05:26:25.403902+0000","last_peered":"2024-05-08T05:26:25.403902+0000","last_clean":"2024-05-08T05:26:25.403902+0000","last_became_active":"2024-05-08T05:25:22.643315+0000","last_became_peered":"2024-05-08T05:25:22.643315+0000","last_unstale":"2024-05-08T05:26:25.403902+0000","last_undegraded":"2024-05-08T05:26:25.403902+0000","last_fullsized":"2024-05-08T05:26:25.403902+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:53.514629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306242+0000","last_change":"2024-05-08T05:25:25.663148+0000","last_active":"2024-05-08T05:26:05.306242+0000","last_peered":"2024-05-08T05:26:05.306242+0000","last_clean":"2024-05-08T05:26:05.306242+0000","last_became_active":"2024-05-08T05:25:25.662770+0000","last_became_peered":"2024-05-08T05:25:25.662770+0000","last_unstale":"2024-05-08T05:26:05.306242+0000","last_undegraded":"2024-05-08T05:26:05.306242+0000","last_fullsized":"2024-05-08T05:26:05.306242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:46.458233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146964+0000","last_change":"2024-05-08T05:25:37.054265+0000","last_active":"2024-05-08T05:26:05.146964+0000","last_peered":"2024-05-08T05:26:05.146964+0000","last_clean":"2024-05-08T05:26:05.146964+0000","last_became_active":"2024-05-08T05:25:37.053579+0000","last_became_peered":"2024-05-08T05:25:37.053579+0000","last_unstale":"2024-05-08T05:26:05.146964+0000","last_undegraded":"2024-05-08T05:26:05.146964+0000","last_fullsized":"2024-05-08T05:26:05.146964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:21.748715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.415915+0000","last_change":"2024-05-08T05:25:22.648083+0000","last_active":"2024-05-08T05:26:27.415915+0000","last_peered":"2024-05-08T05:26:27.415915+0000","last_clean":"2024-05-08T05:26:27.415915+0000","last_became_active":"2024-05-08T05:25:22.647650+0000","last_became_peered":"2024-05-08T05:25:22.647650+0000","last_unstale":"2024-05-08T05:26:27.415915+0000","last_undegraded":"2024-05-08T05:26:27.415915+0000","last_fullsized":"2024-05-08T05:26:27.415915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:56.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.140998+0000","last_change":"2024-05-08T05:25:25.668290+0000","last_active":"2024-05-08T05:26:05.140998+0000","last_peered":"2024-05-08T05:26:05.140998+0000","last_clean":"2024-05-08T05:26:05.140998+0000","last_became_active":"2024-05-08T05:25:25.667888+0000","last_became_peered":"2024-05-08T05:25:25.667888+0000","last_unstale":"2024-05-08T05:26:05.140998+0000","last_undegraded":"2024-05-08T05:26:05.140998+0000","last_fullsized":"2024-05-08T05:26:05.140998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:19.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306117+0000","last_change":"2024-05-08T05:25:37.057639+0000","last_active":"2024-05-08T05:26:05.306117+0000","last_peered":"2024-05-08T05:26:05.306117+0000","last_clean":"2024-05-08T05:26:05.306117+0000","last_became_active":"2024-05-08T05:25:37.056848+0000","last_became_peered":"2024-05-08T05:25:37.056848+0000","last_unstale":"2024-05-08T05:26:05.306117+0000","last_undegraded":"2024-05-08T05:26:05.306117+0000","last_fullsized":"2024-05-08T05:26:05.306117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:37.137224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:52.454828+0000","last_change":"2024-05-08T05:25:22.649658+0000","last_active":"2024-05-08T05:26:52.454828+0000","last_peered":"2024-05-08T05:26:52.454828+0000","last_clean":"2024-05-08T05:26:52.454828+0000","last_became_active":"2024-05-08T05:25:22.648771+0000","last_became_peered":"2024-05-08T05:25:22.648771+0000","last_unstale":"2024-05-08T05:26:52.454828+0000","last_undegraded":"2024-05-08T05:26:52.454828+0000","last_fullsized":"2024-05-08T05:26:52.454828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:54.412710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.307197+0000","last_change":"2024-05-08T05:25:25.673931+0000","last_active":"2024-05-08T05:26:05.307197+0000","last_peered":"2024-05-08T05:26:05.307197+0000","last_clean":"2024-05-08T05:26:05.307197+0000","last_became_active":"2024-05-08T05:25:25.673585+0000","last_became_peered":"2024-05-08T05:25:25.673585+0000","last_unstale":"2024-05-08T05:26:05.307197+0000","last_undegraded":"2024-05-08T05:26:05.307197+0000","last_fullsized":"2024-05-08T05:26:05.307197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:27.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.140846+0000","last_change":"2024-05-08T05:25:37.047651+0000","last_active":"2024-05-08T05:26:05.140846+0000","last_peered":"2024-05-08T05:26:05.140846+0000","last_clean":"2024-05-08T05:26:05.140846+0000","last_became_active":"2024-05-08T05:25:37.047246+0000","last_became_peered":"2024-05-08T05:25:37.047246+0000","last_unstale":"2024-05-08T05:26:05.140846+0000","last_undegraded":"2024-05-08T05:26:05.140846+0000","last_fullsized":"2024-05-08T05:26:05.140846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:22.300076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417326+0000","last_change":"2024-05-08T05:25:22.646973+0000","last_active":"2024-05-08T05:26:27.417326+0000","last_peered":"2024-05-08T05:26:27.417326+0000","last_clean":"2024-05-08T05:26:27.417326+0000","last_became_active":"2024-05-08T05:25:22.646232+0000","last_became_peered":"2024-05-08T05:25:22.646232+0000","last_unstale":"2024-05-08T05:26:27.417326+0000","last_undegraded":"2024-05-08T05:26:27.417326+0000","last_fullsized":"2024-05-08T05:26:27.417326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:38.147470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147248+0000","last_change":"2024-05-08T05:25:25.661373+0000","last_active":"2024-05-08T05:26:05.147248+0000","last_peered":"2024-05-08T05:26:05.147248+0000","last_clean":"2024-05-08T05:26:05.147248+0000","last_became_active":"2024-05-08T05:25:25.661080+0000","last_became_peered":"2024-05-08T05:25:25.661080+0000","last_unstale":"2024-05-08T05:26:05.147248+0000","last_undegraded":"2024-05-08T05:26:05.147248+0000","last_fullsized":"2024-05-08T05:26:05.147248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:18.718610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144077+0000","last_change":"2024-05-08T05:25:37.051797+0000","last_active":"2024-05-08T05:26:05.144077+0000","last_peered":"2024-05-08T05:26:05.144077+0000","last_clean":"2024-05-08T05:26:05.144077+0000","last_became_active":"2024-05-08T05:25:37.051381+0000","last_became_peered":"2024-05-08T05:25:37.051381+0000","last_unstale":"2024-05-08T05:26:05.144077+0000","last_undegraded":"2024-05-08T05:26:05.144077+0000","last_fullsized":"2024-05-08T05:26:05.144077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:34.337512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:26.408648+0000","last_change":"2024-05-08T05:25:22.748510+0000","last_active":"2024-05-08T05:26:26.408648+0000","last_peered":"2024-05-08T05:26:26.408648+0000","last_clean":"2024-05-08T05:26:26.408648+0000","last_became_active":"2024-05-08T05:25:22.748140+0000","last_became_peered":"2024-05-08T05:25:22.748140+0000","last_unstale":"2024-05-08T05:26:26.408648+0000","last_undegraded":"2024-05-08T05:26:26.408648+0000","last_fullsized":"2024-05-08T05:26:26.408648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:32.325653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306615+0000","last_change":"2024-05-08T05:25:25.671261+0000","last_active":"2024-05-08T05:26:05.306615+0000","last_peered":"2024-05-08T05:26:05.306615+0000","last_clean":"2024-05-08T05:26:05.306615+0000","last_became_active":"2024-05-08T05:25:25.670937+0000","last_became_peered":"2024-05-08T05:25:25.670937+0000","last_unstale":"2024-05-08T05:26:05.306615+0000","last_undegraded":"2024-05-08T05:26:05.306615+0000","last_fullsized":"2024-05-08T05:26:05.306615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:41.375908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.151127+0000","last_change":"2024-05-08T05:26:05.150754+0000","last_active":"2024-05-08T05:26:05.151127+0000","last_peered":"2024-05-08T05:26:05.151127+0000","last_clean":"2024-05-08T05:26:05.151127+0000","last_became_active":"2024-05-08T05:26:05.150331+0000","last_became_peered":"2024-05-08T05:26:05.150331+0000","last_unstale":"2024-05-08T05:26:05.151127+0000","last_undegraded":"2024-05-08T05:26:05.151127+0000","last_fullsized":"2024-05-08T05:26:05.151127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:23.309103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:52.460615+0000","last_change":"2024-05-08T05:25:22.645627+0000","last_active":"2024-05-08T05:26:52.460615+0000","last_peered":"2024-05-08T05:26:52.460615+0000","last_clean":"2024-05-08T05:26:52.460615+0000","last_became_active":"2024-05-08T05:25:22.645243+0000","last_became_peered":"2024-05-08T05:25:22.645243+0000","last_unstale":"2024-05-08T05:26:52.460615+0000","last_undegraded":"2024-05-08T05:26:52.460615+0000","last_fullsized":"2024-05-08T05:26:52.460615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:23.256903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60380,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":88,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144143+0000","last_change":"2024-05-08T05:25:25.661258+0000","last_active":"2024-05-08T05:26:05.144143+0000","last_peered":"2024-05-08T05:26:05.144143+0000","last_clean":"2024-05-08T05:26:05.144143+0000","last_became_active":"2024-05-08T05:25:25.660925+0000","last_became_peered":"2024-05-08T05:25:25.660925+0000","last_unstale":"2024-05-08T05:26:05.144143+0000","last_undegraded":"2024-05-08T05:26:05.144143+0000","last_fullsized":"2024-05-08T05:26:05.144143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:58.764540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.150117+0000","last_change":"2024-05-08T05:25:37.053539+0000","last_active":"2024-05-08T05:26:05.150117+0000","last_peered":"2024-05-08T05:26:05.150117+0000","last_clean":"2024-05-08T05:26:05.150117+0000","last_became_active":"2024-05-08T05:25:37.053102+0000","last_became_peered":"2024-05-08T05:25:37.053102+0000","last_unstale":"2024-05-08T05:26:05.150117+0000","last_undegraded":"2024-05-08T05:26:05.150117+0000","last_fullsized":"2024-05-08T05:26:05.150117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:48.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:28.419558+0000","last_change":"2024-05-08T05:25:22.646651+0000","last_active":"2024-05-08T05:26:28.419558+0000","last_peered":"2024-05-08T05:26:28.419558+0000","last_clean":"2024-05-08T05:26:28.419558+0000","last_became_active":"2024-05-08T05:25:22.646285+0000","last_became_peered":"2024-05-08T05:25:22.646285+0000","last_unstale":"2024-05-08T05:26:28.419558+0000","last_undegraded":"2024-05-08T05:26:28.419558+0000","last_fullsized":"2024-05-08T05:26:28.419558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.843858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306602+0000","last_change":"2024-05-08T05:25:25.666018+0000","last_active":"2024-05-08T05:26:05.306602+0000","last_peered":"2024-05-08T05:26:05.306602+0000","last_clean":"2024-05-08T05:26:05.306602+0000","last_became_active":"2024-05-08T05:25:25.663609+0000","last_became_peered":"2024-05-08T05:25:25.663609+0000","last_unstale":"2024-05-08T05:26:05.306602+0000","last_undegraded":"2024-05-08T05:26:05.306602+0000","last_fullsized":"2024-05-08T05:26:05.306602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:36.223121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141910+0000","last_change":"2024-05-08T05:25:37.052907+0000","last_active":"2024-05-08T05:26:05.141910+0000","last_peered":"2024-05-08T05:26:05.141910+0000","last_clean":"2024-05-08T05:26:05.141910+0000","last_became_active":"2024-05-08T05:25:37.052506+0000","last_became_peered":"2024-05-08T05:25:37.052506+0000","last_unstale":"2024-05-08T05:26:05.141910+0000","last_undegraded":"2024-05-08T05:26:05.141910+0000","last_fullsized":"2024-05-08T05:26:05.141910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:41.356996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.415990+0000","last_change":"2024-05-08T05:25:22.646671+0000","last_active":"2024-05-08T05:26:27.415990+0000","last_peered":"2024-05-08T05:26:27.415990+0000","last_clean":"2024-05-08T05:26:27.415990+0000","last_became_active":"2024-05-08T05:25:22.646202+0000","last_became_peered":"2024-05-08T05:25:22.646202+0000","last_unstale":"2024-05-08T05:26:27.415990+0000","last_undegraded":"2024-05-08T05:26:27.415990+0000","last_fullsized":"2024-05-08T05:26:27.415990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:59.224960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143532+0000","last_change":"2024-05-08T05:25:25.666969+0000","last_active":"2024-05-08T05:26:05.143532+0000","last_peered":"2024-05-08T05:26:05.143532+0000","last_clean":"2024-05-08T05:26:05.143532+0000","last_became_active":"2024-05-08T05:25:25.666482+0000","last_became_peered":"2024-05-08T05:25:25.666482+0000","last_unstale":"2024-05-08T05:26:05.143532+0000","last_undegraded":"2024-05-08T05:26:05.143532+0000","last_fullsized":"2024-05-08T05:26:05.143532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:07.819116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141632+0000","last_change":"2024-05-08T05:25:37.055807+0000","last_active":"2024-05-08T05:26:05.141632+0000","last_peered":"2024-05-08T05:26:05.141632+0000","last_clean":"2024-05-08T05:26:05.141632+0000","last_became_active":"2024-05-08T05:25:37.054778+0000","last_became_peered":"2024-05-08T05:25:37.054778+0000","last_unstale":"2024-05-08T05:26:05.141632+0000","last_undegraded":"2024-05-08T05:26:05.141632+0000","last_fullsized":"2024-05-08T05:26:05.141632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:33.767885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:49.450604+0000","last_change":"2024-05-08T05:25:22.647093+0000","last_active":"2024-05-08T05:26:49.450604+0000","last_peered":"2024-05-08T05:26:49.450604+0000","last_clean":"2024-05-08T05:26:49.450604+0000","last_became_active":"2024-05-08T05:25:22.646733+0000","last_became_peered":"2024-05-08T05:25:22.646733+0000","last_unstale":"2024-05-08T05:26:49.450604+0000","last_undegraded":"2024-05-08T05:26:49.450604+0000","last_fullsized":"2024-05-08T05:26:49.450604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:04.346568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":27,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143602+0000","last_change":"2024-05-08T05:25:25.666565+0000","last_active":"2024-05-08T05:26:05.143602+0000","last_peered":"2024-05-08T05:26:05.143602+0000","last_clean":"2024-05-08T05:26:05.143602+0000","last_became_active":"2024-05-08T05:25:25.666138+0000","last_became_peered":"2024-05-08T05:25:25.666138+0000","last_unstale":"2024-05-08T05:26:05.143602+0000","last_undegraded":"2024-05-08T05:26:05.143602+0000","last_fullsized":"2024-05-08T05:26:05.143602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:05.396530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306274+0000","last_change":"2024-05-08T05:25:37.060933+0000","last_active":"2024-05-08T05:26:05.306274+0000","last_peered":"2024-05-08T05:26:05.306274+0000","last_clean":"2024-05-08T05:26:05.306274+0000","last_became_active":"2024-05-08T05:25:37.060388+0000","last_became_peered":"2024-05-08T05:25:37.060388+0000","last_unstale":"2024-05-08T05:26:05.306274+0000","last_undegraded":"2024-05-08T05:26:05.306274+0000","last_fullsized":"2024-05-08T05:26:05.306274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:09.776051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417329+0000","last_change":"2024-05-08T05:26:05.156863+0000","last_active":"2024-05-08T05:26:27.417329+0000","last_peered":"2024-05-08T05:26:27.417329+0000","last_clean":"2024-05-08T05:26:27.417329+0000","last_became_active":"2024-05-08T05:26:05.149008+0000","last_became_peered":"2024-05-08T05:26:05.149008+0000","last_unstale":"2024-05-08T05:26:27.417329+0000","last_undegraded":"2024-05-08T05:26:27.417329+0000","last_fullsized":"2024-05-08T05:26:27.417329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:56.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.148746+0000","last_change":"2024-05-08T05:25:25.670415+0000","last_active":"2024-05-08T05:26:05.148746+0000","last_peered":"2024-05-08T05:26:05.148746+0000","last_clean":"2024-05-08T05:26:05.148746+0000","last_became_active":"2024-05-08T05:25:25.670095+0000","last_became_peered":"2024-05-08T05:25:25.670095+0000","last_unstale":"2024-05-08T05:26:05.148746+0000","last_undegraded":"2024-05-08T05:26:05.148746+0000","last_fullsized":"2024-05-08T05:26:05.148746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.489203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.308609+0000","last_change":"2024-05-08T05:26:05.307721+0000","last_active":"2024-05-08T05:26:05.308609+0000","last_peered":"2024-05-08T05:26:05.308609+0000","last_clean":"2024-05-08T05:26:05.308609+0000","last_became_active":"2024-05-08T05:26:05.307279+0000","last_became_peered":"2024-05-08T05:26:05.307279+0000","last_unstale":"2024-05-08T05:26:05.308609+0000","last_undegraded":"2024-05-08T05:26:05.308609+0000","last_fullsized":"2024-05-08T05:26:05.308609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:08.857770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.397844+0000","last_change":"2024-05-08T05:25:22.644279+0000","last_active":"2024-05-08T05:26:27.397844+0000","last_peered":"2024-05-08T05:26:27.397844+0000","last_clean":"2024-05-08T05:26:27.397844+0000","last_became_active":"2024-05-08T05:25:22.643900+0000","last_became_peered":"2024-05-08T05:25:22.643900+0000","last_unstale":"2024-05-08T05:26:27.397844+0000","last_undegraded":"2024-05-08T05:26:27.397844+0000","last_fullsized":"2024-05-08T05:26:27.397844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:32.435843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142048+0000","last_change":"2024-05-08T05:25:25.670130+0000","last_active":"2024-05-08T05:26:05.142048+0000","last_peered":"2024-05-08T05:26:05.142048+0000","last_clean":"2024-05-08T05:26:05.142048+0000","last_became_active":"2024-05-08T05:25:25.669784+0000","last_became_peered":"2024-05-08T05:25:25.669784+0000","last_unstale":"2024-05-08T05:26:05.142048+0000","last_undegraded":"2024-05-08T05:26:05.142048+0000","last_fullsized":"2024-05-08T05:26:05.142048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:07.188883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.308804+0000","last_change":"2024-05-08T05:26:05.308804+0000","last_active":"2024-05-08T05:26:05.308804+0000","last_peered":"2024-05-08T05:26:05.308804+0000","last_clean":"2024-05-08T05:26:05.308804+0000","last_became_active":"2024-05-08T05:26:05.308226+0000","last_became_peered":"2024-05-08T05:26:05.308226+0000","last_unstale":"2024-05-08T05:26:05.308804+0000","last_undegraded":"2024-05-08T05:26:05.308804+0000","last_fullsized":"2024-05-08T05:26:05.308804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:33.442023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141456+0000","last_change":"2024-05-08T05:25:22.748409+0000","last_active":"2024-05-08T05:26:05.141456+0000","last_peered":"2024-05-08T05:26:05.141456+0000","last_clean":"2024-05-08T05:26:05.141456+0000","last_became_active":"2024-05-08T05:25:22.748062+0000","last_became_peered":"2024-05-08T05:25:22.748062+0000","last_unstale":"2024-05-08T05:26:05.141456+0000","last_undegraded":"2024-05-08T05:26:05.141456+0000","last_fullsized":"2024-05-08T05:26:05.141456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:42.147084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142773+0000","last_change":"2024-05-08T05:25:25.677057+0000","last_active":"2024-05-08T05:26:05.142773+0000","last_peered":"2024-05-08T05:26:05.142773+0000","last_clean":"2024-05-08T05:26:05.142773+0000","last_became_active":"2024-05-08T05:25:25.676697+0000","last_became_peered":"2024-05-08T05:25:25.676697+0000","last_unstale":"2024-05-08T05:26:05.142773+0000","last_undegraded":"2024-05-08T05:26:05.142773+0000","last_fullsized":"2024-05-08T05:26:05.142773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:40.786327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306055+0000","last_change":"2024-05-08T05:25:37.053903+0000","last_active":"2024-05-08T05:26:05.306055+0000","last_peered":"2024-05-08T05:26:05.306055+0000","last_clean":"2024-05-08T05:26:05.306055+0000","last_became_active":"2024-05-08T05:25:37.053375+0000","last_became_peered":"2024-05-08T05:25:37.053375+0000","last_unstale":"2024-05-08T05:26:05.306055+0000","last_undegraded":"2024-05-08T05:26:05.306055+0000","last_fullsized":"2024-05-08T05:26:05.306055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:19.374355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:49.450446+0000","last_change":"2024-05-08T05:25:22.646931+0000","last_active":"2024-05-08T05:26:49.450446+0000","last_peered":"2024-05-08T05:26:49.450446+0000","last_clean":"2024-05-08T05:26:49.450446+0000","last_became_active":"2024-05-08T05:25:22.646599+0000","last_became_peered":"2024-05-08T05:25:22.646599+0000","last_unstale":"2024-05-08T05:26:49.450446+0000","last_undegraded":"2024-05-08T05:26:49.450446+0000","last_fullsized":"2024-05-08T05:26:49.450446+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:55.873089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":25,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147977+0000","last_change":"2024-05-08T05:26:05.147977+0000","last_active":"2024-05-08T05:26:05.147977+0000","last_peered":"2024-05-08T05:26:05.147977+0000","last_clean":"2024-05-08T05:26:05.147977+0000","last_became_active":"2024-05-08T05:26:05.147637+0000","last_became_peered":"2024-05-08T05:26:05.147637+0000","last_unstale":"2024-05-08T05:26:05.147977+0000","last_undegraded":"2024-05-08T05:26:05.147977+0000","last_fullsized":"2024-05-08T05:26:05.147977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:59.671306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306320+0000","last_change":"2024-05-08T05:25:37.057913+0000","last_active":"2024-05-08T05:26:05.306320+0000","last_peered":"2024-05-08T05:26:05.306320+0000","last_clean":"2024-05-08T05:26:05.306320+0000","last_became_active":"2024-05-08T05:25:37.057474+0000","last_became_peered":"2024-05-08T05:25:37.057474+0000","last_unstale":"2024-05-08T05:26:05.306320+0000","last_undegraded":"2024-05-08T05:26:05.306320+0000","last_fullsized":"2024-05-08T05:26:05.306320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:19.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147331+0000","last_change":"2024-05-08T05:25:25.663594+0000","last_active":"2024-05-08T05:26:05.147331+0000","last_peered":"2024-05-08T05:26:05.147331+0000","last_clean":"2024-05-08T05:26:05.147331+0000","last_became_active":"2024-05-08T05:25:25.663275+0000","last_became_peered":"2024-05-08T05:25:25.663275+0000","last_unstale":"2024-05-08T05:26:05.147331+0000","last_undegraded":"2024-05-08T05:26:05.147331+0000","last_fullsized":"2024-05-08T05:26:05.147331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:20.581166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":70,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:50.451423+0000","last_change":"2024-05-08T05:25:22.748316+0000","last_active":"2024-05-08T05:26:50.451423+0000","last_peered":"2024-05-08T05:26:50.451423+0000","last_clean":"2024-05-08T05:26:50.451423+0000","last_became_active":"2024-05-08T05:25:22.747819+0000","last_became_peered":"2024-05-08T05:25:22.747819+0000","last_unstale":"2024-05-08T05:26:50.451423+0000","last_undegraded":"2024-05-08T05:26:50.451423+0000","last_fullsized":"2024-05-08T05:26:50.451423+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:50:42.689853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143892+0000","last_change":"2024-05-08T05:25:37.058271+0000","last_active":"2024-05-08T05:26:05.143892+0000","last_peered":"2024-05-08T05:26:05.143892+0000","last_clean":"2024-05-08T05:26:05.143892+0000","last_became_active":"2024-05-08T05:25:37.057290+0000","last_became_peered":"2024-05-08T05:25:37.057290+0000","last_unstale":"2024-05-08T05:26:05.143892+0000","last_undegraded":"2024-05-08T05:26:05.143892+0000","last_fullsized":"2024-05-08T05:26:05.143892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:47.369909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306025+0000","last_change":"2024-05-08T05:25:25.666000+0000","last_active":"2024-05-08T05:26:05.306025+0000","last_peered":"2024-05-08T05:26:05.306025+0000","last_clean":"2024-05-08T05:26:05.306025+0000","last_became_active":"2024-05-08T05:25:25.665663+0000","last_became_peered":"2024-05-08T05:25:25.665663+0000","last_unstale":"2024-05-08T05:26:05.306025+0000","last_undegraded":"2024-05-08T05:26:05.306025+0000","last_fullsized":"2024-05-08T05:26:05.306025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:44.328879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:28.419249+0000","last_change":"2024-05-08T05:25:22.648770+0000","last_active":"2024-05-08T05:26:28.419249+0000","last_peered":"2024-05-08T05:26:28.419249+0000","last_clean":"2024-05-08T05:26:28.419249+0000","last_became_active":"2024-05-08T05:25:22.648119+0000","last_became_peered":"2024-05-08T05:25:22.648119+0000","last_unstale":"2024-05-08T05:26:28.419249+0000","last_undegraded":"2024-05-08T05:26:28.419249+0000","last_fullsized":"2024-05-08T05:26:28.419249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:55.813080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143307+0000","last_change":"2024-05-08T05:23:55.820436+0000","last_active":"2024-05-08T05:26:05.143307+0000","last_peered":"2024-05-08T05:26:05.143307+0000","last_clean":"2024-05-08T05:26:05.143307+0000","last_became_active":"2024-05-08T05:23:55.814786+0000","last_became_peered":"2024-05-08T05:23:55.814786+0000","last_unstale":"2024-05-08T05:26:05.143307+0000","last_undegraded":"2024-05-08T05:26:05.143307+0000","last_fullsized":"2024-05-08T05:26:05.143307+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_clean_scrub_stamp":"2024-05-08T05:20:32.659283+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:11.169455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147318+0000","last_change":"2024-05-08T05:25:37.057287+0000","last_active":"2024-05-08T05:26:05.147318+0000","last_peered":"2024-05-08T05:26:05.147318+0000","last_clean":"2024-05-08T05:26:05.147318+0000","last_became_active":"2024-05-08T05:25:37.056871+0000","last_became_peered":"2024-05-08T05:25:37.056871+0000","last_unstale":"2024-05-08T05:26:05.147318+0000","last_undegraded":"2024-05-08T05:26:05.147318+0000","last_fullsized":"2024-05-08T05:26:05.147318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:19.004199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.305801+0000","last_change":"2024-05-08T05:25:25.664795+0000","last_active":"2024-05-08T05:26:05.305801+0000","last_peered":"2024-05-08T05:26:05.305801+0000","last_clean":"2024-05-08T05:26:05.305801+0000","last_became_active":"2024-05-08T05:25:25.664444+0000","last_became_peered":"2024-05-08T05:25:25.664444+0000","last_unstale":"2024-05-08T05:26:05.305801+0000","last_undegraded":"2024-05-08T05:26:05.305801+0000","last_fullsized":"2024-05-08T05:26:05.305801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:27.172530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:25.402073+0000","last_change":"2024-05-08T05:25:22.645205+0000","last_active":"2024-05-08T05:26:25.402073+0000","last_peered":"2024-05-08T05:26:25.402073+0000","last_clean":"2024-05-08T05:26:25.402073+0000","last_became_active":"2024-05-08T05:25:22.644720+0000","last_became_peered":"2024-05-08T05:25:22.644720+0000","last_unstale":"2024-05-08T05:26:25.402073+0000","last_undegraded":"2024-05-08T05:26:25.402073+0000","last_fullsized":"2024-05-08T05:26:25.402073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:05.630188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141854+0000","last_change":"2024-05-08T05:25:37.055151+0000","last_active":"2024-05-08T05:26:05.141854+0000","last_peered":"2024-05-08T05:26:05.141854+0000","last_clean":"2024-05-08T05:26:05.141854+0000","last_became_active":"2024-05-08T05:25:37.054737+0000","last_became_peered":"2024-05-08T05:25:37.054737+0000","last_unstale":"2024-05-08T05:26:05.141854+0000","last_undegraded":"2024-05-08T05:26:05.141854+0000","last_fullsized":"2024-05-08T05:26:05.141854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:49.135453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417886+0000","last_change":"2024-05-08T05:26:05.148665+0000","last_active":"2024-05-08T05:26:27.417886+0000","last_peered":"2024-05-08T05:26:27.417886+0000","last_clean":"2024-05-08T05:26:27.417886+0000","last_became_active":"2024-05-08T05:26:05.148283+0000","last_became_peered":"2024-05-08T05:26:05.148283+0000","last_unstale":"2024-05-08T05:26:27.417886+0000","last_undegraded":"2024-05-08T05:26:27.417886+0000","last_fullsized":"2024-05-08T05:26:27.417886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:26.197521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306567+0000","last_change":"2024-05-08T05:25:25.669130+0000","last_active":"2024-05-08T05:26:05.306567+0000","last_peered":"2024-05-08T05:26:05.306567+0000","last_clean":"2024-05-08T05:26:05.306567+0000","last_became_active":"2024-05-08T05:25:25.668812+0000","last_became_peered":"2024-05-08T05:25:25.668812+0000","last_unstale":"2024-05-08T05:26:05.306567+0000","last_undegraded":"2024-05-08T05:26:05.306567+0000","last_fullsized":"2024-05-08T05:26:05.306567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:34.514248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306134+0000","last_change":"2024-05-08T05:25:37.057146+0000","last_active":"2024-05-08T05:26:05.306134+0000","last_peered":"2024-05-08T05:26:05.306134+0000","last_clean":"2024-05-08T05:26:05.306134+0000","last_became_active":"2024-05-08T05:25:37.056706+0000","last_became_peered":"2024-05-08T05:25:37.056706+0000","last_unstale":"2024-05-08T05:26:05.306134+0000","last_undegraded":"2024-05-08T05:26:05.306134+0000","last_fullsized":"2024-05-08T05:26:05.306134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:09.724088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'15","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:48.680508+0000","last_change":"2024-05-08T05:25:22.644573+0000","last_active":"2024-05-08T05:26:48.680508+0000","last_peered":"2024-05-08T05:26:48.680508+0000","last_clean":"2024-05-08T05:26:48.680508+0000","last_became_active":"2024-05-08T05:25:22.644231+0000","last_became_peered":"2024-05-08T05:25:22.644231+0000","last_unstale":"2024-05-08T05:26:48.680508+0000","last_undegraded":"2024-05-08T05:26:48.680508+0000","last_fullsized":"2024-05-08T05:26:48.680508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:02.931823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306804+0000","last_change":"2024-05-08T05:25:25.671036+0000","last_active":"2024-05-08T05:26:05.306804+0000","last_peered":"2024-05-08T05:26:05.306804+0000","last_clean":"2024-05-08T05:26:05.306804+0000","last_became_active":"2024-05-08T05:25:25.669170+0000","last_became_peered":"2024-05-08T05:25:25.669170+0000","last_unstale":"2024-05-08T05:26:05.306804+0000","last_undegraded":"2024-05-08T05:26:05.306804+0000","last_fullsized":"2024-05-08T05:26:05.306804+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:18.963232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147050+0000","last_change":"2024-05-08T05:25:37.050656+0000","last_active":"2024-05-08T05:26:05.147050+0000","last_peered":"2024-05-08T05:26:05.147050+0000","last_clean":"2024-05-08T05:26:05.147050+0000","last_became_active":"2024-05-08T05:25:37.050051+0000","last_became_peered":"2024-05-08T05:25:37.050051+0000","last_unstale":"2024-05-08T05:26:05.147050+0000","last_undegraded":"2024-05-08T05:26:05.147050+0000","last_fullsized":"2024-05-08T05:26:05.147050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:56:07.316216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417957+0000","last_change":"2024-05-08T05:25:23.614874+0000","last_active":"2024-05-08T05:26:27.417957+0000","last_peered":"2024-05-08T05:26:27.417957+0000","last_clean":"2024-05-08T05:26:27.417957+0000","last_became_active":"2024-05-08T05:25:23.614550+0000","last_became_peered":"2024-05-08T05:25:23.614550+0000","last_unstale":"2024-05-08T05:26:27.417957+0000","last_undegraded":"2024-05-08T05:26:27.417957+0000","last_fullsized":"2024-05-08T05:26:27.417957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:10.183477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147147+0000","last_change":"2024-05-08T05:25:25.658710+0000","last_active":"2024-05-08T05:26:05.147147+0000","last_peered":"2024-05-08T05:26:05.147147+0000","last_clean":"2024-05-08T05:26:05.147147+0000","last_became_active":"2024-05-08T05:25:25.658393+0000","last_became_peered":"2024-05-08T05:25:25.658393+0000","last_unstale":"2024-05-08T05:26:05.147147+0000","last_undegraded":"2024-05-08T05:26:05.147147+0000","last_fullsized":"2024-05-08T05:26:05.147147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:02.629256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143395+0000","last_change":"2024-05-08T05:25:37.059976+0000","last_active":"2024-05-08T05:26:05.143395+0000","last_peered":"2024-05-08T05:26:05.143395+0000","last_clean":"2024-05-08T05:26:05.143395+0000","last_became_active":"2024-05-08T05:25:37.059474+0000","last_became_peered":"2024-05-08T05:25:37.059474+0000","last_unstale":"2024-05-08T05:26:05.143395+0000","last_undegraded":"2024-05-08T05:26:05.143395+0000","last_fullsized":"2024-05-08T05:26:05.143395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:51.946242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:24.398321+0000","last_change":"2024-05-08T05:25:22.748271+0000","last_active":"2024-05-08T05:26:24.398321+0000","last_peered":"2024-05-08T05:26:24.398321+0000","last_clean":"2024-05-08T05:26:24.398321+0000","last_became_active":"2024-05-08T05:25:22.747915+0000","last_became_peered":"2024-05-08T05:25:22.747915+0000","last_unstale":"2024-05-08T05:26:24.398321+0000","last_undegraded":"2024-05-08T05:26:24.398321+0000","last_fullsized":"2024-05-08T05:26:24.398321+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:35.262298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143012+0000","last_change":"2024-05-08T05:25:25.668722+0000","last_active":"2024-05-08T05:26:05.143012+0000","last_peered":"2024-05-08T05:26:05.143012+0000","last_clean":"2024-05-08T05:26:05.143012+0000","last_became_active":"2024-05-08T05:25:25.668223+0000","last_became_peered":"2024-05-08T05:25:25.668223+0000","last_unstale":"2024-05-08T05:26:05.143012+0000","last_undegraded":"2024-05-08T05:26:05.143012+0000","last_fullsized":"2024-05-08T05:26:05.143012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:43.511567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141728+0000","last_change":"2024-05-08T05:25:37.056131+0000","last_active":"2024-05-08T05:26:05.141728+0000","last_peered":"2024-05-08T05:26:05.141728+0000","last_clean":"2024-05-08T05:26:05.141728+0000","last_became_active":"2024-05-08T05:25:37.055424+0000","last_became_peered":"2024-05-08T05:25:37.055424+0000","last_unstale":"2024-05-08T05:26:05.141728+0000","last_undegraded":"2024-05-08T05:26:05.141728+0000","last_fullsized":"2024-05-08T05:26:05.141728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:43.279408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417490+0000","last_change":"2024-05-08T05:25:22.643908+0000","last_active":"2024-05-08T05:26:27.417490+0000","last_peered":"2024-05-08T05:26:27.417490+0000","last_clean":"2024-05-08T05:26:27.417490+0000","last_became_active":"2024-05-08T05:25:22.643542+0000","last_became_peered":"2024-05-08T05:25:22.643542+0000","last_unstale":"2024-05-08T05:26:27.417490+0000","last_undegraded":"2024-05-08T05:26:27.417490+0000","last_fullsized":"2024-05-08T05:26:27.417490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:01.671285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141649+0000","last_change":"2024-05-08T05:25:25.667843+0000","last_active":"2024-05-08T05:26:05.141649+0000","last_peered":"2024-05-08T05:26:05.141649+0000","last_clean":"2024-05-08T05:26:05.141649+0000","last_became_active":"2024-05-08T05:25:25.667462+0000","last_became_peered":"2024-05-08T05:25:25.667462+0000","last_unstale":"2024-05-08T05:26:05.141649+0000","last_undegraded":"2024-05-08T05:26:05.141649+0000","last_fullsized":"2024-05-08T05:26:05.141649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:51.325104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144596+0000","last_change":"2024-05-08T05:25:37.053017+0000","last_active":"2024-05-08T05:26:05.144596+0000","last_peered":"2024-05-08T05:26:05.144596+0000","last_clean":"2024-05-08T05:26:05.144596+0000","last_became_active":"2024-05-08T05:25:37.052601+0000","last_became_peered":"2024-05-08T05:25:37.052601+0000","last_unstale":"2024-05-08T05:26:05.144596+0000","last_undegraded":"2024-05-08T05:26:05.144596+0000","last_fullsized":"2024-05-08T05:26:05.144596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:35.393835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:26.410273+0000","last_change":"2024-05-08T05:25:22.644564+0000","last_active":"2024-05-08T05:26:26.410273+0000","last_peered":"2024-05-08T05:26:26.410273+0000","last_clean":"2024-05-08T05:26:26.410273+0000","last_became_active":"2024-05-08T05:25:22.644170+0000","last_became_peered":"2024-05-08T05:25:22.644170+0000","last_unstale":"2024-05-08T05:26:26.410273+0000","last_undegraded":"2024-05-08T05:26:26.410273+0000","last_fullsized":"2024-05-08T05:26:26.410273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:24.757574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306612+0000","last_change":"2024-05-08T05:25:25.676983+0000","last_active":"2024-05-08T05:26:05.306612+0000","last_peered":"2024-05-08T05:26:05.306612+0000","last_clean":"2024-05-08T05:26:05.306612+0000","last_became_active":"2024-05-08T05:25:25.676610+0000","last_became_peered":"2024-05-08T05:25:25.676610+0000","last_unstale":"2024-05-08T05:26:05.306612+0000","last_undegraded":"2024-05-08T05:26:05.306612+0000","last_fullsized":"2024-05-08T05:26:05.306612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:39.137935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143383+0000","last_change":"2024-05-08T05:25:37.059381+0000","last_active":"2024-05-08T05:26:05.143383+0000","last_peered":"2024-05-08T05:26:05.143383+0000","last_clean":"2024-05-08T05:26:05.143383+0000","last_became_active":"2024-05-08T05:25:37.056851+0000","last_became_peered":"2024-05-08T05:25:37.056851+0000","last_unstale":"2024-05-08T05:26:05.143383+0000","last_undegraded":"2024-05-08T05:26:05.143383+0000","last_fullsized":"2024-05-08T05:26:05.143383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:19.339331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:24.396255+0000","last_change":"2024-05-08T05:25:22.648099+0000","last_active":"2024-05-08T05:26:24.396255+0000","last_peered":"2024-05-08T05:26:24.396255+0000","last_clean":"2024-05-08T05:26:24.396255+0000","last_became_active":"2024-05-08T05:25:22.647736+0000","last_became_peered":"2024-05-08T05:25:22.647736+0000","last_unstale":"2024-05-08T05:26:24.396255+0000","last_undegraded":"2024-05-08T05:26:24.396255+0000","last_fullsized":"2024-05-08T05:26:24.396255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:12.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144712+0000","last_change":"2024-05-08T05:25:25.667565+0000","last_active":"2024-05-08T05:26:05.144712+0000","last_peered":"2024-05-08T05:26:05.144712+0000","last_clean":"2024-05-08T05:26:05.144712+0000","last_became_active":"2024-05-08T05:25:25.667217+0000","last_became_peered":"2024-05-08T05:25:25.667217+0000","last_unstale":"2024-05-08T05:26:05.144712+0000","last_undegraded":"2024-05-08T05:26:05.144712+0000","last_fullsized":"2024-05-08T05:26:05.144712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:05.228227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306478+0000","last_change":"2024-05-08T05:25:37.056923+0000","last_active":"2024-05-08T05:26:05.306478+0000","last_peered":"2024-05-08T05:26:05.306478+0000","last_clean":"2024-05-08T05:26:05.306478+0000","last_became_active":"2024-05-08T05:25:37.056496+0000","last_became_peered":"2024-05-08T05:25:37.056496+0000","last_unstale":"2024-05-08T05:26:05.306478+0000","last_undegraded":"2024-05-08T05:26:05.306478+0000","last_fullsized":"2024-05-08T05:26:05.306478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:21.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":71,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:49.451042+0000","last_change":"2024-05-08T05:25:23.614900+0000","last_active":"2024-05-08T05:26:49.451042+0000","last_peered":"2024-05-08T05:26:49.451042+0000","last_clean":"2024-05-08T05:26:49.451042+0000","last_became_active":"2024-05-08T05:25:23.614510+0000","last_became_peered":"2024-05-08T05:25:23.614510+0000","last_unstale":"2024-05-08T05:26:49.451042+0000","last_undegraded":"2024-05-08T05:26:49.451042+0000","last_fullsized":"2024-05-08T05:26:49.451042+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:19.539656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":26,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144199+0000","last_change":"2024-05-08T05:25:25.669016+0000","last_active":"2024-05-08T05:26:05.144199+0000","last_peered":"2024-05-08T05:26:05.144199+0000","last_clean":"2024-05-08T05:26:05.144199+0000","last_became_active":"2024-05-08T05:25:25.667809+0000","last_became_peered":"2024-05-08T05:25:25.667809+0000","last_unstale":"2024-05-08T05:26:05.144199+0000","last_undegraded":"2024-05-08T05:26:05.144199+0000","last_fullsized":"2024-05-08T05:26:05.144199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:56.475622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146478+0000","last_change":"2024-05-08T05:25:37.053630+0000","last_active":"2024-05-08T05:26:05.146478+0000","last_peered":"2024-05-08T05:26:05.146478+0000","last_clean":"2024-05-08T05:26:05.146478+0000","last_became_active":"2024-05-08T05:25:37.053069+0000","last_became_peered":"2024-05-08T05:25:37.053069+0000","last_unstale":"2024-05-08T05:26:05.146478+0000","last_undegraded":"2024-05-08T05:26:05.146478+0000","last_fullsized":"2024-05-08T05:26:05.146478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:49:56.693447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.415813+0000","last_change":"2024-05-08T05:25:22.645284+0000","last_active":"2024-05-08T05:26:27.415813+0000","last_peered":"2024-05-08T05:26:27.415813+0000","last_clean":"2024-05-08T05:26:27.415813+0000","last_became_active":"2024-05-08T05:25:22.644897+0000","last_became_peered":"2024-05-08T05:25:22.644897+0000","last_unstale":"2024-05-08T05:26:27.415813+0000","last_undegraded":"2024-05-08T05:26:27.415813+0000","last_fullsized":"2024-05-08T05:26:27.415813+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:53:31.423412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144032+0000","last_change":"2024-05-08T05:25:25.664930+0000","last_active":"2024-05-08T05:26:05.144032+0000","last_peered":"2024-05-08T05:26:05.144032+0000","last_clean":"2024-05-08T05:26:05.144032+0000","last_became_active":"2024-05-08T05:25:25.664395+0000","last_became_peered":"2024-05-08T05:25:25.664395+0000","last_unstale":"2024-05-08T05:26:05.144032+0000","last_undegraded":"2024-05-08T05:26:05.144032+0000","last_fullsized":"2024-05-08T05:26:05.144032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:37.971474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146841+0000","last_change":"2024-05-08T05:25:37.053258+0000","last_active":"2024-05-08T05:26:05.146841+0000","last_peered":"2024-05-08T05:26:05.146841+0000","last_clean":"2024-05-08T05:26:05.146841+0000","last_became_active":"2024-05-08T05:25:37.052786+0000","last_became_peered":"2024-05-08T05:25:37.052786+0000","last_unstale":"2024-05-08T05:26:05.146841+0000","last_undegraded":"2024-05-08T05:26:05.146841+0000","last_fullsized":"2024-05-08T05:26:05.146841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:07.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.416807+0000","last_change":"2024-05-08T05:25:22.642619+0000","last_active":"2024-05-08T05:26:27.416807+0000","last_peered":"2024-05-08T05:26:27.416807+0000","last_clean":"2024-05-08T05:26:27.416807+0000","last_became_active":"2024-05-08T05:25:22.642237+0000","last_became_peered":"2024-05-08T05:25:22.642237+0000","last_unstale":"2024-05-08T05:26:27.416807+0000","last_undegraded":"2024-05-08T05:26:27.416807+0000","last_fullsized":"2024-05-08T05:26:27.416807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:03:21.133179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.145164+0000","last_change":"2024-05-08T05:25:25.663771+0000","last_active":"2024-05-08T05:26:05.145164+0000","last_peered":"2024-05-08T05:26:05.145164+0000","last_clean":"2024-05-08T05:26:05.145164+0000","last_became_active":"2024-05-08T05:25:25.663281+0000","last_became_peered":"2024-05-08T05:25:25.663281+0000","last_unstale":"2024-05-08T05:26:05.145164+0000","last_undegraded":"2024-05-08T05:26:05.145164+0000","last_fullsized":"2024-05-08T05:26:05.145164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:43.892781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141801+0000","last_change":"2024-05-08T05:25:25.665446+0000","last_active":"2024-05-08T05:26:05.141801+0000","last_peered":"2024-05-08T05:26:05.141801+0000","last_clean":"2024-05-08T05:26:05.141801+0000","last_became_active":"2024-05-08T05:25:25.664389+0000","last_became_peered":"2024-05-08T05:25:25.664389+0000","last_unstale":"2024-05-08T05:26:05.141801+0000","last_undegraded":"2024-05-08T05:26:05.141801+0000","last_fullsized":"2024-05-08T05:26:05.141801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.149870+0000","last_change":"2024-05-08T05:26:05.149870+0000","last_active":"2024-05-08T05:26:05.149870+0000","last_peered":"2024-05-08T05:26:05.149870+0000","last_clean":"2024-05-08T05:26:05.149870+0000","last_became_active":"2024-05-08T05:26:05.149476+0000","last_became_peered":"2024-05-08T05:26:05.149476+0000","last_unstale":"2024-05-08T05:26:05.149870+0000","last_undegraded":"2024-05-08T05:26:05.149870+0000","last_fullsized":"2024-05-08T05:26:05.149870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:40:43.401832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146604+0000","last_change":"2024-05-08T05:25:25.667836+0000","last_active":"2024-05-08T05:26:05.146604+0000","last_peered":"2024-05-08T05:26:05.146604+0000","last_clean":"2024-05-08T05:26:05.146604+0000","last_became_active":"2024-05-08T05:25:25.667496+0000","last_became_peered":"2024-05-08T05:25:25.667496+0000","last_unstale":"2024-05-08T05:26:05.146604+0000","last_undegraded":"2024-05-08T05:26:05.146604+0000","last_fullsized":"2024-05-08T05:26:05.146604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:01.482837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146732+0000","last_change":"2024-05-08T05:25:25.667279+0000","last_active":"2024-05-08T05:26:05.146732+0000","last_peered":"2024-05-08T05:26:05.146732+0000","last_clean":"2024-05-08T05:26:05.146732+0000","last_became_active":"2024-05-08T05:25:25.666960+0000","last_became_peered":"2024-05-08T05:25:25.666960+0000","last_unstale":"2024-05-08T05:26:05.146732+0000","last_undegraded":"2024-05-08T05:26:05.146732+0000","last_fullsized":"2024-05-08T05:26:05.146732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:06.335850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306217+0000","last_change":"2024-05-08T05:25:25.667644+0000","last_active":"2024-05-08T05:26:05.306217+0000","last_peered":"2024-05-08T05:26:05.306217+0000","last_clean":"2024-05-08T05:26:05.306217+0000","last_became_active":"2024-05-08T05:25:25.667133+0000","last_became_peered":"2024-05-08T05:25:25.667133+0000","last_unstale":"2024-05-08T05:26:05.306217+0000","last_undegraded":"2024-05-08T05:26:05.306217+0000","last_fullsized":"2024-05-08T05:26:05.306217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:03.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142612+0000","last_change":"2024-05-08T05:25:25.670761+0000","last_active":"2024-05-08T05:26:05.142612+0000","last_peered":"2024-05-08T05:26:05.142612+0000","last_clean":"2024-05-08T05:26:05.142612+0000","last_became_active":"2024-05-08T05:25:25.670342+0000","last_became_peered":"2024-05-08T05:25:25.670342+0000","last_unstale":"2024-05-08T05:26:05.142612+0000","last_undegraded":"2024-05-08T05:26:05.142612+0000","last_fullsized":"2024-05-08T05:26:05.142612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:08.069332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146945+0000","last_change":"2024-05-08T05:25:25.664512+0000","last_active":"2024-05-08T05:26:05.146945+0000","last_peered":"2024-05-08T05:26:05.146945+0000","last_clean":"2024-05-08T05:26:05.146945+0000","last_became_active":"2024-05-08T05:25:25.664191+0000","last_became_peered":"2024-05-08T05:25:25.664191+0000","last_unstale":"2024-05-08T05:26:05.146945+0000","last_undegraded":"2024-05-08T05:26:05.146945+0000","last_fullsized":"2024-05-08T05:26:05.146945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:07.586637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306937+0000","last_change":"2024-05-08T05:25:25.675711+0000","last_active":"2024-05-08T05:26:05.306937+0000","last_peered":"2024-05-08T05:26:05.306937+0000","last_clean":"2024-05-08T05:26:05.306937+0000","last_became_active":"2024-05-08T05:25:25.671890+0000","last_became_peered":"2024-05-08T05:25:25.671890+0000","last_unstale":"2024-05-08T05:26:05.306937+0000","last_undegraded":"2024-05-08T05:26:05.306937+0000","last_fullsized":"2024-05-08T05:26:05.306937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:57:49.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147151+0000","last_change":"2024-05-08T05:25:25.662603+0000","last_active":"2024-05-08T05:26:05.147151+0000","last_peered":"2024-05-08T05:26:05.147151+0000","last_clean":"2024-05-08T05:26:05.147151+0000","last_became_active":"2024-05-08T05:25:25.662287+0000","last_became_peered":"2024-05-08T05:25:25.662287+0000","last_unstale":"2024-05-08T05:26:05.147151+0000","last_undegraded":"2024-05-08T05:26:05.147151+0000","last_fullsized":"2024-05-08T05:26:05.147151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:02.152557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306376+0000","last_change":"2024-05-08T05:25:25.667309+0000","last_active":"2024-05-08T05:26:05.306376+0000","last_peered":"2024-05-08T05:26:05.306376+0000","last_clean":"2024-05-08T05:26:05.306376+0000","last_became_active":"2024-05-08T05:25:25.666968+0000","last_became_peered":"2024-05-08T05:25:25.666968+0000","last_unstale":"2024-05-08T05:26:05.306376+0000","last_undegraded":"2024-05-08T05:26:05.306376+0000","last_fullsized":"2024-05-08T05:26:05.306376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:28.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306760+0000","last_change":"2024-05-08T05:25:25.673579+0000","last_active":"2024-05-08T05:26:05.306760+0000","last_peered":"2024-05-08T05:26:05.306760+0000","last_clean":"2024-05-08T05:26:05.306760+0000","last_became_active":"2024-05-08T05:25:25.673130+0000","last_became_peered":"2024-05-08T05:25:25.673130+0000","last_unstale":"2024-05-08T05:26:05.306760+0000","last_undegraded":"2024-05-08T05:26:05.306760+0000","last_fullsized":"2024-05-08T05:26:05.306760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:01.291319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306227+0000","last_change":"2024-05-08T05:25:25.667359+0000","last_active":"2024-05-08T05:26:05.306227+0000","last_peered":"2024-05-08T05:26:05.306227+0000","last_clean":"2024-05-08T05:26:05.306227+0000","last_became_active":"2024-05-08T05:25:25.667044+0000","last_became_peered":"2024-05-08T05:25:25.667044+0000","last_unstale":"2024-05-08T05:26:05.306227+0000","last_undegraded":"2024-05-08T05:26:05.306227+0000","last_fullsized":"2024-05-08T05:26:05.306227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:54.973053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.150407+0000","last_change":"2024-05-08T05:26:05.150407+0000","last_active":"2024-05-08T05:26:05.150407+0000","last_peered":"2024-05-08T05:26:05.150407+0000","last_clean":"2024-05-08T05:26:05.150407+0000","last_became_active":"2024-05-08T05:26:05.149916+0000","last_became_peered":"2024-05-08T05:26:05.149916+0000","last_unstale":"2024-05-08T05:26:05.150407+0000","last_undegraded":"2024-05-08T05:26:05.150407+0000","last_fullsized":"2024-05-08T05:26:05.150407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:05:57.075928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247352,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68366,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":283,"num_read_kb":222,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":202822,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678048,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":502749,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874278,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":1149833,"data_compressed":62314,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037804,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":506272,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234033,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":502442,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462964,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1099833,"data_compressed":62314,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626491,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":498071,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":558343,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":499527,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":558842,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":497388,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986245,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":502257,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822719,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1094633,"data_compressed":62314,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":61385,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":61884,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:26:56.030 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:26:56.290 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:56.740 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:56 smithi059 ceph-mon[30304]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:26:56.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:56 smithi143 ceph-mon[34337]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:26:56.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:56 smithi096 ceph-mon[34253]: from='client.14799 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:26:57.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:57 smithi143 ceph-mon[34337]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T05:26:57.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:57 smithi096 ceph-mon[34253]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T05:26:57.808 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:57 smithi059 ceph-mon[30304]: pgmap v344: 129 pgs: 129 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-05-08T05:26:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.040+0000 7f7ead121700 1 -- 172.21.15.59:0/1893241118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 msgr2=0x7f7ea8107020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.040+0000 7f7ead121700 1 --2- 172.21.15.59:0/1893241118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 0x7f7ea8107020 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c0099c0 tx=0x7f7e9c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.040+0000 7f7ead121700 1 -- 172.21.15.59:0/1893241118 shutdown_connections 2024-05-08T05:26:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.040+0000 7f7ead121700 1 --2- 172.21.15.59:0/1893241118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ea8107700 0x7f7ea810f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.040+0000 7f7ead121700 1 --2- 172.21.15.59:0/1893241118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ea8102620 0x7f7ea8102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.040+0000 7f7ead121700 1 --2- 172.21.15.59:0/1893241118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 0x7f7ea8107020 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.040+0000 7f7ead121700 1 -- 172.21.15.59:0/1893241118 >> 172.21.15.59:0/1893241118 conn(0x7f7ea8079440 msgr2=0x7f7ea8079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.040+0000 7f7ead121700 1 -- 172.21.15.59:0/1893241118 shutdown_connections 2024-05-08T05:26:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.040+0000 7f7ead121700 1 -- 172.21.15.59:0/1893241118 wait complete. 2024-05-08T05:26:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ead121700 1 Processor -- start 2024-05-08T05:26:58.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ead121700 1 -- start start 2024-05-08T05:26:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ead121700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ea8102620 0x7f7ea819ab10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:58.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ead121700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 0x7f7ea819b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ead121700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ea8107700 0x7f7ea81a5b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ead121700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea8112670 con 0x7f7ea8102fd0 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ead121700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea81124f0 con 0x7f7ea8107700 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ead121700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7ea81127f0 con 0x7f7ea8102620 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ea659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 0x7f7ea819b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ea659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 0x7f7ea819b050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35484/0 (socket says 172.21.15.59:35484) 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.041+0000 7f7ea659c700 1 -- 172.21.15.59:0/817762668 learned_addr learned my addr 172.21.15.59:0/817762668 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea759e700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ea8107700 0x7f7ea81a5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea6d9d700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ea8102620 0x7f7ea819ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea659c700 1 -- 172.21.15.59:0/817762668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ea8102620 msgr2=0x7f7ea819ab10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea659c700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ea8102620 0x7f7ea819ab10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea659c700 1 -- 172.21.15.59:0/817762668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ea8107700 msgr2=0x7f7ea81a5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea659c700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ea8107700 0x7f7ea81a5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea659c700 1 -- 172.21.15.59:0/817762668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ea81a60d0 con 0x7f7ea8102fd0 2024-05-08T05:26:58.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea759e700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ea8107700 0x7f7ea81a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:26:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea6d9d700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ea8102620 0x7f7ea819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:26:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ea659c700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 0x7f7ea819b050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009f50 tx=0x7f7e9c0037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7e8ffff700 1 -- 172.21.15.59:0/817762668 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c027070 con 0x7f7ea8102fd0 2024-05-08T05:26:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ea81a62a0 con 0x7f7ea8102fd0 2024-05-08T05:26:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7e8ffff700 1 -- 172.21.15.59:0/817762668 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7e9c004140 con 0x7f7ea8102fd0 2024-05-08T05:26:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7e8ffff700 1 -- 172.21.15.59:0/817762668 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7e9c00f460 con 0x7f7ea8102fd0 2024-05-08T05:26:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.042+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ea81a66f0 con 0x7f7ea8102fd0 2024-05-08T05:26:58.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.043+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ea806a190 con 0x7f7ea8102fd0 2024-05-08T05:26:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.045+0000 7f7e8ffff700 1 -- 172.21.15.59:0/817762668 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7e9c02a070 con 0x7f7ea8102fd0 2024-05-08T05:26:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.045+0000 7f7e8ffff700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7e900b2910 0x7f7e900b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:26:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.046+0000 7f7ea6d9d700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7e900b2910 0x7f7e900b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:26:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.046+0000 7f7e8ffff700 1 -- 172.21.15.59:0/817762668 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f7e9c0e3be0 con 0x7f7ea8102fd0 2024-05-08T05:26:58.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.046+0000 7f7ea6d9d700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7e900b2910 0x7f7e900b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e98007900 tx=0x7f7e98008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:26:58.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.049+0000 7f7e8ffff700 1 -- 172.21.15.59:0/817762668 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f7e9c031070 con 0x7f7ea8102fd0 2024-05-08T05:26:58.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.264+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f7ea80638e0 con 0x7f7e900b2910 2024-05-08T05:26:58.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.274+0000 7f7e8ffff700 1 -- 172.21.15.59:0/817762668 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340640 (secure 0 0 0) 0x7f7ea80638e0 con 0x7f7e900b2910 2024-05-08T05:26:58.278 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.278+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7e900b2910 msgr2=0x7f7e900b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.278+0000 7f7ead121700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7e900b2910 0x7f7e900b4dd0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7e98007900 tx=0x7f7e98008040 comp rx=0 tx=0).stop 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.278+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 msgr2=0x7f7ea819b050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.278+0000 7f7ead121700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 0x7f7ea819b050 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7e9c009f50 tx=0x7f7e9c0037c0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.278+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 shutdown_connections 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.278+0000 7f7ead121700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f7e900b2910 0x7f7e900b4dd0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.279+0000 7f7ead121700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7ea8107700 0x7f7ea81a5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.279+0000 7f7ead121700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7ea8102620 0x7f7ea819ab10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.279+0000 7f7ead121700 1 --2- 172.21.15.59:0/817762668 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7ea8102fd0 0x7f7ea819b050 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.279+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 >> 172.21.15.59:0/817762668 conn(0x7f7ea8079440 msgr2=0x7f7ea81088d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.279+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 shutdown_connections 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:26:58.279+0000 7f7ead121700 1 -- 172.21.15.59:0/817762668 wait complete. 2024-05-08T05:26:58.281 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T05:26:59.001 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":345,"stamp":"2024-05-08T05:26:58.002817+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658968,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":389,"num_read_kb":296,"num_write":375,"num_write_kb":2293,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":14,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":309,"ondisk_log_size":309,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":356952,"kb_used_data":11960,"kb_used_omap":18,"kb_used_meta":344877,"kb_avail":1124584872,"statfs":{"total":1151940427776,"available":1151574908928,"internally_reserved":0,"allocated":12247040,"data_stored":7972466,"data_compressed":425568,"data_compressed_allocated":577536,"data_compressed_original":2654208,"omap_allocated":19141,"internal_metadata":353154363},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":15901,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":108,"num_read_kb":86,"num_write":14,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003392"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142305+0000","last_change":"2024-05-08T05:25:25.667895+0000","last_active":"2024-05-08T05:26:05.142305+0000","last_peered":"2024-05-08T05:26:05.142305+0000","last_clean":"2024-05-08T05:26:05.142305+0000","last_became_active":"2024-05-08T05:25:25.667563+0000","last_became_peered":"2024-05-08T05:25:25.667563+0000","last_unstale":"2024-05-08T05:26:05.142305+0000","last_undegraded":"2024-05-08T05:26:05.142305+0000","last_fullsized":"2024-05-08T05:26:05.142305+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:47.677082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147060+0000","last_change":"2024-05-08T05:25:25.673346+0000","last_active":"2024-05-08T05:26:05.147060+0000","last_peered":"2024-05-08T05:26:05.147060+0000","last_clean":"2024-05-08T05:26:05.147060+0000","last_became_active":"2024-05-08T05:25:25.673011+0000","last_became_peered":"2024-05-08T05:25:25.673011+0000","last_unstale":"2024-05-08T05:26:05.147060+0000","last_undegraded":"2024-05-08T05:26:05.147060+0000","last_fullsized":"2024-05-08T05:26:05.147060+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:21:05.091177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146886+0000","last_change":"2024-05-08T05:25:25.670469+0000","last_active":"2024-05-08T05:26:05.146886+0000","last_peered":"2024-05-08T05:26:05.146886+0000","last_clean":"2024-05-08T05:26:05.146886+0000","last_became_active":"2024-05-08T05:25:25.669700+0000","last_became_peered":"2024-05-08T05:25:25.669700+0000","last_unstale":"2024-05-08T05:26:05.146886+0000","last_undegraded":"2024-05-08T05:26:05.146886+0000","last_fullsized":"2024-05-08T05:26:05.146886+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:06.046633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.145008+0000","last_change":"2024-05-08T05:25:25.659937+0000","last_active":"2024-05-08T05:26:05.145008+0000","last_peered":"2024-05-08T05:26:05.145008+0000","last_clean":"2024-05-08T05:26:05.145008+0000","last_became_active":"2024-05-08T05:25:25.658892+0000","last_became_peered":"2024-05-08T05:25:25.658892+0000","last_unstale":"2024-05-08T05:26:05.145008+0000","last_undegraded":"2024-05-08T05:26:05.145008+0000","last_fullsized":"2024-05-08T05:26:05.145008+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:10.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143347+0000","last_change":"2024-05-08T05:25:25.669132+0000","last_active":"2024-05-08T05:26:05.143347+0000","last_peered":"2024-05-08T05:26:05.143347+0000","last_clean":"2024-05-08T05:26:05.143347+0000","last_became_active":"2024-05-08T05:25:25.668556+0000","last_became_peered":"2024-05-08T05:25:25.668556+0000","last_unstale":"2024-05-08T05:26:05.143347+0000","last_undegraded":"2024-05-08T05:26:05.143347+0000","last_fullsized":"2024-05-08T05:26:05.143347+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:28.443763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143577+0000","last_change":"2024-05-08T05:25:25.668978+0000","last_active":"2024-05-08T05:26:05.143577+0000","last_peered":"2024-05-08T05:26:05.143577+0000","last_clean":"2024-05-08T05:26:05.143577+0000","last_became_active":"2024-05-08T05:25:25.668647+0000","last_became_peered":"2024-05-08T05:25:25.668647+0000","last_unstale":"2024-05-08T05:26:05.143577+0000","last_undegraded":"2024-05-08T05:26:05.143577+0000","last_fullsized":"2024-05-08T05:26:05.143577+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:07:00.962620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143778+0000","last_change":"2024-05-08T05:25:25.664784+0000","last_active":"2024-05-08T05:26:05.143778+0000","last_peered":"2024-05-08T05:26:05.143778+0000","last_clean":"2024-05-08T05:26:05.143778+0000","last_became_active":"2024-05-08T05:25:25.664390+0000","last_became_peered":"2024-05-08T05:25:25.664390+0000","last_unstale":"2024-05-08T05:26:05.143778+0000","last_undegraded":"2024-05-08T05:26:05.143778+0000","last_fullsized":"2024-05-08T05:26:05.143778+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:08:35.358796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306345+0000","last_change":"2024-05-08T05:25:25.677139+0000","last_active":"2024-05-08T05:26:05.306345+0000","last_peered":"2024-05-08T05:26:05.306345+0000","last_clean":"2024-05-08T05:26:05.306345+0000","last_became_active":"2024-05-08T05:25:25.676444+0000","last_became_peered":"2024-05-08T05:25:25.676444+0000","last_unstale":"2024-05-08T05:26:05.306345+0000","last_undegraded":"2024-05-08T05:26:05.306345+0000","last_fullsized":"2024-05-08T05:26:05.306345+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:56.704152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.307286+0000","last_change":"2024-05-08T05:25:25.675593+0000","last_active":"2024-05-08T05:26:05.307286+0000","last_peered":"2024-05-08T05:26:05.307286+0000","last_clean":"2024-05-08T05:26:05.307286+0000","last_became_active":"2024-05-08T05:25:25.675136+0000","last_became_peered":"2024-05-08T05:25:25.675136+0000","last_unstale":"2024-05-08T05:26:05.307286+0000","last_undegraded":"2024-05-08T05:26:05.307286+0000","last_fullsized":"2024-05-08T05:26:05.307286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:49.042956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143328+0000","last_change":"2024-05-08T05:25:25.669384+0000","last_active":"2024-05-08T05:26:05.143328+0000","last_peered":"2024-05-08T05:26:05.143328+0000","last_clean":"2024-05-08T05:26:05.143328+0000","last_became_active":"2024-05-08T05:25:25.668861+0000","last_became_peered":"2024-05-08T05:25:25.668861+0000","last_unstale":"2024-05-08T05:26:05.143328+0000","last_undegraded":"2024-05-08T05:26:05.143328+0000","last_fullsized":"2024-05-08T05:26:05.143328+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:57.491242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144002+0000","last_change":"2024-05-08T05:25:25.669463+0000","last_active":"2024-05-08T05:26:05.144002+0000","last_peered":"2024-05-08T05:26:05.144002+0000","last_clean":"2024-05-08T05:26:05.144002+0000","last_became_active":"2024-05-08T05:25:25.668672+0000","last_became_peered":"2024-05-08T05:25:25.668672+0000","last_unstale":"2024-05-08T05:26:05.144002+0000","last_undegraded":"2024-05-08T05:26:05.144002+0000","last_fullsized":"2024-05-08T05:26:05.144002+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:30.779473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144902+0000","last_change":"2024-05-08T05:25:25.670378+0000","last_active":"2024-05-08T05:26:05.144902+0000","last_peered":"2024-05-08T05:26:05.144902+0000","last_clean":"2024-05-08T05:26:05.144902+0000","last_became_active":"2024-05-08T05:25:25.669801+0000","last_became_peered":"2024-05-08T05:25:25.669801+0000","last_unstale":"2024-05-08T05:26:05.144902+0000","last_undegraded":"2024-05-08T05:26:05.144902+0000","last_fullsized":"2024-05-08T05:26:05.144902+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:49.614163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306176+0000","last_change":"2024-05-08T05:25:25.670874+0000","last_active":"2024-05-08T05:26:05.306176+0000","last_peered":"2024-05-08T05:26:05.306176+0000","last_clean":"2024-05-08T05:26:05.306176+0000","last_became_active":"2024-05-08T05:25:25.670557+0000","last_became_peered":"2024-05-08T05:25:25.670557+0000","last_unstale":"2024-05-08T05:26:05.306176+0000","last_undegraded":"2024-05-08T05:26:05.306176+0000","last_fullsized":"2024-05-08T05:26:05.306176+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:58.594223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146585+0000","last_change":"2024-05-08T05:25:25.670015+0000","last_active":"2024-05-08T05:26:05.146585+0000","last_peered":"2024-05-08T05:26:05.146585+0000","last_clean":"2024-05-08T05:26:05.146585+0000","last_became_active":"2024-05-08T05:25:25.669493+0000","last_became_peered":"2024-05-08T05:25:25.669493+0000","last_unstale":"2024-05-08T05:26:05.146585+0000","last_undegraded":"2024-05-08T05:26:05.146585+0000","last_fullsized":"2024-05-08T05:26:05.146585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:49.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146559+0000","last_change":"2024-05-08T05:25:25.659651+0000","last_active":"2024-05-08T05:26:05.146559+0000","last_peered":"2024-05-08T05:26:05.146559+0000","last_clean":"2024-05-08T05:26:05.146559+0000","last_became_active":"2024-05-08T05:25:25.659308+0000","last_became_peered":"2024-05-08T05:25:25.659308+0000","last_unstale":"2024-05-08T05:26:05.146559+0000","last_undegraded":"2024-05-08T05:26:05.146559+0000","last_fullsized":"2024-05-08T05:26:05.146559+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:49.945474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306249+0000","last_change":"2024-05-08T05:25:25.663696+0000","last_active":"2024-05-08T05:26:05.306249+0000","last_peered":"2024-05-08T05:26:05.306249+0000","last_clean":"2024-05-08T05:26:05.306249+0000","last_became_active":"2024-05-08T05:25:25.662590+0000","last_became_peered":"2024-05-08T05:25:25.662590+0000","last_unstale":"2024-05-08T05:26:05.306249+0000","last_undegraded":"2024-05-08T05:26:05.306249+0000","last_fullsized":"2024-05-08T05:26:05.306249+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:29:21.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142073+0000","last_change":"2024-05-08T05:25:25.668552+0000","last_active":"2024-05-08T05:26:05.142073+0000","last_peered":"2024-05-08T05:26:05.142073+0000","last_clean":"2024-05-08T05:26:05.142073+0000","last_became_active":"2024-05-08T05:25:25.668185+0000","last_became_peered":"2024-05-08T05:25:25.668185+0000","last_unstale":"2024-05-08T05:26:05.142073+0000","last_undegraded":"2024-05-08T05:26:05.142073+0000","last_fullsized":"2024-05-08T05:26:05.142073+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:25.762909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141657+0000","last_change":"2024-05-08T05:25:25.668259+0000","last_active":"2024-05-08T05:26:05.141657+0000","last_peered":"2024-05-08T05:26:05.141657+0000","last_clean":"2024-05-08T05:26:05.141657+0000","last_became_active":"2024-05-08T05:25:25.667767+0000","last_became_peered":"2024-05-08T05:25:25.667767+0000","last_unstale":"2024-05-08T05:26:05.141657+0000","last_undegraded":"2024-05-08T05:26:05.141657+0000","last_fullsized":"2024-05-08T05:26:05.141657+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:36:34.984139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147777+0000","last_change":"2024-05-08T05:25:25.663381+0000","last_active":"2024-05-08T05:26:05.147777+0000","last_peered":"2024-05-08T05:26:05.147777+0000","last_clean":"2024-05-08T05:26:05.147777+0000","last_became_active":"2024-05-08T05:25:25.663019+0000","last_became_peered":"2024-05-08T05:25:25.663019+0000","last_unstale":"2024-05-08T05:26:05.147777+0000","last_undegraded":"2024-05-08T05:26:05.147777+0000","last_fullsized":"2024-05-08T05:26:05.147777+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:58.440446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147242+0000","last_change":"2024-05-08T05:25:37.052656+0000","last_active":"2024-05-08T05:26:05.147242+0000","last_peered":"2024-05-08T05:26:05.147242+0000","last_clean":"2024-05-08T05:26:05.147242+0000","last_became_active":"2024-05-08T05:25:37.052237+0000","last_became_peered":"2024-05-08T05:25:37.052237+0000","last_unstale":"2024-05-08T05:26:05.147242+0000","last_undegraded":"2024-05-08T05:26:05.147242+0000","last_fullsized":"2024-05-08T05:26:05.147242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:54.354488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":79,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:52.454150+0000","last_change":"2024-05-08T05:25:22.649492+0000","last_active":"2024-05-08T05:26:52.454150+0000","last_peered":"2024-05-08T05:26:52.454150+0000","last_clean":"2024-05-08T05:26:52.454150+0000","last_became_active":"2024-05-08T05:25:22.649132+0000","last_became_peered":"2024-05-08T05:25:22.649132+0000","last_unstale":"2024-05-08T05:26:52.454150+0000","last_undegraded":"2024-05-08T05:26:52.454150+0000","last_fullsized":"2024-05-08T05:26:52.454150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:24.474848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":27,"num_read_kb":27,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306757+0000","last_change":"2024-05-08T05:25:25.667811+0000","last_active":"2024-05-08T05:26:05.306757+0000","last_peered":"2024-05-08T05:26:05.306757+0000","last_clean":"2024-05-08T05:26:05.306757+0000","last_became_active":"2024-05-08T05:25:25.667449+0000","last_became_peered":"2024-05-08T05:25:25.667449+0000","last_unstale":"2024-05-08T05:26:05.306757+0000","last_undegraded":"2024-05-08T05:26:05.306757+0000","last_fullsized":"2024-05-08T05:26:05.306757+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:29.293702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":29,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.308251+0000","last_change":"2024-05-08T05:26:05.308251+0000","last_active":"2024-05-08T05:26:05.308251+0000","last_peered":"2024-05-08T05:26:05.308251+0000","last_clean":"2024-05-08T05:26:05.308251+0000","last_became_active":"2024-05-08T05:26:05.307622+0000","last_became_peered":"2024-05-08T05:26:05.307622+0000","last_unstale":"2024-05-08T05:26:05.308251+0000","last_undegraded":"2024-05-08T05:26:05.308251+0000","last_fullsized":"2024-05-08T05:26:05.308251+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:51.116094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":41,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306902+0000","last_change":"2024-05-08T05:25:22.748130+0000","last_active":"2024-05-08T05:26:05.306902+0000","last_peered":"2024-05-08T05:26:05.306902+0000","last_clean":"2024-05-08T05:26:05.306902+0000","last_became_active":"2024-05-08T05:25:22.747769+0000","last_became_peered":"2024-05-08T05:25:22.747769+0000","last_unstale":"2024-05-08T05:26:05.306902+0000","last_undegraded":"2024-05-08T05:26:05.306902+0000","last_fullsized":"2024-05-08T05:26:05.306902+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:04.086851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147250+0000","last_change":"2024-05-08T05:25:25.672114+0000","last_active":"2024-05-08T05:26:05.147250+0000","last_peered":"2024-05-08T05:26:05.147250+0000","last_clean":"2024-05-08T05:26:05.147250+0000","last_became_active":"2024-05-08T05:25:25.671761+0000","last_became_peered":"2024-05-08T05:25:25.671761+0000","last_unstale":"2024-05-08T05:26:05.147250+0000","last_undegraded":"2024-05-08T05:26:05.147250+0000","last_fullsized":"2024-05-08T05:26:05.147250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:40.684565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143884+0000","last_change":"2024-05-08T05:25:37.058375+0000","last_active":"2024-05-08T05:26:05.143884+0000","last_peered":"2024-05-08T05:26:05.143884+0000","last_clean":"2024-05-08T05:26:05.143884+0000","last_became_active":"2024-05-08T05:25:37.057844+0000","last_became_peered":"2024-05-08T05:25:37.057844+0000","last_unstale":"2024-05-08T05:26:05.143884+0000","last_undegraded":"2024-05-08T05:26:05.143884+0000","last_fullsized":"2024-05-08T05:26:05.143884+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:51:42.881830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":53,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:28.420331+0000","last_change":"2024-05-08T05:25:22.646177+0000","last_active":"2024-05-08T05:26:28.420331+0000","last_peered":"2024-05-08T05:26:28.420331+0000","last_clean":"2024-05-08T05:26:28.420331+0000","last_became_active":"2024-05-08T05:25:22.645755+0000","last_became_peered":"2024-05-08T05:25:22.645755+0000","last_unstale":"2024-05-08T05:26:28.420331+0000","last_undegraded":"2024-05-08T05:26:28.420331+0000","last_fullsized":"2024-05-08T05:26:28.420331+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:08:27.891850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306826+0000","last_change":"2024-05-08T05:25:25.673300+0000","last_active":"2024-05-08T05:26:05.306826+0000","last_peered":"2024-05-08T05:26:05.306826+0000","last_clean":"2024-05-08T05:26:05.306826+0000","last_became_active":"2024-05-08T05:25:25.672881+0000","last_became_peered":"2024-05-08T05:25:25.672881+0000","last_unstale":"2024-05-08T05:26:05.306826+0000","last_undegraded":"2024-05-08T05:26:05.306826+0000","last_fullsized":"2024-05-08T05:26:05.306826+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:16.111775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143086+0000","last_change":"2024-05-08T05:25:37.058903+0000","last_active":"2024-05-08T05:26:05.143086+0000","last_peered":"2024-05-08T05:26:05.143086+0000","last_clean":"2024-05-08T05:26:05.143086+0000","last_became_active":"2024-05-08T05:25:37.058508+0000","last_became_peered":"2024-05-08T05:25:37.058508+0000","last_unstale":"2024-05-08T05:26:05.143086+0000","last_undegraded":"2024-05-08T05:26:05.143086+0000","last_fullsized":"2024-05-08T05:26:05.143086+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:15:16.605748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.418635+0000","last_change":"2024-05-08T05:25:22.646191+0000","last_active":"2024-05-08T05:26:27.418635+0000","last_peered":"2024-05-08T05:26:27.418635+0000","last_clean":"2024-05-08T05:26:27.418635+0000","last_became_active":"2024-05-08T05:25:22.645187+0000","last_became_peered":"2024-05-08T05:25:22.645187+0000","last_unstale":"2024-05-08T05:26:27.418635+0000","last_undegraded":"2024-05-08T05:26:27.418635+0000","last_fullsized":"2024-05-08T05:26:27.418635+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:13.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.308149+0000","last_change":"2024-05-08T05:26:05.308149+0000","last_active":"2024-05-08T05:26:05.308149+0000","last_peered":"2024-05-08T05:26:05.308149+0000","last_clean":"2024-05-08T05:26:05.308149+0000","last_became_active":"2024-05-08T05:26:05.307602+0000","last_became_peered":"2024-05-08T05:26:05.307602+0000","last_unstale":"2024-05-08T05:26:05.308149+0000","last_undegraded":"2024-05-08T05:26:05.308149+0000","last_fullsized":"2024-05-08T05:26:05.308149+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:02.791894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146867+0000","last_change":"2024-05-08T05:25:37.059052+0000","last_active":"2024-05-08T05:26:05.146867+0000","last_peered":"2024-05-08T05:26:05.146867+0000","last_clean":"2024-05-08T05:26:05.146867+0000","last_became_active":"2024-05-08T05:25:37.058545+0000","last_became_peered":"2024-05-08T05:25:37.058545+0000","last_unstale":"2024-05-08T05:26:05.146867+0000","last_undegraded":"2024-05-08T05:26:05.146867+0000","last_fullsized":"2024-05-08T05:26:05.146867+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:31.521284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"88'6","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:53.457002+0000","last_change":"2024-05-08T05:25:22.648874+0000","last_active":"2024-05-08T05:26:53.457002+0000","last_peered":"2024-05-08T05:26:53.457002+0000","last_clean":"2024-05-08T05:26:53.457002+0000","last_became_active":"2024-05-08T05:25:22.648477+0000","last_became_peered":"2024-05-08T05:25:22.648477+0000","last_unstale":"2024-05-08T05:26:53.457002+0000","last_undegraded":"2024-05-08T05:26:53.457002+0000","last_fullsized":"2024-05-08T05:26:53.457002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:08.075904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":3,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143228+0000","last_change":"2024-05-08T05:25:25.662449+0000","last_active":"2024-05-08T05:26:05.143228+0000","last_peered":"2024-05-08T05:26:05.143228+0000","last_clean":"2024-05-08T05:26:05.143228+0000","last_became_active":"2024-05-08T05:25:25.662089+0000","last_became_peered":"2024-05-08T05:25:25.662089+0000","last_unstale":"2024-05-08T05:26:05.143228+0000","last_undegraded":"2024-05-08T05:26:05.143228+0000","last_fullsized":"2024-05-08T05:26:05.143228+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:31.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147434+0000","last_change":"2024-05-08T05:25:37.055548+0000","last_active":"2024-05-08T05:26:05.147434+0000","last_peered":"2024-05-08T05:26:05.147434+0000","last_clean":"2024-05-08T05:26:05.147434+0000","last_became_active":"2024-05-08T05:25:37.055138+0000","last_became_peered":"2024-05-08T05:25:37.055138+0000","last_unstale":"2024-05-08T05:26:05.147434+0000","last_undegraded":"2024-05-08T05:26:05.147434+0000","last_fullsized":"2024-05-08T05:26:05.147434+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:45.697831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:26.411198+0000","last_change":"2024-05-08T05:25:22.648072+0000","last_active":"2024-05-08T05:26:26.411198+0000","last_peered":"2024-05-08T05:26:26.411198+0000","last_clean":"2024-05-08T05:26:26.411198+0000","last_became_active":"2024-05-08T05:25:22.647457+0000","last_became_peered":"2024-05-08T05:25:22.647457+0000","last_unstale":"2024-05-08T05:26:26.411198+0000","last_undegraded":"2024-05-08T05:26:26.411198+0000","last_fullsized":"2024-05-08T05:26:26.411198+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:17.476479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.148392+0000","last_change":"2024-05-08T05:25:25.662698+0000","last_active":"2024-05-08T05:26:05.148392+0000","last_peered":"2024-05-08T05:26:05.148392+0000","last_clean":"2024-05-08T05:26:05.148392+0000","last_became_active":"2024-05-08T05:25:25.662368+0000","last_became_peered":"2024-05-08T05:25:25.662368+0000","last_unstale":"2024-05-08T05:26:05.148392+0000","last_undegraded":"2024-05-08T05:26:05.148392+0000","last_fullsized":"2024-05-08T05:26:05.148392+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:38.511037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142093+0000","last_change":"2024-05-08T05:25:37.054407+0000","last_active":"2024-05-08T05:26:05.142093+0000","last_peered":"2024-05-08T05:26:05.142093+0000","last_clean":"2024-05-08T05:26:05.142093+0000","last_became_active":"2024-05-08T05:25:37.053999+0000","last_became_peered":"2024-05-08T05:25:37.053999+0000","last_unstale":"2024-05-08T05:26:05.142093+0000","last_undegraded":"2024-05-08T05:26:05.142093+0000","last_fullsized":"2024-05-08T05:26:05.142093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:16.942601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:25.403902+0000","last_change":"2024-05-08T05:25:22.643654+0000","last_active":"2024-05-08T05:26:25.403902+0000","last_peered":"2024-05-08T05:26:25.403902+0000","last_clean":"2024-05-08T05:26:25.403902+0000","last_became_active":"2024-05-08T05:25:22.643315+0000","last_became_peered":"2024-05-08T05:25:22.643315+0000","last_unstale":"2024-05-08T05:26:25.403902+0000","last_undegraded":"2024-05-08T05:26:25.403902+0000","last_fullsized":"2024-05-08T05:26:25.403902+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:53.514629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306242+0000","last_change":"2024-05-08T05:25:25.663148+0000","last_active":"2024-05-08T05:26:05.306242+0000","last_peered":"2024-05-08T05:26:05.306242+0000","last_clean":"2024-05-08T05:26:05.306242+0000","last_became_active":"2024-05-08T05:25:25.662770+0000","last_became_peered":"2024-05-08T05:25:25.662770+0000","last_unstale":"2024-05-08T05:26:05.306242+0000","last_undegraded":"2024-05-08T05:26:05.306242+0000","last_fullsized":"2024-05-08T05:26:05.306242+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:46.458233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146964+0000","last_change":"2024-05-08T05:25:37.054265+0000","last_active":"2024-05-08T05:26:05.146964+0000","last_peered":"2024-05-08T05:26:05.146964+0000","last_clean":"2024-05-08T05:26:05.146964+0000","last_became_active":"2024-05-08T05:25:37.053579+0000","last_became_peered":"2024-05-08T05:25:37.053579+0000","last_unstale":"2024-05-08T05:26:05.146964+0000","last_undegraded":"2024-05-08T05:26:05.146964+0000","last_fullsized":"2024-05-08T05:26:05.146964+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:21.748715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.415915+0000","last_change":"2024-05-08T05:25:22.648083+0000","last_active":"2024-05-08T05:26:27.415915+0000","last_peered":"2024-05-08T05:26:27.415915+0000","last_clean":"2024-05-08T05:26:27.415915+0000","last_became_active":"2024-05-08T05:25:22.647650+0000","last_became_peered":"2024-05-08T05:25:22.647650+0000","last_unstale":"2024-05-08T05:26:27.415915+0000","last_undegraded":"2024-05-08T05:26:27.415915+0000","last_fullsized":"2024-05-08T05:26:27.415915+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:56.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.140998+0000","last_change":"2024-05-08T05:25:25.668290+0000","last_active":"2024-05-08T05:26:05.140998+0000","last_peered":"2024-05-08T05:26:05.140998+0000","last_clean":"2024-05-08T05:26:05.140998+0000","last_became_active":"2024-05-08T05:25:25.667888+0000","last_became_peered":"2024-05-08T05:25:25.667888+0000","last_unstale":"2024-05-08T05:26:05.140998+0000","last_undegraded":"2024-05-08T05:26:05.140998+0000","last_fullsized":"2024-05-08T05:26:05.140998+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:19.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306117+0000","last_change":"2024-05-08T05:25:37.057639+0000","last_active":"2024-05-08T05:26:05.306117+0000","last_peered":"2024-05-08T05:26:05.306117+0000","last_clean":"2024-05-08T05:26:05.306117+0000","last_became_active":"2024-05-08T05:25:37.056848+0000","last_became_peered":"2024-05-08T05:25:37.056848+0000","last_unstale":"2024-05-08T05:26:05.306117+0000","last_undegraded":"2024-05-08T05:26:05.306117+0000","last_fullsized":"2024-05-08T05:26:05.306117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:37.137224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":93,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:52.454828+0000","last_change":"2024-05-08T05:25:22.649658+0000","last_active":"2024-05-08T05:26:52.454828+0000","last_peered":"2024-05-08T05:26:52.454828+0000","last_clean":"2024-05-08T05:26:52.454828+0000","last_became_active":"2024-05-08T05:25:22.648771+0000","last_became_peered":"2024-05-08T05:25:22.648771+0000","last_unstale":"2024-05-08T05:26:52.454828+0000","last_undegraded":"2024-05-08T05:26:52.454828+0000","last_fullsized":"2024-05-08T05:26:52.454828+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:54.412710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":50,"num_read_kb":50,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.307197+0000","last_change":"2024-05-08T05:25:25.673931+0000","last_active":"2024-05-08T05:26:05.307197+0000","last_peered":"2024-05-08T05:26:05.307197+0000","last_clean":"2024-05-08T05:26:05.307197+0000","last_became_active":"2024-05-08T05:25:25.673585+0000","last_became_peered":"2024-05-08T05:25:25.673585+0000","last_unstale":"2024-05-08T05:26:05.307197+0000","last_undegraded":"2024-05-08T05:26:05.307197+0000","last_fullsized":"2024-05-08T05:26:05.307197+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:27.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.140846+0000","last_change":"2024-05-08T05:25:37.047651+0000","last_active":"2024-05-08T05:26:05.140846+0000","last_peered":"2024-05-08T05:26:05.140846+0000","last_clean":"2024-05-08T05:26:05.140846+0000","last_became_active":"2024-05-08T05:25:37.047246+0000","last_became_peered":"2024-05-08T05:25:37.047246+0000","last_unstale":"2024-05-08T05:26:05.140846+0000","last_undegraded":"2024-05-08T05:26:05.140846+0000","last_fullsized":"2024-05-08T05:26:05.140846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:22.300076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417326+0000","last_change":"2024-05-08T05:25:22.646973+0000","last_active":"2024-05-08T05:26:27.417326+0000","last_peered":"2024-05-08T05:26:27.417326+0000","last_clean":"2024-05-08T05:26:27.417326+0000","last_became_active":"2024-05-08T05:25:22.646232+0000","last_became_peered":"2024-05-08T05:25:22.646232+0000","last_unstale":"2024-05-08T05:26:27.417326+0000","last_undegraded":"2024-05-08T05:26:27.417326+0000","last_fullsized":"2024-05-08T05:26:27.417326+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:38.147470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147248+0000","last_change":"2024-05-08T05:25:25.661373+0000","last_active":"2024-05-08T05:26:05.147248+0000","last_peered":"2024-05-08T05:26:05.147248+0000","last_clean":"2024-05-08T05:26:05.147248+0000","last_became_active":"2024-05-08T05:25:25.661080+0000","last_became_peered":"2024-05-08T05:25:25.661080+0000","last_unstale":"2024-05-08T05:26:05.147248+0000","last_undegraded":"2024-05-08T05:26:05.147248+0000","last_fullsized":"2024-05-08T05:26:05.147248+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:18.718610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144077+0000","last_change":"2024-05-08T05:25:37.051797+0000","last_active":"2024-05-08T05:26:05.144077+0000","last_peered":"2024-05-08T05:26:05.144077+0000","last_clean":"2024-05-08T05:26:05.144077+0000","last_became_active":"2024-05-08T05:25:37.051381+0000","last_became_peered":"2024-05-08T05:25:37.051381+0000","last_unstale":"2024-05-08T05:26:05.144077+0000","last_undegraded":"2024-05-08T05:26:05.144077+0000","last_fullsized":"2024-05-08T05:26:05.144077+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:34.337512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:26.408648+0000","last_change":"2024-05-08T05:25:22.748510+0000","last_active":"2024-05-08T05:26:26.408648+0000","last_peered":"2024-05-08T05:26:26.408648+0000","last_clean":"2024-05-08T05:26:26.408648+0000","last_became_active":"2024-05-08T05:25:22.748140+0000","last_became_peered":"2024-05-08T05:25:22.748140+0000","last_unstale":"2024-05-08T05:26:26.408648+0000","last_undegraded":"2024-05-08T05:26:26.408648+0000","last_fullsized":"2024-05-08T05:26:26.408648+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:32.325653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306615+0000","last_change":"2024-05-08T05:25:25.671261+0000","last_active":"2024-05-08T05:26:05.306615+0000","last_peered":"2024-05-08T05:26:05.306615+0000","last_clean":"2024-05-08T05:26:05.306615+0000","last_became_active":"2024-05-08T05:25:25.670937+0000","last_became_peered":"2024-05-08T05:25:25.670937+0000","last_unstale":"2024-05-08T05:26:05.306615+0000","last_undegraded":"2024-05-08T05:26:05.306615+0000","last_fullsized":"2024-05-08T05:26:05.306615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:41.375908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.151127+0000","last_change":"2024-05-08T05:26:05.150754+0000","last_active":"2024-05-08T05:26:05.151127+0000","last_peered":"2024-05-08T05:26:05.151127+0000","last_clean":"2024-05-08T05:26:05.151127+0000","last_became_active":"2024-05-08T05:26:05.150331+0000","last_became_peered":"2024-05-08T05:26:05.150331+0000","last_unstale":"2024-05-08T05:26:05.151127+0000","last_undegraded":"2024-05-08T05:26:05.151127+0000","last_fullsized":"2024-05-08T05:26:05.151127+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:23.309103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"88'34","reported_seq":131,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:52.460615+0000","last_change":"2024-05-08T05:25:22.645627+0000","last_active":"2024-05-08T05:26:52.460615+0000","last_peered":"2024-05-08T05:26:52.460615+0000","last_clean":"2024-05-08T05:26:52.460615+0000","last_became_active":"2024-05-08T05:25:22.645243+0000","last_became_peered":"2024-05-08T05:25:22.645243+0000","last_unstale":"2024-05-08T05:26:52.460615+0000","last_undegraded":"2024-05-08T05:26:52.460615+0000","last_fullsized":"2024-05-08T05:26:52.460615+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":34,"log_dups_size":0,"ondisk_log_size":34,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:23.256903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":60380,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":88,"num_write":36,"num_write_kb":73,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144143+0000","last_change":"2024-05-08T05:25:25.661258+0000","last_active":"2024-05-08T05:26:05.144143+0000","last_peered":"2024-05-08T05:26:05.144143+0000","last_clean":"2024-05-08T05:26:05.144143+0000","last_became_active":"2024-05-08T05:25:25.660925+0000","last_became_peered":"2024-05-08T05:25:25.660925+0000","last_unstale":"2024-05-08T05:26:05.144143+0000","last_undegraded":"2024-05-08T05:26:05.144143+0000","last_fullsized":"2024-05-08T05:26:05.144143+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:58.764540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.150117+0000","last_change":"2024-05-08T05:25:37.053539+0000","last_active":"2024-05-08T05:26:05.150117+0000","last_peered":"2024-05-08T05:26:05.150117+0000","last_clean":"2024-05-08T05:26:05.150117+0000","last_became_active":"2024-05-08T05:25:37.053102+0000","last_became_peered":"2024-05-08T05:25:37.053102+0000","last_unstale":"2024-05-08T05:26:05.150117+0000","last_undegraded":"2024-05-08T05:26:05.150117+0000","last_fullsized":"2024-05-08T05:26:05.150117+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:48.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":50,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:28.419558+0000","last_change":"2024-05-08T05:25:22.646651+0000","last_active":"2024-05-08T05:26:28.419558+0000","last_peered":"2024-05-08T05:26:28.419558+0000","last_clean":"2024-05-08T05:26:28.419558+0000","last_became_active":"2024-05-08T05:25:22.646285+0000","last_became_peered":"2024-05-08T05:25:22.646285+0000","last_unstale":"2024-05-08T05:26:28.419558+0000","last_undegraded":"2024-05-08T05:26:28.419558+0000","last_fullsized":"2024-05-08T05:26:28.419558+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.843858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306602+0000","last_change":"2024-05-08T05:25:25.666018+0000","last_active":"2024-05-08T05:26:05.306602+0000","last_peered":"2024-05-08T05:26:05.306602+0000","last_clean":"2024-05-08T05:26:05.306602+0000","last_became_active":"2024-05-08T05:25:25.663609+0000","last_became_peered":"2024-05-08T05:25:25.663609+0000","last_unstale":"2024-05-08T05:26:05.306602+0000","last_undegraded":"2024-05-08T05:26:05.306602+0000","last_fullsized":"2024-05-08T05:26:05.306602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:36.223121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141910+0000","last_change":"2024-05-08T05:25:37.052907+0000","last_active":"2024-05-08T05:26:05.141910+0000","last_peered":"2024-05-08T05:26:05.141910+0000","last_clean":"2024-05-08T05:26:05.141910+0000","last_became_active":"2024-05-08T05:25:37.052506+0000","last_became_peered":"2024-05-08T05:25:37.052506+0000","last_unstale":"2024-05-08T05:26:05.141910+0000","last_undegraded":"2024-05-08T05:26:05.141910+0000","last_fullsized":"2024-05-08T05:26:05.141910+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:41.356996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.415990+0000","last_change":"2024-05-08T05:25:22.646671+0000","last_active":"2024-05-08T05:26:27.415990+0000","last_peered":"2024-05-08T05:26:27.415990+0000","last_clean":"2024-05-08T05:26:27.415990+0000","last_became_active":"2024-05-08T05:25:22.646202+0000","last_became_peered":"2024-05-08T05:25:22.646202+0000","last_unstale":"2024-05-08T05:26:27.415990+0000","last_undegraded":"2024-05-08T05:26:27.415990+0000","last_fullsized":"2024-05-08T05:26:27.415990+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:59.224960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143532+0000","last_change":"2024-05-08T05:25:25.666969+0000","last_active":"2024-05-08T05:26:05.143532+0000","last_peered":"2024-05-08T05:26:05.143532+0000","last_clean":"2024-05-08T05:26:05.143532+0000","last_became_active":"2024-05-08T05:25:25.666482+0000","last_became_peered":"2024-05-08T05:25:25.666482+0000","last_unstale":"2024-05-08T05:26:05.143532+0000","last_undegraded":"2024-05-08T05:26:05.143532+0000","last_fullsized":"2024-05-08T05:26:05.143532+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:07.819116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141632+0000","last_change":"2024-05-08T05:25:37.055807+0000","last_active":"2024-05-08T05:26:05.141632+0000","last_peered":"2024-05-08T05:26:05.141632+0000","last_clean":"2024-05-08T05:26:05.141632+0000","last_became_active":"2024-05-08T05:25:37.054778+0000","last_became_peered":"2024-05-08T05:25:37.054778+0000","last_unstale":"2024-05-08T05:26:05.141632+0000","last_undegraded":"2024-05-08T05:26:05.141632+0000","last_fullsized":"2024-05-08T05:26:05.141632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:33.767885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"88'6","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:54.457883+0000","last_change":"2024-05-08T05:25:22.647093+0000","last_active":"2024-05-08T05:26:54.457883+0000","last_peered":"2024-05-08T05:26:54.457883+0000","last_clean":"2024-05-08T05:26:54.457883+0000","last_became_active":"2024-05-08T05:25:22.646733+0000","last_became_peered":"2024-05-08T05:25:22.646733+0000","last_unstale":"2024-05-08T05:26:54.457883+0000","last_undegraded":"2024-05-08T05:26:54.457883+0000","last_fullsized":"2024-05-08T05:26:54.457883+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:04.346568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143602+0000","last_change":"2024-05-08T05:25:25.666565+0000","last_active":"2024-05-08T05:26:05.143602+0000","last_peered":"2024-05-08T05:26:05.143602+0000","last_clean":"2024-05-08T05:26:05.143602+0000","last_became_active":"2024-05-08T05:25:25.666138+0000","last_became_peered":"2024-05-08T05:25:25.666138+0000","last_unstale":"2024-05-08T05:26:05.143602+0000","last_undegraded":"2024-05-08T05:26:05.143602+0000","last_fullsized":"2024-05-08T05:26:05.143602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:05.396530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306274+0000","last_change":"2024-05-08T05:25:37.060933+0000","last_active":"2024-05-08T05:26:05.306274+0000","last_peered":"2024-05-08T05:26:05.306274+0000","last_clean":"2024-05-08T05:26:05.306274+0000","last_became_active":"2024-05-08T05:25:37.060388+0000","last_became_peered":"2024-05-08T05:25:37.060388+0000","last_unstale":"2024-05-08T05:26:05.306274+0000","last_undegraded":"2024-05-08T05:26:05.306274+0000","last_fullsized":"2024-05-08T05:26:05.306274+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:09.776051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":74,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417329+0000","last_change":"2024-05-08T05:26:05.156863+0000","last_active":"2024-05-08T05:26:27.417329+0000","last_peered":"2024-05-08T05:26:27.417329+0000","last_clean":"2024-05-08T05:26:27.417329+0000","last_became_active":"2024-05-08T05:26:05.149008+0000","last_became_peered":"2024-05-08T05:26:05.149008+0000","last_unstale":"2024-05-08T05:26:27.417329+0000","last_undegraded":"2024-05-08T05:26:27.417329+0000","last_fullsized":"2024-05-08T05:26:27.417329+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:56.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.148746+0000","last_change":"2024-05-08T05:25:25.670415+0000","last_active":"2024-05-08T05:26:05.148746+0000","last_peered":"2024-05-08T05:26:05.148746+0000","last_clean":"2024-05-08T05:26:05.148746+0000","last_became_active":"2024-05-08T05:25:25.670095+0000","last_became_peered":"2024-05-08T05:25:25.670095+0000","last_unstale":"2024-05-08T05:26:05.148746+0000","last_undegraded":"2024-05-08T05:26:05.148746+0000","last_fullsized":"2024-05-08T05:26:05.148746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.489203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":12,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.308609+0000","last_change":"2024-05-08T05:26:05.307721+0000","last_active":"2024-05-08T05:26:05.308609+0000","last_peered":"2024-05-08T05:26:05.308609+0000","last_clean":"2024-05-08T05:26:05.308609+0000","last_became_active":"2024-05-08T05:26:05.307279+0000","last_became_peered":"2024-05-08T05:26:05.307279+0000","last_unstale":"2024-05-08T05:26:05.308609+0000","last_undegraded":"2024-05-08T05:26:05.308609+0000","last_fullsized":"2024-05-08T05:26:05.308609+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:08.857770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":43,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.397844+0000","last_change":"2024-05-08T05:25:22.644279+0000","last_active":"2024-05-08T05:26:27.397844+0000","last_peered":"2024-05-08T05:26:27.397844+0000","last_clean":"2024-05-08T05:26:27.397844+0000","last_became_active":"2024-05-08T05:25:22.643900+0000","last_became_peered":"2024-05-08T05:25:22.643900+0000","last_unstale":"2024-05-08T05:26:27.397844+0000","last_undegraded":"2024-05-08T05:26:27.397844+0000","last_fullsized":"2024-05-08T05:26:27.397844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:32.435843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142048+0000","last_change":"2024-05-08T05:25:25.670130+0000","last_active":"2024-05-08T05:26:05.142048+0000","last_peered":"2024-05-08T05:26:05.142048+0000","last_clean":"2024-05-08T05:26:05.142048+0000","last_became_active":"2024-05-08T05:25:25.669784+0000","last_became_peered":"2024-05-08T05:25:25.669784+0000","last_unstale":"2024-05-08T05:26:05.142048+0000","last_undegraded":"2024-05-08T05:26:05.142048+0000","last_fullsized":"2024-05-08T05:26:05.142048+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:07.188883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":31,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.308804+0000","last_change":"2024-05-08T05:26:05.308804+0000","last_active":"2024-05-08T05:26:05.308804+0000","last_peered":"2024-05-08T05:26:05.308804+0000","last_clean":"2024-05-08T05:26:05.308804+0000","last_became_active":"2024-05-08T05:26:05.308226+0000","last_became_peered":"2024-05-08T05:26:05.308226+0000","last_unstale":"2024-05-08T05:26:05.308804+0000","last_undegraded":"2024-05-08T05:26:05.308804+0000","last_fullsized":"2024-05-08T05:26:05.308804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:33.442023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":40,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141456+0000","last_change":"2024-05-08T05:25:22.748409+0000","last_active":"2024-05-08T05:26:05.141456+0000","last_peered":"2024-05-08T05:26:05.141456+0000","last_clean":"2024-05-08T05:26:05.141456+0000","last_became_active":"2024-05-08T05:25:22.748062+0000","last_became_peered":"2024-05-08T05:25:22.748062+0000","last_unstale":"2024-05-08T05:26:05.141456+0000","last_undegraded":"2024-05-08T05:26:05.141456+0000","last_fullsized":"2024-05-08T05:26:05.141456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:42.147084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142773+0000","last_change":"2024-05-08T05:25:25.677057+0000","last_active":"2024-05-08T05:26:05.142773+0000","last_peered":"2024-05-08T05:26:05.142773+0000","last_clean":"2024-05-08T05:26:05.142773+0000","last_became_active":"2024-05-08T05:25:25.676697+0000","last_became_peered":"2024-05-08T05:25:25.676697+0000","last_unstale":"2024-05-08T05:26:05.142773+0000","last_undegraded":"2024-05-08T05:26:05.142773+0000","last_fullsized":"2024-05-08T05:26:05.142773+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:40.786327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306055+0000","last_change":"2024-05-08T05:25:37.053903+0000","last_active":"2024-05-08T05:26:05.306055+0000","last_peered":"2024-05-08T05:26:05.306055+0000","last_clean":"2024-05-08T05:26:05.306055+0000","last_became_active":"2024-05-08T05:25:37.053375+0000","last_became_peered":"2024-05-08T05:25:37.053375+0000","last_unstale":"2024-05-08T05:26:05.306055+0000","last_undegraded":"2024-05-08T05:26:05.306055+0000","last_fullsized":"2024-05-08T05:26:05.306055+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:19.374355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"88'6","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:54.457701+0000","last_change":"2024-05-08T05:25:22.646931+0000","last_active":"2024-05-08T05:26:54.457701+0000","last_peered":"2024-05-08T05:26:54.457701+0000","last_clean":"2024-05-08T05:26:54.457701+0000","last_became_active":"2024-05-08T05:25:22.646599+0000","last_became_peered":"2024-05-08T05:25:22.646599+0000","last_unstale":"2024-05-08T05:26:54.457701+0000","last_undegraded":"2024-05-08T05:26:54.457701+0000","last_fullsized":"2024-05-08T05:26:54.457701+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:55.873089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":30,"num_read_kb":2,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147977+0000","last_change":"2024-05-08T05:26:05.147977+0000","last_active":"2024-05-08T05:26:05.147977+0000","last_peered":"2024-05-08T05:26:05.147977+0000","last_clean":"2024-05-08T05:26:05.147977+0000","last_became_active":"2024-05-08T05:26:05.147637+0000","last_became_peered":"2024-05-08T05:26:05.147637+0000","last_unstale":"2024-05-08T05:26:05.147977+0000","last_undegraded":"2024-05-08T05:26:05.147977+0000","last_fullsized":"2024-05-08T05:26:05.147977+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:59.671306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306320+0000","last_change":"2024-05-08T05:25:37.057913+0000","last_active":"2024-05-08T05:26:05.306320+0000","last_peered":"2024-05-08T05:26:05.306320+0000","last_clean":"2024-05-08T05:26:05.306320+0000","last_became_active":"2024-05-08T05:25:37.057474+0000","last_became_peered":"2024-05-08T05:25:37.057474+0000","last_unstale":"2024-05-08T05:26:05.306320+0000","last_undegraded":"2024-05-08T05:26:05.306320+0000","last_fullsized":"2024-05-08T05:26:05.306320+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:19.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147331+0000","last_change":"2024-05-08T05:25:25.663594+0000","last_active":"2024-05-08T05:26:05.147331+0000","last_peered":"2024-05-08T05:26:05.147331+0000","last_clean":"2024-05-08T05:26:05.147331+0000","last_became_active":"2024-05-08T05:25:25.663275+0000","last_became_peered":"2024-05-08T05:25:25.663275+0000","last_unstale":"2024-05-08T05:26:05.147331+0000","last_undegraded":"2024-05-08T05:26:05.147331+0000","last_fullsized":"2024-05-08T05:26:05.147331+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:20.581166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"88'5","reported_seq":75,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:55.458786+0000","last_change":"2024-05-08T05:25:22.748316+0000","last_active":"2024-05-08T05:26:55.458786+0000","last_peered":"2024-05-08T05:26:55.458786+0000","last_clean":"2024-05-08T05:26:55.458786+0000","last_became_active":"2024-05-08T05:25:22.747819+0000","last_became_peered":"2024-05-08T05:25:22.747819+0000","last_unstale":"2024-05-08T05:26:55.458786+0000","last_undegraded":"2024-05-08T05:26:55.458786+0000","last_fullsized":"2024-05-08T05:26:55.458786+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:50:42.689853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":30,"num_read_kb":30,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":25,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143892+0000","last_change":"2024-05-08T05:25:37.058271+0000","last_active":"2024-05-08T05:26:05.143892+0000","last_peered":"2024-05-08T05:26:05.143892+0000","last_clean":"2024-05-08T05:26:05.143892+0000","last_became_active":"2024-05-08T05:25:37.057290+0000","last_became_peered":"2024-05-08T05:25:37.057290+0000","last_unstale":"2024-05-08T05:26:05.143892+0000","last_undegraded":"2024-05-08T05:26:05.143892+0000","last_fullsized":"2024-05-08T05:26:05.143892+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:47.369909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306025+0000","last_change":"2024-05-08T05:25:25.666000+0000","last_active":"2024-05-08T05:26:05.306025+0000","last_peered":"2024-05-08T05:26:05.306025+0000","last_clean":"2024-05-08T05:26:05.306025+0000","last_became_active":"2024-05-08T05:25:25.665663+0000","last_became_peered":"2024-05-08T05:25:25.665663+0000","last_unstale":"2024-05-08T05:26:05.306025+0000","last_undegraded":"2024-05-08T05:26:05.306025+0000","last_fullsized":"2024-05-08T05:26:05.306025+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:44.328879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":48,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:28.419249+0000","last_change":"2024-05-08T05:25:22.648770+0000","last_active":"2024-05-08T05:26:28.419249+0000","last_peered":"2024-05-08T05:26:28.419249+0000","last_clean":"2024-05-08T05:26:28.419249+0000","last_became_active":"2024-05-08T05:25:22.648119+0000","last_became_peered":"2024-05-08T05:25:22.648119+0000","last_unstale":"2024-05-08T05:26:28.419249+0000","last_undegraded":"2024-05-08T05:26:28.419249+0000","last_fullsized":"2024-05-08T05:26:28.419249+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:55.813080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":115,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143307+0000","last_change":"2024-05-08T05:23:55.820436+0000","last_active":"2024-05-08T05:26:05.143307+0000","last_peered":"2024-05-08T05:26:05.143307+0000","last_clean":"2024-05-08T05:26:05.143307+0000","last_became_active":"2024-05-08T05:23:55.814786+0000","last_became_peered":"2024-05-08T05:23:55.814786+0000","last_unstale":"2024-05-08T05:26:05.143307+0000","last_undegraded":"2024-05-08T05:26:05.143307+0000","last_fullsized":"2024-05-08T05:26:05.143307+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_clean_scrub_stamp":"2024-05-08T05:20:32.659283+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:11.169455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147318+0000","last_change":"2024-05-08T05:25:37.057287+0000","last_active":"2024-05-08T05:26:05.147318+0000","last_peered":"2024-05-08T05:26:05.147318+0000","last_clean":"2024-05-08T05:26:05.147318+0000","last_became_active":"2024-05-08T05:25:37.056871+0000","last_became_peered":"2024-05-08T05:25:37.056871+0000","last_unstale":"2024-05-08T05:26:05.147318+0000","last_undegraded":"2024-05-08T05:26:05.147318+0000","last_fullsized":"2024-05-08T05:26:05.147318+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:19.004199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.305801+0000","last_change":"2024-05-08T05:25:25.664795+0000","last_active":"2024-05-08T05:26:05.305801+0000","last_peered":"2024-05-08T05:26:05.305801+0000","last_clean":"2024-05-08T05:26:05.305801+0000","last_became_active":"2024-05-08T05:25:25.664444+0000","last_became_peered":"2024-05-08T05:25:25.664444+0000","last_unstale":"2024-05-08T05:26:05.305801+0000","last_undegraded":"2024-05-08T05:26:05.305801+0000","last_fullsized":"2024-05-08T05:26:05.305801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:27.172530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:25.402073+0000","last_change":"2024-05-08T05:25:22.645205+0000","last_active":"2024-05-08T05:26:25.402073+0000","last_peered":"2024-05-08T05:26:25.402073+0000","last_clean":"2024-05-08T05:26:25.402073+0000","last_became_active":"2024-05-08T05:25:22.644720+0000","last_became_peered":"2024-05-08T05:25:22.644720+0000","last_unstale":"2024-05-08T05:26:25.402073+0000","last_undegraded":"2024-05-08T05:26:25.402073+0000","last_fullsized":"2024-05-08T05:26:25.402073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:05.630188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141854+0000","last_change":"2024-05-08T05:25:37.055151+0000","last_active":"2024-05-08T05:26:05.141854+0000","last_peered":"2024-05-08T05:26:05.141854+0000","last_clean":"2024-05-08T05:26:05.141854+0000","last_became_active":"2024-05-08T05:25:37.054737+0000","last_became_peered":"2024-05-08T05:25:37.054737+0000","last_unstale":"2024-05-08T05:26:05.141854+0000","last_undegraded":"2024-05-08T05:26:05.141854+0000","last_fullsized":"2024-05-08T05:26:05.141854+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:49.135453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":16,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417886+0000","last_change":"2024-05-08T05:26:05.148665+0000","last_active":"2024-05-08T05:26:27.417886+0000","last_peered":"2024-05-08T05:26:27.417886+0000","last_clean":"2024-05-08T05:26:27.417886+0000","last_became_active":"2024-05-08T05:26:05.148283+0000","last_became_peered":"2024-05-08T05:26:05.148283+0000","last_unstale":"2024-05-08T05:26:27.417886+0000","last_undegraded":"2024-05-08T05:26:27.417886+0000","last_fullsized":"2024-05-08T05:26:27.417886+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:26.197521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306567+0000","last_change":"2024-05-08T05:25:25.669130+0000","last_active":"2024-05-08T05:26:05.306567+0000","last_peered":"2024-05-08T05:26:05.306567+0000","last_clean":"2024-05-08T05:26:05.306567+0000","last_became_active":"2024-05-08T05:25:25.668812+0000","last_became_peered":"2024-05-08T05:25:25.668812+0000","last_unstale":"2024-05-08T05:26:05.306567+0000","last_undegraded":"2024-05-08T05:26:05.306567+0000","last_fullsized":"2024-05-08T05:26:05.306567+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:34.514248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306134+0000","last_change":"2024-05-08T05:25:37.057146+0000","last_active":"2024-05-08T05:26:05.306134+0000","last_peered":"2024-05-08T05:26:05.306134+0000","last_clean":"2024-05-08T05:26:05.306134+0000","last_became_active":"2024-05-08T05:25:37.056706+0000","last_became_peered":"2024-05-08T05:25:37.056706+0000","last_unstale":"2024-05-08T05:26:05.306134+0000","last_undegraded":"2024-05-08T05:26:05.306134+0000","last_fullsized":"2024-05-08T05:26:05.306134+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:09.724088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'15","reported_seq":54,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:48.680508+0000","last_change":"2024-05-08T05:25:22.644573+0000","last_active":"2024-05-08T05:26:48.680508+0000","last_peered":"2024-05-08T05:26:48.680508+0000","last_clean":"2024-05-08T05:26:48.680508+0000","last_became_active":"2024-05-08T05:25:22.644231+0000","last_became_peered":"2024-05-08T05:25:22.644231+0000","last_unstale":"2024-05-08T05:26:48.680508+0000","last_undegraded":"2024-05-08T05:26:48.680508+0000","last_fullsized":"2024-05-08T05:26:48.680508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:02.931823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306804+0000","last_change":"2024-05-08T05:25:25.671036+0000","last_active":"2024-05-08T05:26:05.306804+0000","last_peered":"2024-05-08T05:26:05.306804+0000","last_clean":"2024-05-08T05:26:05.306804+0000","last_became_active":"2024-05-08T05:25:25.669170+0000","last_became_peered":"2024-05-08T05:25:25.669170+0000","last_unstale":"2024-05-08T05:26:05.306804+0000","last_undegraded":"2024-05-08T05:26:05.306804+0000","last_fullsized":"2024-05-08T05:26:05.306804+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:18.963232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147050+0000","last_change":"2024-05-08T05:25:37.050656+0000","last_active":"2024-05-08T05:26:05.147050+0000","last_peered":"2024-05-08T05:26:05.147050+0000","last_clean":"2024-05-08T05:26:05.147050+0000","last_became_active":"2024-05-08T05:25:37.050051+0000","last_became_peered":"2024-05-08T05:25:37.050051+0000","last_unstale":"2024-05-08T05:26:05.147050+0000","last_undegraded":"2024-05-08T05:26:05.147050+0000","last_fullsized":"2024-05-08T05:26:05.147050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:56:07.316216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":47,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417957+0000","last_change":"2024-05-08T05:25:23.614874+0000","last_active":"2024-05-08T05:26:27.417957+0000","last_peered":"2024-05-08T05:26:27.417957+0000","last_clean":"2024-05-08T05:26:27.417957+0000","last_became_active":"2024-05-08T05:25:23.614550+0000","last_became_peered":"2024-05-08T05:25:23.614550+0000","last_unstale":"2024-05-08T05:26:27.417957+0000","last_undegraded":"2024-05-08T05:26:27.417957+0000","last_fullsized":"2024-05-08T05:26:27.417957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:10.183477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147147+0000","last_change":"2024-05-08T05:25:25.658710+0000","last_active":"2024-05-08T05:26:05.147147+0000","last_peered":"2024-05-08T05:26:05.147147+0000","last_clean":"2024-05-08T05:26:05.147147+0000","last_became_active":"2024-05-08T05:25:25.658393+0000","last_became_peered":"2024-05-08T05:25:25.658393+0000","last_unstale":"2024-05-08T05:26:05.147147+0000","last_undegraded":"2024-05-08T05:26:05.147147+0000","last_fullsized":"2024-05-08T05:26:05.147147+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:02.629256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143395+0000","last_change":"2024-05-08T05:25:37.059976+0000","last_active":"2024-05-08T05:26:05.143395+0000","last_peered":"2024-05-08T05:26:05.143395+0000","last_clean":"2024-05-08T05:26:05.143395+0000","last_became_active":"2024-05-08T05:25:37.059474+0000","last_became_peered":"2024-05-08T05:25:37.059474+0000","last_unstale":"2024-05-08T05:26:05.143395+0000","last_undegraded":"2024-05-08T05:26:05.143395+0000","last_fullsized":"2024-05-08T05:26:05.143395+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:51.946242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:24.398321+0000","last_change":"2024-05-08T05:25:22.748271+0000","last_active":"2024-05-08T05:26:24.398321+0000","last_peered":"2024-05-08T05:26:24.398321+0000","last_clean":"2024-05-08T05:26:24.398321+0000","last_became_active":"2024-05-08T05:25:22.747915+0000","last_became_peered":"2024-05-08T05:25:22.747915+0000","last_unstale":"2024-05-08T05:26:24.398321+0000","last_undegraded":"2024-05-08T05:26:24.398321+0000","last_fullsized":"2024-05-08T05:26:24.398321+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:35.262298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143012+0000","last_change":"2024-05-08T05:25:25.668722+0000","last_active":"2024-05-08T05:26:05.143012+0000","last_peered":"2024-05-08T05:26:05.143012+0000","last_clean":"2024-05-08T05:26:05.143012+0000","last_became_active":"2024-05-08T05:25:25.668223+0000","last_became_peered":"2024-05-08T05:25:25.668223+0000","last_unstale":"2024-05-08T05:26:05.143012+0000","last_undegraded":"2024-05-08T05:26:05.143012+0000","last_fullsized":"2024-05-08T05:26:05.143012+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:43.511567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141728+0000","last_change":"2024-05-08T05:25:37.056131+0000","last_active":"2024-05-08T05:26:05.141728+0000","last_peered":"2024-05-08T05:26:05.141728+0000","last_clean":"2024-05-08T05:26:05.141728+0000","last_became_active":"2024-05-08T05:25:37.055424+0000","last_became_peered":"2024-05-08T05:25:37.055424+0000","last_unstale":"2024-05-08T05:26:05.141728+0000","last_undegraded":"2024-05-08T05:26:05.141728+0000","last_fullsized":"2024-05-08T05:26:05.141728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:43.279408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":46,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.417490+0000","last_change":"2024-05-08T05:25:22.643908+0000","last_active":"2024-05-08T05:26:27.417490+0000","last_peered":"2024-05-08T05:26:27.417490+0000","last_clean":"2024-05-08T05:26:27.417490+0000","last_became_active":"2024-05-08T05:25:22.643542+0000","last_became_peered":"2024-05-08T05:25:22.643542+0000","last_unstale":"2024-05-08T05:26:27.417490+0000","last_undegraded":"2024-05-08T05:26:27.417490+0000","last_fullsized":"2024-05-08T05:26:27.417490+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:01.671285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141649+0000","last_change":"2024-05-08T05:25:25.667843+0000","last_active":"2024-05-08T05:26:05.141649+0000","last_peered":"2024-05-08T05:26:05.141649+0000","last_clean":"2024-05-08T05:26:05.141649+0000","last_became_active":"2024-05-08T05:25:25.667462+0000","last_became_peered":"2024-05-08T05:25:25.667462+0000","last_unstale":"2024-05-08T05:26:05.141649+0000","last_undegraded":"2024-05-08T05:26:05.141649+0000","last_fullsized":"2024-05-08T05:26:05.141649+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:51.325104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":23,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144596+0000","last_change":"2024-05-08T05:25:37.053017+0000","last_active":"2024-05-08T05:26:05.144596+0000","last_peered":"2024-05-08T05:26:05.144596+0000","last_clean":"2024-05-08T05:26:05.144596+0000","last_became_active":"2024-05-08T05:25:37.052601+0000","last_became_peered":"2024-05-08T05:25:37.052601+0000","last_unstale":"2024-05-08T05:26:05.144596+0000","last_undegraded":"2024-05-08T05:26:05.144596+0000","last_fullsized":"2024-05-08T05:26:05.144596+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:35.393835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":42,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:26.410273+0000","last_change":"2024-05-08T05:25:22.644564+0000","last_active":"2024-05-08T05:26:26.410273+0000","last_peered":"2024-05-08T05:26:26.410273+0000","last_clean":"2024-05-08T05:26:26.410273+0000","last_became_active":"2024-05-08T05:25:22.644170+0000","last_became_peered":"2024-05-08T05:25:22.644170+0000","last_unstale":"2024-05-08T05:26:26.410273+0000","last_undegraded":"2024-05-08T05:26:26.410273+0000","last_fullsized":"2024-05-08T05:26:26.410273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:24.757574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306612+0000","last_change":"2024-05-08T05:25:25.676983+0000","last_active":"2024-05-08T05:26:05.306612+0000","last_peered":"2024-05-08T05:26:05.306612+0000","last_clean":"2024-05-08T05:26:05.306612+0000","last_became_active":"2024-05-08T05:25:25.676610+0000","last_became_peered":"2024-05-08T05:25:25.676610+0000","last_unstale":"2024-05-08T05:26:05.306612+0000","last_undegraded":"2024-05-08T05:26:05.306612+0000","last_fullsized":"2024-05-08T05:26:05.306612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:39.137935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.143383+0000","last_change":"2024-05-08T05:25:37.059381+0000","last_active":"2024-05-08T05:26:05.143383+0000","last_peered":"2024-05-08T05:26:05.143383+0000","last_clean":"2024-05-08T05:26:05.143383+0000","last_became_active":"2024-05-08T05:25:37.056851+0000","last_became_peered":"2024-05-08T05:25:37.056851+0000","last_unstale":"2024-05-08T05:26:05.143383+0000","last_undegraded":"2024-05-08T05:26:05.143383+0000","last_fullsized":"2024-05-08T05:26:05.143383+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:19.339331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:24.396255+0000","last_change":"2024-05-08T05:25:22.648099+0000","last_active":"2024-05-08T05:26:24.396255+0000","last_peered":"2024-05-08T05:26:24.396255+0000","last_clean":"2024-05-08T05:26:24.396255+0000","last_became_active":"2024-05-08T05:25:22.647736+0000","last_became_peered":"2024-05-08T05:25:22.647736+0000","last_unstale":"2024-05-08T05:26:24.396255+0000","last_undegraded":"2024-05-08T05:26:24.396255+0000","last_fullsized":"2024-05-08T05:26:24.396255+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:12.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144712+0000","last_change":"2024-05-08T05:25:25.667565+0000","last_active":"2024-05-08T05:26:05.144712+0000","last_peered":"2024-05-08T05:26:05.144712+0000","last_clean":"2024-05-08T05:26:05.144712+0000","last_became_active":"2024-05-08T05:25:25.667217+0000","last_became_peered":"2024-05-08T05:25:25.667217+0000","last_unstale":"2024-05-08T05:26:05.144712+0000","last_undegraded":"2024-05-08T05:26:05.144712+0000","last_fullsized":"2024-05-08T05:26:05.144712+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:05.228227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306478+0000","last_change":"2024-05-08T05:25:37.056923+0000","last_active":"2024-05-08T05:26:05.306478+0000","last_peered":"2024-05-08T05:26:05.306478+0000","last_clean":"2024-05-08T05:26:05.306478+0000","last_became_active":"2024-05-08T05:25:37.056496+0000","last_became_peered":"2024-05-08T05:25:37.056496+0000","last_unstale":"2024-05-08T05:26:05.306478+0000","last_undegraded":"2024-05-08T05:26:05.306478+0000","last_fullsized":"2024-05-08T05:26:05.306478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:21.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"88'6","reported_seq":76,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:54.458514+0000","last_change":"2024-05-08T05:25:23.614900+0000","last_active":"2024-05-08T05:26:54.458514+0000","last_peered":"2024-05-08T05:26:54.458514+0000","last_clean":"2024-05-08T05:26:54.458514+0000","last_became_active":"2024-05-08T05:25:23.614510+0000","last_became_peered":"2024-05-08T05:25:23.614510+0000","last_unstale":"2024-05-08T05:26:54.458514+0000","last_undegraded":"2024-05-08T05:26:54.458514+0000","last_fullsized":"2024-05-08T05:26:54.458514+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:19.539656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1561,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":31,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144199+0000","last_change":"2024-05-08T05:25:25.669016+0000","last_active":"2024-05-08T05:26:05.144199+0000","last_peered":"2024-05-08T05:26:05.144199+0000","last_clean":"2024-05-08T05:26:05.144199+0000","last_became_active":"2024-05-08T05:25:25.667809+0000","last_became_peered":"2024-05-08T05:25:25.667809+0000","last_unstale":"2024-05-08T05:26:05.144199+0000","last_undegraded":"2024-05-08T05:26:05.144199+0000","last_fullsized":"2024-05-08T05:26:05.144199+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:56.475622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":22,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146478+0000","last_change":"2024-05-08T05:25:37.053630+0000","last_active":"2024-05-08T05:26:05.146478+0000","last_peered":"2024-05-08T05:26:05.146478+0000","last_clean":"2024-05-08T05:26:05.146478+0000","last_became_active":"2024-05-08T05:25:37.053069+0000","last_became_peered":"2024-05-08T05:25:37.053069+0000","last_unstale":"2024-05-08T05:26:05.146478+0000","last_undegraded":"2024-05-08T05:26:05.146478+0000","last_fullsized":"2024-05-08T05:26:05.146478+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:49:56.693447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":44,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.415813+0000","last_change":"2024-05-08T05:25:22.645284+0000","last_active":"2024-05-08T05:26:27.415813+0000","last_peered":"2024-05-08T05:26:27.415813+0000","last_clean":"2024-05-08T05:26:27.415813+0000","last_became_active":"2024-05-08T05:25:22.644897+0000","last_became_peered":"2024-05-08T05:25:22.644897+0000","last_unstale":"2024-05-08T05:26:27.415813+0000","last_undegraded":"2024-05-08T05:26:27.415813+0000","last_fullsized":"2024-05-08T05:26:27.415813+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:53:31.423412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.144032+0000","last_change":"2024-05-08T05:25:25.664930+0000","last_active":"2024-05-08T05:26:05.144032+0000","last_peered":"2024-05-08T05:26:05.144032+0000","last_clean":"2024-05-08T05:26:05.144032+0000","last_became_active":"2024-05-08T05:25:25.664395+0000","last_became_peered":"2024-05-08T05:25:25.664395+0000","last_unstale":"2024-05-08T05:26:05.144032+0000","last_undegraded":"2024-05-08T05:26:05.144032+0000","last_fullsized":"2024-05-08T05:26:05.144032+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:37.971474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":24,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146841+0000","last_change":"2024-05-08T05:25:37.053258+0000","last_active":"2024-05-08T05:26:05.146841+0000","last_peered":"2024-05-08T05:26:05.146841+0000","last_clean":"2024-05-08T05:26:05.146841+0000","last_became_active":"2024-05-08T05:25:37.052786+0000","last_became_peered":"2024-05-08T05:25:37.052786+0000","last_unstale":"2024-05-08T05:26:05.146841+0000","last_undegraded":"2024-05-08T05:26:05.146841+0000","last_fullsized":"2024-05-08T05:26:05.146841+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:07.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":45,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:27.416807+0000","last_change":"2024-05-08T05:25:22.642619+0000","last_active":"2024-05-08T05:26:27.416807+0000","last_peered":"2024-05-08T05:26:27.416807+0000","last_clean":"2024-05-08T05:26:27.416807+0000","last_became_active":"2024-05-08T05:25:22.642237+0000","last_became_peered":"2024-05-08T05:25:22.642237+0000","last_unstale":"2024-05-08T05:26:27.416807+0000","last_undegraded":"2024-05-08T05:26:27.416807+0000","last_fullsized":"2024-05-08T05:26:27.416807+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:03:21.133179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.145164+0000","last_change":"2024-05-08T05:25:25.663771+0000","last_active":"2024-05-08T05:26:05.145164+0000","last_peered":"2024-05-08T05:26:05.145164+0000","last_clean":"2024-05-08T05:26:05.145164+0000","last_became_active":"2024-05-08T05:25:25.663281+0000","last_became_peered":"2024-05-08T05:25:25.663281+0000","last_unstale":"2024-05-08T05:26:05.145164+0000","last_undegraded":"2024-05-08T05:26:05.145164+0000","last_fullsized":"2024-05-08T05:26:05.145164+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:43.892781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.141801+0000","last_change":"2024-05-08T05:25:25.665446+0000","last_active":"2024-05-08T05:26:05.141801+0000","last_peered":"2024-05-08T05:26:05.141801+0000","last_clean":"2024-05-08T05:26:05.141801+0000","last_became_active":"2024-05-08T05:25:25.664389+0000","last_became_peered":"2024-05-08T05:25:25.664389+0000","last_unstale":"2024-05-08T05:26:05.141801+0000","last_undegraded":"2024-05-08T05:26:05.141801+0000","last_fullsized":"2024-05-08T05:26:05.141801+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":39,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.149870+0000","last_change":"2024-05-08T05:26:05.149870+0000","last_active":"2024-05-08T05:26:05.149870+0000","last_peered":"2024-05-08T05:26:05.149870+0000","last_clean":"2024-05-08T05:26:05.149870+0000","last_became_active":"2024-05-08T05:26:05.149476+0000","last_became_peered":"2024-05-08T05:26:05.149476+0000","last_unstale":"2024-05-08T05:26:05.149870+0000","last_undegraded":"2024-05-08T05:26:05.149870+0000","last_fullsized":"2024-05-08T05:26:05.149870+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:40:43.401832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146604+0000","last_change":"2024-05-08T05:25:25.667836+0000","last_active":"2024-05-08T05:26:05.146604+0000","last_peered":"2024-05-08T05:26:05.146604+0000","last_clean":"2024-05-08T05:26:05.146604+0000","last_became_active":"2024-05-08T05:25:25.667496+0000","last_became_peered":"2024-05-08T05:25:25.667496+0000","last_unstale":"2024-05-08T05:26:05.146604+0000","last_undegraded":"2024-05-08T05:26:05.146604+0000","last_fullsized":"2024-05-08T05:26:05.146604+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:01.482837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146732+0000","last_change":"2024-05-08T05:25:25.667279+0000","last_active":"2024-05-08T05:26:05.146732+0000","last_peered":"2024-05-08T05:26:05.146732+0000","last_clean":"2024-05-08T05:26:05.146732+0000","last_became_active":"2024-05-08T05:25:25.666960+0000","last_became_peered":"2024-05-08T05:25:25.666960+0000","last_unstale":"2024-05-08T05:26:05.146732+0000","last_undegraded":"2024-05-08T05:26:05.146732+0000","last_fullsized":"2024-05-08T05:26:05.146732+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:06.335850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306217+0000","last_change":"2024-05-08T05:25:25.667644+0000","last_active":"2024-05-08T05:26:05.306217+0000","last_peered":"2024-05-08T05:26:05.306217+0000","last_clean":"2024-05-08T05:26:05.306217+0000","last_became_active":"2024-05-08T05:25:25.667133+0000","last_became_peered":"2024-05-08T05:25:25.667133+0000","last_unstale":"2024-05-08T05:26:05.306217+0000","last_undegraded":"2024-05-08T05:26:05.306217+0000","last_fullsized":"2024-05-08T05:26:05.306217+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:03.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":32,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.142612+0000","last_change":"2024-05-08T05:25:25.670761+0000","last_active":"2024-05-08T05:26:05.142612+0000","last_peered":"2024-05-08T05:26:05.142612+0000","last_clean":"2024-05-08T05:26:05.142612+0000","last_became_active":"2024-05-08T05:25:25.670342+0000","last_became_peered":"2024-05-08T05:25:25.670342+0000","last_unstale":"2024-05-08T05:26:05.142612+0000","last_undegraded":"2024-05-08T05:26:05.142612+0000","last_fullsized":"2024-05-08T05:26:05.142612+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:08.069332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.146945+0000","last_change":"2024-05-08T05:25:25.664512+0000","last_active":"2024-05-08T05:26:05.146945+0000","last_peered":"2024-05-08T05:26:05.146945+0000","last_clean":"2024-05-08T05:26:05.146945+0000","last_became_active":"2024-05-08T05:25:25.664191+0000","last_became_peered":"2024-05-08T05:25:25.664191+0000","last_unstale":"2024-05-08T05:26:05.146945+0000","last_undegraded":"2024-05-08T05:26:05.146945+0000","last_fullsized":"2024-05-08T05:26:05.146945+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:07.586637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306937+0000","last_change":"2024-05-08T05:25:25.675711+0000","last_active":"2024-05-08T05:26:05.306937+0000","last_peered":"2024-05-08T05:26:05.306937+0000","last_clean":"2024-05-08T05:26:05.306937+0000","last_became_active":"2024-05-08T05:25:25.671890+0000","last_became_peered":"2024-05-08T05:25:25.671890+0000","last_unstale":"2024-05-08T05:26:05.306937+0000","last_undegraded":"2024-05-08T05:26:05.306937+0000","last_fullsized":"2024-05-08T05:26:05.306937+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:57:49.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.147151+0000","last_change":"2024-05-08T05:25:25.662603+0000","last_active":"2024-05-08T05:26:05.147151+0000","last_peered":"2024-05-08T05:26:05.147151+0000","last_clean":"2024-05-08T05:26:05.147151+0000","last_became_active":"2024-05-08T05:25:25.662287+0000","last_became_peered":"2024-05-08T05:25:25.662287+0000","last_unstale":"2024-05-08T05:26:05.147151+0000","last_undegraded":"2024-05-08T05:26:05.147151+0000","last_fullsized":"2024-05-08T05:26:05.147151+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:02.152557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":33,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306376+0000","last_change":"2024-05-08T05:25:25.667309+0000","last_active":"2024-05-08T05:26:05.306376+0000","last_peered":"2024-05-08T05:26:05.306376+0000","last_clean":"2024-05-08T05:26:05.306376+0000","last_became_active":"2024-05-08T05:25:25.666968+0000","last_became_peered":"2024-05-08T05:25:25.666968+0000","last_unstale":"2024-05-08T05:26:05.306376+0000","last_undegraded":"2024-05-08T05:26:05.306376+0000","last_fullsized":"2024-05-08T05:26:05.306376+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:28.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306760+0000","last_change":"2024-05-08T05:25:25.673579+0000","last_active":"2024-05-08T05:26:05.306760+0000","last_peered":"2024-05-08T05:26:05.306760+0000","last_clean":"2024-05-08T05:26:05.306760+0000","last_became_active":"2024-05-08T05:25:25.673130+0000","last_became_peered":"2024-05-08T05:25:25.673130+0000","last_unstale":"2024-05-08T05:26:05.306760+0000","last_undegraded":"2024-05-08T05:26:05.306760+0000","last_fullsized":"2024-05-08T05:26:05.306760+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:01.291319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":34,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.306227+0000","last_change":"2024-05-08T05:25:25.667359+0000","last_active":"2024-05-08T05:26:05.306227+0000","last_peered":"2024-05-08T05:26:05.306227+0000","last_clean":"2024-05-08T05:26:05.306227+0000","last_became_active":"2024-05-08T05:25:25.667044+0000","last_became_peered":"2024-05-08T05:25:25.667044+0000","last_unstale":"2024-05-08T05:26:05.306227+0000","last_undegraded":"2024-05-08T05:26:05.306227+0000","last_fullsized":"2024-05-08T05:26:05.306227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:54.973053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":88,"state":"active+clean","last_fresh":"2024-05-08T05:26:05.150407+0000","last_change":"2024-05-08T05:26:05.150407+0000","last_active":"2024-05-08T05:26:05.150407+0000","last_peered":"2024-05-08T05:26:05.150407+0000","last_clean":"2024-05-08T05:26:05.150407+0000","last_became_active":"2024-05-08T05:26:05.149916+0000","last_became_peered":"2024-05-08T05:26:05.149916+0000","last_unstale":"2024-05-08T05:26:05.150407+0000","last_undegraded":"2024-05-08T05:26:05.150407+0000","last_fullsized":"2024-05-08T05:26:05.150407+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:05:57.075928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247352,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68366,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":308,"num_read_kb":227,"num_write":250,"num_write_kb":141,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":565248,"data_stored":205098,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":215,"ondisk_log_size":215,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678049,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27976,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717176,"statfs":{"total":95995035648,"available":95966388224,"internally_reserved":0,"allocated":983040,"data_stored":502749,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874278,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28112,"kb_used_data":1096,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717040,"statfs":{"total":95995035648,"available":95966248960,"internally_reserved":0,"allocated":1122304,"data_stored":1149833,"data_compressed":62314,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037804,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27996,"kb_used_data":980,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717156,"statfs":{"total":95995035648,"available":95966367744,"internally_reserved":0,"allocated":1003520,"data_stored":506272,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234034,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":502442,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462965,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":1099833,"data_compressed":62314,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626492,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27968,"kb_used_data":952,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717184,"statfs":{"total":95995035648,"available":95966396416,"internally_reserved":0,"allocated":974848,"data_stored":498071,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574933,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1040384,"data_stored":559481,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":499527,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411408,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1028,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1052672,"data_stored":559980,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182474,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32140,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713012,"statfs":{"total":95995035648,"available":95962124288,"internally_reserved":0,"allocated":987136,"data_stored":497388,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986246,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32144,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713008,"statfs":{"total":95995035648,"available":95962120192,"internally_reserved":0,"allocated":991232,"data_stored":502257,"data_compressed":26514,"data_compressed_allocated":36864,"data_compressed_original":73728,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822720,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32208,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712944,"statfs":{"total":95995035648,"available":95962054656,"internally_reserved":0,"allocated":1056768,"data_stored":1094633,"data_compressed":62314,"data_compressed_allocated":81920,"data_compressed_original":663552,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":62523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":63022,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1203,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1113,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4315,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":114688,"data_stored":62507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:26:59.011 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T05:26:59.012 INFO:tasks.cephadm:Setting up client nodes... 2024-05-08T05:26:59.012 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-05-08T05:26:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:26:59.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:59 smithi143 ceph-mon[34337]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-08T05:26:59.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:26:59 smithi143 ceph-mon[34337]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:26:59.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:59 smithi059 ceph-mon[30304]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-08T05:26:59.758 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:26:59 smithi059 ceph-mon[30304]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:26:59.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:59 smithi096 ceph-mon[34253]: pgmap v345: 129 pgs: 129 active+clean; 644 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 10 op/s 2024-05-08T05:26:59.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:26:59 smithi096 ceph-mon[34253]: from='client.14805 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.033+0000 7f26f84ba700 1 -- 172.21.15.59:0/1600868630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103fc0 msgr2=0x7f26f010a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.033+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1600868630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103fc0 0x7f26f010a870 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f26e800b0d0 tx=0x7f26e800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.033+0000 7f26f84ba700 1 -- 172.21.15.59:0/1600868630 shutdown_connections 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.033+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1600868630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103fc0 0x7f26f010a870 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.033+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1600868630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26f0102420 0x7f26f0102840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.033+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1600868630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f0103620 0x7f26f0103a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.033+0000 7f26f84ba700 1 -- 172.21.15.59:0/1600868630 >> 172.21.15.59:0/1600868630 conn(0x7f26f00fd9c0 msgr2=0x7f26f00ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.033+0000 7f26f84ba700 1 -- 172.21.15.59:0/1600868630 shutdown_connections 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.033+0000 7f26f84ba700 1 -- 172.21.15.59:0/1600868630 wait complete. 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f84ba700 1 Processor -- start 2024-05-08T05:27:01.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f84ba700 1 -- start start 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f84ba700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f0102420 0x7f26f0199500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f84ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103620 0x7f26f0199a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f84ba700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26f0103fc0 0x7f26f01a0ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f84ba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f26f010df40 con 0x7f26f0103620 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f84ba700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f26f010ddc0 con 0x7f26f0103fc0 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f84ba700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f26f010e0c0 con 0x7f26f0102420 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f5a55700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103620 0x7f26f0199a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f5a55700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103620 0x7f26f0199a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55068/0 (socket says 172.21.15.59:55068) 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f5a55700 1 -- 172.21.15.59:0/1016609157 learned_addr learned my addr 172.21.15.59:0/1016609157 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.034+0000 7f26f6256700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f0102420 0x7f26f0199500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f6a57700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26f0103fc0 0x7f26f01a0ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f5a55700 1 -- 172.21.15.59:0/1016609157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f0102420 msgr2=0x7f26f0199500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f5a55700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f0102420 0x7f26f0199500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f5a55700 1 -- 172.21.15.59:0/1016609157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26f0103fc0 msgr2=0x7f26f01a0ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f5a55700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26f0103fc0 0x7f26f01a0ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f5a55700 1 -- 172.21.15.59:0/1016609157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26f01a1160 con 0x7f26f0103620 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f6256700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f0102420 0x7f26f0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f6a57700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26f0103fc0 0x7f26f01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f5a55700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103620 0x7f26f0199a40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c9e0 tx=0x7f26e000cda0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26e77fe700 1 -- 172.21.15.59:0/1016609157 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0007ac0 con 0x7f26f0103620 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26f01a1450 con 0x7f26f0103620 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26e77fe700 1 -- 172.21.15.59:0/1016609157 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26e0007c60 con 0x7f26f0103620 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26e77fe700 1 -- 172.21.15.59:0/1016609157 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26e0018660 con 0x7f26f0103620 2024-05-08T05:27:01.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.035+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26f01a1a10 con 0x7f26f0103620 2024-05-08T05:27:01.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.036+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26f00519e0 con 0x7f26f0103620 2024-05-08T05:27:01.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.037+0000 7f26e77fe700 1 -- 172.21.15.59:0/1016609157 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f26e000f450 con 0x7f26f0103620 2024-05-08T05:27:01.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.038+0000 7f26e77fe700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f26dc0b29b0 0x7f26dc0b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:01.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.038+0000 7f26f6256700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f26dc0b29b0 0x7f26dc0b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:01.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.038+0000 7f26e77fe700 1 -- 172.21.15.59:0/1016609157 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f26e00e2b30 con 0x7f26f0103620 2024-05-08T05:27:01.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.039+0000 7f26f6256700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f26dc0b29b0 0x7f26dc0b4e70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f26f0103480 tx=0x7f26ec008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:01.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.042+0000 7f26e77fe700 1 -- 172.21.15.59:0/1016609157 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f26e0014030 con 0x7f26f0103620 2024-05-08T05:27:01.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.344+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f26f01a1ef0 con 0x7f26f0103620 2024-05-08T05:27:01.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.347+0000 7f26e77fe700 1 -- 172.21.15.59:0/1016609157 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f26e00aa390 con 0x7f26f0103620 2024-05-08T05:27:01.347 INFO:teuthology.orchestra.run.smithi059.stdout:[client.0] 2024-05-08T05:27:01.347 INFO:teuthology.orchestra.run.smithi059.stdout: key = AQAlDTtmZ0CXFBAAK53ZRbFT7rccmvEf665g8Q== 2024-05-08T05:27:01.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f26dc0b29b0 msgr2=0x7f26dc0b4e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f26dc0b29b0 0x7f26dc0b4e70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f26f0103480 tx=0x7f26ec008040 comp rx=0 tx=0).stop 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103620 msgr2=0x7f26f0199a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103620 0x7f26f0199a40 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f26e000c9e0 tx=0x7f26e000cda0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 shutdown_connections 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f26dc0b29b0 0x7f26dc0b4e70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f26f0103fc0 0x7f26f01a0ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f26f0102420 0x7f26f0199500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 --2- 172.21.15.59:0/1016609157 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f26f0103620 0x7f26f0199a40 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 >> 172.21.15.59:0/1016609157 conn(0x7f26f00fd9c0 msgr2=0x7f26f00ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 shutdown_connections 2024-05-08T05:27:01.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:01.349+0000 7f26f84ba700 1 -- 172.21.15.59:0/1016609157 wait complete. 2024-05-08T05:27:01.803 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:01 smithi059 ceph-mon[30304]: pgmap v346: 129 pgs: 129 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-05-08T05:27:01.803 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:01 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1016609157' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:27:01.803 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:01 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1016609157' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T05:27:01.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:01 smithi143 ceph-mon[34337]: pgmap v346: 129 pgs: 129 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-05-08T05:27:01.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:01 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1016609157' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:27:01.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:01 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1016609157' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T05:27:02.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:01 smithi096 ceph-mon[34253]: pgmap v346: 129 pgs: 129 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 8.8 KiB/s rd, 2.1 KiB/s wr, 11 op/s 2024-05-08T05:27:02.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:01 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1016609157' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-05-08T05:27:02.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:01 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1016609157' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-05-08T05:27:02.053 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:27:02.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-05-08T05:27:02.053 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-05-08T05:27:02.094 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-05-08T05:27:02.094 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-05-08T05:27:02.095 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph mgr dump --format=json 2024-05-08T05:27:02.387 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:03.803 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:03 smithi059 ceph-mon[30304]: pgmap v347: 129 pgs: 129 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T05:27:03.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:03 smithi143 ceph-mon[34337]: pgmap v347: 129 pgs: 129 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T05:27:04.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:03 smithi096 ceph-mon[34253]: pgmap v347: 129 pgs: 129 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.168+0000 7f0e66f27700 1 -- 172.21.15.59:0/3987998650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 msgr2=0x7f0e601098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.168+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3987998650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 0x7f0e601098f0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0e50009990 tx=0x7f0e50009ca0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.168+0000 7f0e66f27700 1 -- 172.21.15.59:0/3987998650 shutdown_connections 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.168+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3987998650 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6006bb40 0x7f0e60110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.168+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3987998650 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 0x7f0e601098f0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.168+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3987998650 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e6006b180 0x7f0e6006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.168+0000 7f0e66f27700 1 -- 172.21.15.59:0/3987998650 >> 172.21.15.59:0/3987998650 conn(0x7f0e60079350 msgr2=0x7f0e60079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.168+0000 7f0e66f27700 1 -- 172.21.15.59:0/3987998650 shutdown_connections 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.168+0000 7f0e66f27700 1 -- 172.21.15.59:0/3987998650 wait complete. 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e66f27700 1 Processor -- start 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e66f27700 1 -- start start 2024-05-08T05:27:04.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e66f27700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6006b180 0x7f0e6019f900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e66f27700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e6006bb40 0x7f0e6019fe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e66f27700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 0x7f0e60199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e66f27700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e60113720 con 0x7f0e60109510 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e66f27700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601135a0 con 0x7f0e6006b180 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e66f27700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0e601138a0 con 0x7f0e6006bb40 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e654c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 0x7f0e60199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e654c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 0x7f0e60199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55082/0 (socket says 172.21.15.59:55082) 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.169+0000 7f0e654c4700 1 -- 172.21.15.59:0/3248128542 learned_addr learned my addr 172.21.15.59:0/3248128542 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e64cc3700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6006b180 0x7f0e6019f900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e5ffff700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e6006bb40 0x7f0e6019fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e654c4700 1 -- 172.21.15.59:0/3248128542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e6006bb40 msgr2=0x7f0e6019fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e654c4700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e6006bb40 0x7f0e6019fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e654c4700 1 -- 172.21.15.59:0/3248128542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6006b180 msgr2=0x7f0e6019f900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:04.172 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e654c4700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6006b180 0x7f0e6019f900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e654c4700 1 -- 172.21.15.59:0/3248128542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e6019a220 con 0x7f0e60109510 2024-05-08T05:27:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e64cc3700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6006b180 0x7f0e6019f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:27:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e654c4700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 0x7f0e60199ac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b970 tx=0x7f0e5800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e5dffb700 1 -- 172.21.15.59:0/3248128542 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58019070 con 0x7f0e60109510 2024-05-08T05:27:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e5dffb700 1 -- 172.21.15.59:0/3248128542 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e5800cde0 con 0x7f0e60109510 2024-05-08T05:27:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e6019a510 con 0x7f0e60109510 2024-05-08T05:27:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e5dffb700 1 -- 172.21.15.59:0/3248128542 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e58007370 con 0x7f0e60109510 2024-05-08T05:27:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.170+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e601a67d0 con 0x7f0e60109510 2024-05-08T05:27:04.173 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.171+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e6010b9f0 con 0x7f0e60109510 2024-05-08T05:27:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.173+0000 7f0e5dffb700 1 -- 172.21.15.59:0/3248128542 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0e5801c070 con 0x7f0e60109510 2024-05-08T05:27:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.174+0000 7f0e5dffb700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e480b2910 0x7f0e480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.174+0000 7f0e64cc3700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e480b2910 0x7f0e480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:04.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.174+0000 7f0e5dffb700 1 -- 172.21.15.59:0/3248128542 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 9092+0+0 (secure 0 0 0) 0x7f0e580e1b20 con 0x7f0e60109510 2024-05-08T05:27:04.175 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.174+0000 7f0e64cc3700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e480b2910 0x7f0e480b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006010 tx=0x7f0e5000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:04.178 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.178+0000 7f0e5dffb700 1 -- 172.21.15.59:0/3248128542 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e58051df0 con 0x7f0e60109510 2024-05-08T05:27:04.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.464+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f0e6019aff0 con 0x7f0e60109510 2024-05-08T05:27:04.469 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.469+0000 7f0e5dffb700 1 -- 172.21.15.59:0/3248128542 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+288644 (secure 0 0 0) 0x7f0e580a9490 con 0x7f0e60109510 2024-05-08T05:27:04.473 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e480b2910 msgr2=0x7f0e480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e480b2910 0x7f0e480b4dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f0e50006010 tx=0x7f0e5000b540 comp rx=0 tx=0).stop 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 msgr2=0x7f0e60199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 0x7f0e60199ac0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f0e5800b970 tx=0x7f0e5800bc80 comp rx=0 tx=0).stop 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 shutdown_connections 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e60109510 0x7f0e60199ac0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e480b2910 0x7f0e480b4dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e6006b180 0x7f0e6019f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 --2- 172.21.15.59:0/3248128542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e6006bb40 0x7f0e6019fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:04.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 >> 172.21.15.59:0/3248128542 conn(0x7f0e60079350 msgr2=0x7f0e60101240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:04.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 shutdown_connections 2024-05-08T05:27:04.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:04.473+0000 7f0e66f27700 1 -- 172.21.15.59:0/3248128542 wait complete. 2024-05-08T05:27:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-08T05:27:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:04 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-08T05:27:04.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:04 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3248128542' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T05:27:04.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-08T05:27:04.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:04 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-08T05:27:04.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:04 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3248128542' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T05:27:05.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]: dispatch 2024-05-08T05:27:05.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:04 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]: dispatch 2024-05-08T05:27:05.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:04 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3248128542' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-05-08T05:27:05.297 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6800","nonce":546809822},{"type":"v1","addr":"172.21.15.59:6801","nonce":546809822}]},"active_addr":"172.21.15.59:6801/546809822","active_change":"2024-05-08T05:18:03.888785+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24106,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24110,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.59:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":2839794658}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3042855743}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":3190007358}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":1220365613}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.59:0","nonce":2740190298}]}]} 2024-05-08T05:27:05.302 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-05-08T05:27:05.302 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-05-08T05:27:05.302 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:27:05.558 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:05.809 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:05 smithi059 ceph-mon[30304]: pgmap v348: 129 pgs: 129 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T05:27:05.809 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-08T05:27:05.809 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-08T05:27:05.809 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:05 smithi059 ceph-mon[30304]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T05:27:05.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:05 smithi143 ceph-mon[34337]: pgmap v348: 129 pgs: 129 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T05:27:05.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-08T05:27:05.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-08T05:27:05.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:05 smithi143 ceph-mon[34337]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T05:27:06.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:05 smithi096 ceph-mon[34253]: pgmap v348: 129 pgs: 129 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 682 B/s wr, 10 op/s 2024-05-08T05:27:06.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 1]}]': finished 2024-05-08T05:27:06.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [8, 11]}]': finished 2024-05-08T05:27:06.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:05 smithi096 ceph-mon[34253]: osdmap e89: 12 total, 12 up, 12 in 2024-05-08T05:27:06.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:06 smithi059 ceph-mon[30304]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T05:27:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:06 smithi143 ceph-mon[34337]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T05:27:07.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:06 smithi096 ceph-mon[34253]: osdmap e90: 12 total, 12 up, 12 in 2024-05-08T05:27:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.235+0000 7f354727a700 1 -- 172.21.15.59:0/3363248229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540108850 msgr2=0x7f35401103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.235+0000 7f354727a700 1 --2- 172.21.15.59:0/3363248229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540108850 0x7f35401103d0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f353800b0d0 tx=0x7f353800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 -- 172.21.15.59:0/3363248229 shutdown_connections 2024-05-08T05:27:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 --2- 172.21.15.59:0/3363248229 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540108850 0x7f35401103d0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 --2- 172.21.15.59:0/3363248229 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540103750 0x7f3540103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 --2- 172.21.15.59:0/3363248229 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3540104100 0x7f3540108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 -- 172.21.15.59:0/3363248229 >> 172.21.15.59:0/3363248229 conn(0x7f354007a600 msgr2=0x7f354007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 -- 172.21.15.59:0/3363248229 shutdown_connections 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 -- 172.21.15.59:0/3363248229 wait complete. 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 Processor -- start 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 -- start start 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.236+0000 7f354727a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540103750 0x7f354019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f354727a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540104100 0x7f354019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f354727a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3540108850 0x7f3540199840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f354727a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3540113640 con 0x7f3540103750 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f354727a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f35401134c0 con 0x7f3540108850 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f354727a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f35401137c0 con 0x7f3540104100 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545016700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540103750 0x7f354019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545016700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540103750 0x7f354019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55106/0 (socket says 172.21.15.59:55106) 2024-05-08T05:27:07.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545016700 1 -- 172.21.15.59:0/4048194480 learned_addr learned my addr 172.21.15.59:0/4048194480 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545817700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3540108850 0x7f3540199840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3544815700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540104100 0x7f354019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545016700 1 -- 172.21.15.59:0/4048194480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540104100 msgr2=0x7f354019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545016700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540104100 0x7f354019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545016700 1 -- 172.21.15.59:0/4048194480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3540108850 msgr2=0x7f3540199840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545016700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3540108850 0x7f3540199840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545016700 1 -- 172.21.15.59:0/4048194480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f354019a020 con 0x7f3540103750 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f3545016700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540103750 0x7f354019f6c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f353c00d980 tx=0x7f353c00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f35367fc700 1 -- 172.21.15.59:0/4048194480 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c017070 con 0x7f3540103750 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f35367fc700 1 -- 172.21.15.59:0/4048194480 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f353c004830 con 0x7f3540103750 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f354019a310 con 0x7f3540103750 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f35367fc700 1 -- 172.21.15.59:0/4048194480 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f353c005440 con 0x7f3540103750 2024-05-08T05:27:07.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.237+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35401a6520 con 0x7f3540103750 2024-05-08T05:27:07.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.238+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3540104be0 con 0x7f3540103750 2024-05-08T05:27:07.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.240+0000 7f35367fc700 1 -- 172.21.15.59:0/4048194480 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f353c0055e0 con 0x7f3540103750 2024-05-08T05:27:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.241+0000 7f35367fc700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f352c0b2910 0x7f352c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.241+0000 7f3544815700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f352c0b2910 0x7f352c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.241+0000 7f35367fc700 1 -- 172.21.15.59:0/4048194480 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f353c0e2ab0 con 0x7f3540103750 2024-05-08T05:27:07.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.241+0000 7f3544815700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f352c0b2910 0x7f352c0b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3530005ea0 tx=0x7f3530005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:07.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.244+0000 7f35367fc700 1 -- 172.21.15.59:0/4048194480 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f353c020020 con 0x7f3540103750 2024-05-08T05:27:07.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.462+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f354002e510 con 0x7f3540103750 2024-05-08T05:27:07.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.463+0000 7f35367fc700 1 -- 172.21.15.59:0/4048194480 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24695 (secure 0 0 0) 0x7f353c0aa2e0 con 0x7f3540103750 2024-05-08T05:27:07.464 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:27:07.464 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:27:05.550089+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:27:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f352c0b2910 msgr2=0x7f352c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f352c0b2910 0x7f352c0b4dd0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f3530005ea0 tx=0x7f3530005e30 comp rx=0 tx=0).stop 2024-05-08T05:27:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540103750 msgr2=0x7f354019f6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:07.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540103750 0x7f354019f6c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f353c00d980 tx=0x7f353c00dd40 comp rx=0 tx=0).stop 2024-05-08T05:27:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 shutdown_connections 2024-05-08T05:27:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f352c0b2910 0x7f352c0b4dd0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3540108850 0x7f3540199840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3540103750 0x7f354019f6c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 --2- 172.21.15.59:0/4048194480 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3540104100 0x7f354019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 >> 172.21.15.59:0/4048194480 conn(0x7f354007a600 msgr2=0x7f35401099a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 shutdown_connections 2024-05-08T05:27:07.467 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:07.466+0000 7f354727a700 1 -- 172.21.15.59:0/4048194480 wait complete. 2024-05-08T05:27:07.829 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:07 smithi096 ceph-mon[34253]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 639 B/s wr, 11 op/s 2024-05-08T05:27:07.829 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:07 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/4048194480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:27:07.835 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:07 smithi059 ceph-mon[30304]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 639 B/s wr, 11 op/s 2024-05-08T05:27:07.835 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:07 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/4048194480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:27:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:07 smithi143 ceph-mon[34337]: pgmap v351: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 639 B/s wr, 11 op/s 2024-05-08T05:27:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:07 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/4048194480' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:27:08.270 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-05-08T05:27:08.271 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:27:08.533 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:09.728 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:09 smithi059 ceph-mon[30304]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:27:09.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:09 smithi143 ceph-mon[34337]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:27:10.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:09 smithi096 ceph-mon[34253]: pgmap v352: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.297+0000 7f8970449700 1 -- 172.21.15.59:0/2738714593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806b180 msgr2=0x7f896806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.297+0000 7f8970449700 1 --2- 172.21.15.59:0/2738714593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806b180 0x7f896806b600 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f8958009990 tx=0x7f8958009ca0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.297+0000 7f8970449700 1 -- 172.21.15.59:0/2738714593 shutdown_connections 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.297+0000 7f8970449700 1 --2- 172.21.15.59:0/2738714593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f896806bb40 0x7f8968110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.297+0000 7f8970449700 1 --2- 172.21.15.59:0/2738714593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8968109510 0x7f89681098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.297+0000 7f8970449700 1 --2- 172.21.15.59:0/2738714593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806b180 0x7f896806b600 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.297+0000 7f8970449700 1 -- 172.21.15.59:0/2738714593 >> 172.21.15.59:0/2738714593 conn(0x7f8968079350 msgr2=0x7f8968079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.297+0000 7f8970449700 1 -- 172.21.15.59:0/2738714593 shutdown_connections 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.297+0000 7f8970449700 1 -- 172.21.15.59:0/2738714593 wait complete. 2024-05-08T05:27:10.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.298+0000 7f8970449700 1 Processor -- start 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.298+0000 7f8970449700 1 -- start start 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.298+0000 7f8970449700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f896806b180 0x7f89681a1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.298+0000 7f8970449700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806bb40 0x7f89681a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.298+0000 7f8970449700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8968109510 0x7f896819bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.298+0000 7f8970449700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8968115870 con 0x7f896806bb40 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.298+0000 7f8970449700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f89681156f0 con 0x7f8968109510 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.298+0000 7f8970449700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f89681159f0 con 0x7f896806b180 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896d9e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806bb40 0x7f89681a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896d9e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806bb40 0x7f89681a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45252/0 (socket says 172.21.15.59:45252) 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896d9e4700 1 -- 172.21.15.59:0/332359693 learned_addr learned my addr 172.21.15.59:0/332359693 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896e9e6700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8968109510 0x7f896819bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896e1e5700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f896806b180 0x7f89681a1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896d9e4700 1 -- 172.21.15.59:0/332359693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f896806b180 msgr2=0x7f89681a1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:10.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896d9e4700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f896806b180 0x7f89681a1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896d9e4700 1 -- 172.21.15.59:0/332359693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8968109510 msgr2=0x7f896819bb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896d9e4700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8968109510 0x7f896819bb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896d9e4700 1 -- 172.21.15.59:0/332359693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f896819c2a0 con 0x7f896806bb40 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896e1e5700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f896806b180 0x7f89681a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f896d9e4700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806bb40 0x7f89681a1e70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8958000c00 tx=0x7f89580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f895f7fe700 1 -- 172.21.15.59:0/332359693 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8958027070 con 0x7f896806bb40 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f896819c530 con 0x7f896806bb40 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f895f7fe700 1 -- 172.21.15.59:0/332359693 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f89580040e0 con 0x7f896806bb40 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f895f7fe700 1 -- 172.21.15.59:0/332359693 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f895800f460 con 0x7f896806bb40 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.299+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f89681a8800 con 0x7f896806bb40 2024-05-08T05:27:10.318 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.300+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f896810b9f0 con 0x7f896806bb40 2024-05-08T05:27:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.302+0000 7f895f7fe700 1 -- 172.21.15.59:0/332359693 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f895802a070 con 0x7f896806bb40 2024-05-08T05:27:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.303+0000 7f895f7fe700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f89540b2910 0x7f89540b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.303+0000 7f896e1e5700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f89540b2910 0x7f89540b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.303+0000 7f895f7fe700 1 -- 172.21.15.59:0/332359693 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f89580e3710 con 0x7f896806bb40 2024-05-08T05:27:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.303+0000 7f896e1e5700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f89540b2910 0x7f89540b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89640076e0 tx=0x7f8964007670 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:10.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.306+0000 7f895f7fe700 1 -- 172.21.15.59:0/332359693 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8958013070 con 0x7f896806bb40 2024-05-08T05:27:10.522 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.521+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f896819cfe0 con 0x7f896806bb40 2024-05-08T05:27:10.523 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.523+0000 7f895f7fe700 1 -- 172.21.15.59:0/332359693 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24695 (secure 0 0 0) 0x7f89580aafd0 con 0x7f896806bb40 2024-05-08T05:27:10.524 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:27:10.524 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:27:05.550089+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:27:10.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f89540b2910 msgr2=0x7f89540b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f89540b2910 0x7f89540b4dd0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f89640076e0 tx=0x7f8964007670 comp rx=0 tx=0).stop 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806bb40 msgr2=0x7f89681a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806bb40 0x7f89681a1e70 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f8958000c00 tx=0x7f89580047b0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 shutdown_connections 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f89540b2910 0x7f89540b4dd0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8968109510 0x7f896819bb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f896806b180 0x7f89681a1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 --2- 172.21.15.59:0/332359693 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f896806bb40 0x7f89681a1e70 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 >> 172.21.15.59:0/332359693 conn(0x7f8968079350 msgr2=0x7f89681010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 shutdown_connections 2024-05-08T05:27:10.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:10.525+0000 7f8970449700 1 -- 172.21.15.59:0/332359693 wait complete. 2024-05-08T05:27:10.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:10 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/332359693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:27:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:10 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/332359693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:27:11.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:10 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/332359693' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:27:11.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-05-08T05:27:11.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-05-08T05:27:11.275 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-05-08T05:27:11.276 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-05-08T05:27:11.276 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-05-08T05:27:11.276 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-05-08T05:27:11.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-05-08T05:27:11.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-05-08T05:27:11.278 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-05-08T05:27:11.278 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-05-08T05:27:11.278 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-05-08T05:27:11.279 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-05-08T05:27:11.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:11 smithi059 ceph-mon[30304]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:27:11.942 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:11.945 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:11.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:11 smithi143 ceph-mon[34337]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:27:12.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:11 smithi096 ceph-mon[34253]: pgmap v353: 129 pgs: 1 peering, 128 active+clean; 648 KiB data, 349 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 9 op/s; 0 B/s, 0 objects/s recovering 2024-05-08T05:27:12.350 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.351 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.351 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.352 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.352 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.352 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.352 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.358 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.359 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.365 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:12.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:12 smithi059 ceph-mon[30304]: pgmap v354: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T05:27:12.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:12 smithi143 ceph-mon[34337]: pgmap v354: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T05:27:13.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:12 smithi096 ceph-mon[34253]: pgmap v354: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s rd, 9 op/s; 0 B/s, 1 keys/s, 1 objects/s recovering 2024-05-08T05:27:15.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:15 smithi059 ceph-mon[30304]: pgmap v355: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T05:27:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:15 smithi143 ceph-mon[34337]: pgmap v355: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T05:27:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:15 smithi096 ceph-mon[34253]: pgmap v355: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s; 0 B/s, 1 keys/s, 0 objects/s recovering 2024-05-08T05:27:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:17 smithi059 ceph-mon[30304]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:17.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:17 smithi143 ceph-mon[34337]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:17 smithi096 ceph-mon[34253]: pgmap v356: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 11 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:19 smithi059 ceph-mon[30304]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:19 smithi143 ceph-mon[34337]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:19 smithi096 ceph-mon[34253]: pgmap v357: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:21.322 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:21 smithi059 ceph-mon[30304]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:21 smithi143 ceph-mon[34337]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:21 smithi096 ceph-mon[34253]: pgmap v358: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.434+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2663054971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f4104100 msgr2=0x7f90f4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.434+0000 7f90f9b5b700 1 --2- 172.21.15.59:0/2663054971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f4104100 0x7f90f4108120 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f90e80099c0 tx=0x7f90e8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:27:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.434+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2663054971 shutdown_connections 2024-05-08T05:27:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.434+0000 7f90f9b5b700 1 --2- 172.21.15.59:0/2663054971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f4108850 0x7f90f41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.434+0000 7f90f9b5b700 1 --2- 172.21.15.59:0/2663054971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f4103750 0x7f90f4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:22.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.434+0000 7f90f9b5b700 1 --2- 172.21.15.59:0/2663054971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f4104100 0x7f90f4108120 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.434+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2663054971 >> 172.21.15.59:0/2663054971 conn(0x7f90f407a600 msgr2=0x7f90f407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.435+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2663054971 shutdown_connections 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.435+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2663054971 wait complete. 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.435+0000 7f90f9b5b700 1 Processor -- start 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.435+0000 7f90f9b5b700 1 -- start start 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f9b5b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f4103750 0x7f90f419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f9b5b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f4104100 0x7f90f419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f9b5b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f4108850 0x7f90f41998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f9b5b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90f4113610 con 0x7f90f4104100 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f9b5b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90f4113490 con 0x7f90f4103750 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f9b5b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f90f4113790 con 0x7f90f4108850 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f4104100 0x7f90f419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:22.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f4104100 0x7f90f419fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53898/0 (socket says 172.21.15.59:53898) 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f2ffd700 1 -- 172.21.15.59:0/2306142583 learned_addr learned my addr 172.21.15.59:0/2306142583 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f37fe700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f4103750 0x7f90f419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f3fff700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f4108850 0x7f90f41998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f2ffd700 1 -- 172.21.15.59:0/2306142583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f4108850 msgr2=0x7f90f41998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f2ffd700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f90f4108850 0x7f90f41998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f2ffd700 1 -- 172.21.15.59:0/2306142583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f4103750 msgr2=0x7f90f419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f2ffd700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f90f4103750 0x7f90f419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f2ffd700 1 -- 172.21.15.59:0/2306142583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90f419a050 con 0x7f90f4104100 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f2ffd700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f4104100 0x7f90f419fc70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f90e8006010 tx=0x7f90e80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f0ff9700 1 -- 172.21.15.59:0/2306142583 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8026070 con 0x7f90f4104100 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f0ff9700 1 -- 172.21.15.59:0/2306142583 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90e800f460 con 0x7f90f4104100 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90f419a2e0 con 0x7f90f4104100 2024-05-08T05:27:22.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.436+0000 7f90f0ff9700 1 -- 172.21.15.59:0/2306142583 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90e8009e60 con 0x7f90f4104100 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.437+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90f41a6600 con 0x7f90f4104100 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.437+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f90f4063c30 con 0x7f90f4104100 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.439+0000 7f90f0ff9700 1 -- 172.21.15.59:0/2306142583 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f90e8029070 con 0x7f90f4104100 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.440+0000 7f90f0ff9700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90dc0b2910 0x7f90dc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.440+0000 7f90f37fe700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90dc0b2910 0x7f90dc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.440+0000 7f90f0ff9700 1 -- 172.21.15.59:0/2306142583 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f90e80e3310 con 0x7f90f4104100 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.440+0000 7f90f0ff9700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] conn(0x7f90dc0bdb70 0x7f90dc0c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.440+0000 7f90f0ff9700 1 -- 172.21.15.59:0/2306142583 --> [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f90dc0c06e0 con 0x7f90dc0bdb70 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.440+0000 7f90f0ff9700 1 -- 172.21.15.59:0/2306142583 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f90e80e3760 con 0x7f90f4104100 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.440+0000 7f90f3fff700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] conn(0x7f90dc0bdb70 0x7f90dc0c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.440+0000 7f90f37fe700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90dc0b2910 0x7f90dc0b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90f419aa30 tx=0x7f90e4009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:22.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.441+0000 7f90f3fff700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] conn(0x7f90dc0bdb70 0x7f90dc0c0010 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.441+0000 7f90f0ff9700 1 -- 172.21.15.59:0/2306142583 <== osd.0 v2:172.21.15.59:6802/4120729833 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f90dc0c06e0 con 0x7f90dc0bdb70 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.458+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 --> [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f90f40640b0 con 0x7f90dc0bdb70 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f0ff9700 1 -- 172.21.15.59:0/2306142583 <== osd.0 v2:172.21.15.59:6802/4120729833 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f90f40640b0 con 0x7f90dc0bdb70 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] conn(0x7f90dc0bdb70 msgr2=0x7f90dc0c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] conn(0x7f90dc0bdb70 0x7f90dc0c0010 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90dc0b2910 msgr2=0x7f90dc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f90dc0b2910 0x7f90dc0b4dd0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f90f419aa30 tx=0x7f90e4009260 comp rx=0 tx=0).stop 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f4104100 msgr2=0x7f90f419fc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 --2- 172.21.15.59:0/2306142583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f90f4104100 0x7f90f419fc70 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f90e8006010 tx=0x7f90e80045e0 comp rx=0 tx=0).stop 2024-05-08T05:27:22.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f3fff700 1 -- 172.21.15.59:0/2306142583 reap_dead start 2024-05-08T05:27:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 shutdown_connections 2024-05-08T05:27:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 >> 172.21.15.59:0/2306142583 conn(0x7f90f407a600 msgr2=0x7f90f4101820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 shutdown_connections 2024-05-08T05:27:22.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:22.459+0000 7f90f9b5b700 1 -- 172.21.15.59:0/2306142583 wait complete. 2024-05-08T05:27:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:23 smithi059 ceph-mon[30304]: pgmap v359: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:23 smithi143 ceph-mon[34337]: pgmap v359: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:23 smithi096 ceph-mon[34253]: pgmap v359: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s; 0 B/s, 0 keys/s, 0 objects/s recovering 2024-05-08T05:27:23.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.997+0000 7fc381d33700 1 -- 172.21.15.59:0/2301613948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c06b180 msgr2=0x7fc37c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:23.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.997+0000 7fc381d33700 1 --2- 172.21.15.59:0/2301613948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c06b180 0x7fc37c06b600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc370009990 tx=0x7fc370009ca0 comp rx=0 tx=0).stop 2024-05-08T05:27:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.997+0000 7fc381d33700 1 -- 172.21.15.59:0/2301613948 shutdown_connections 2024-05-08T05:27:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.997+0000 7fc381d33700 1 --2- 172.21.15.59:0/2301613948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc37c06bb40 0x7fc37c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.997+0000 7fc381d33700 1 --2- 172.21.15.59:0/2301613948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c109510 0x7fc37c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.997+0000 7fc381d33700 1 --2- 172.21.15.59:0/2301613948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c06b180 0x7fc37c06b600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.997+0000 7fc381d33700 1 -- 172.21.15.59:0/2301613948 >> 172.21.15.59:0/2301613948 conn(0x7fc37c079350 msgr2=0x7fc37c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.997+0000 7fc381d33700 1 -- 172.21.15.59:0/2301613948 shutdown_connections 2024-05-08T05:27:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.997+0000 7fc381d33700 1 -- 172.21.15.59:0/2301613948 wait complete. 2024-05-08T05:27:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.998+0000 7fc381d33700 1 Processor -- start 2024-05-08T05:27:23.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.998+0000 7fc381d33700 1 -- start start 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.998+0000 7fc381d33700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc37c06b180 0x7fc37c102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.998+0000 7fc381d33700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c06bb40 0x7fc37c1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc381d33700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c109510 0x7fc37c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc381d33700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c113640 con 0x7fc37c06bb40 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc381d33700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1134c0 con 0x7fc37c109510 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc381d33700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc37c1137c0 con 0x7fc37c06b180 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c06bb40 0x7fc37c1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c06bb40 0x7fc37c1029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53926/0 (socket says 172.21.15.59:53926) 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37affd700 1 -- 172.21.15.59:0/443530063 learned_addr learned my addr 172.21.15.59:0/443530063 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:24.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37b7fe700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc37c06b180 0x7fc37c102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37bfff700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c109510 0x7fc37c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37affd700 1 -- 172.21.15.59:0/443530063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc37c06b180 msgr2=0x7fc37c102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37affd700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc37c06b180 0x7fc37c102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37affd700 1 -- 172.21.15.59:0/443530063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c109510 msgr2=0x7fc37c102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37affd700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc37c109510 0x7fc37c102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37affd700 1 -- 172.21.15.59:0/443530063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc37c1a80a0 con 0x7fc37c06bb40 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc37affd700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c06bb40 0x7fc37c1029b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc370000c00 tx=0x7fc3700047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc378ff9700 1 -- 172.21.15.59:0/443530063 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370027070 con 0x7fc37c06bb40 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc37c1a8330 con 0x7fc37c06bb40 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc378ff9700 1 -- 172.21.15.59:0/443530063 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc370003fa0 con 0x7fc37c06bb40 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc378ff9700 1 -- 172.21.15.59:0/443530063 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc370022db0 con 0x7fc37c06bb40 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:23.999+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc37c1a87e0 con 0x7fc37c06bb40 2024-05-08T05:27:24.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.000+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc37c063bb0 con 0x7fc37c06bb40 2024-05-08T05:27:24.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.002+0000 7fc378ff9700 1 -- 172.21.15.59:0/443530063 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fc37002a070 con 0x7fc37c06bb40 2024-05-08T05:27:24.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.003+0000 7fc378ff9700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc3640b2910 0x7fc3640b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.003+0000 7fc37b7fe700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc3640b2910 0x7fc3640b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.003+0000 7fc378ff9700 1 -- 172.21.15.59:0/443530063 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc3700e3a60 con 0x7fc37c06bb40 2024-05-08T05:27:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.003+0000 7fc378ff9700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] conn(0x7fc3640bdb70 0x7fc3640c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.003+0000 7fc378ff9700 1 -- 172.21.15.59:0/443530063 --> [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc370023e50 con 0x7fc3640bdb70 2024-05-08T05:27:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.003+0000 7fc378ff9700 1 -- 172.21.15.59:0/443530063 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fc3700e3eb0 con 0x7fc37c06bb40 2024-05-08T05:27:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.003+0000 7fc37b7fe700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc3640b2910 0x7fc3640b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc36c005fd0 tx=0x7fc36c00af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.003+0000 7fc37bfff700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] conn(0x7fc3640bdb70 0x7fc3640c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:24.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.004+0000 7fc37bfff700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] conn(0x7fc3640bdb70 0x7fc3640c0010 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:24.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.005+0000 7fc378ff9700 1 -- 172.21.15.59:0/443530063 <== osd.4 v2:172.21.15.96:6808/2351393060 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc370023e50 con 0x7fc3640bdb70 2024-05-08T05:27:24.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.022+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 --> [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc37c063fb0 con 0x7fc3640bdb70 2024-05-08T05:27:24.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.022+0000 7fc378ff9700 1 -- 172.21.15.59:0/443530063 <== osd.4 v2:172.21.15.96:6808/2351393060 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc37c063fb0 con 0x7fc3640bdb70 2024-05-08T05:27:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.022+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 >> [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] conn(0x7fc3640bdb70 msgr2=0x7fc3640c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.022+0000 7fc381d33700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] conn(0x7fc3640bdb70 0x7fc3640c0010 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.022+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc3640b2910 msgr2=0x7fc3640b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.023+0000 7fc381d33700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc3640b2910 0x7fc3640b4dd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc36c005fd0 tx=0x7fc36c00af20 comp rx=0 tx=0).stop 2024-05-08T05:27:24.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.023+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c06bb40 msgr2=0x7fc37c1029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.023+0000 7fc381d33700 1 --2- 172.21.15.59:0/443530063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc37c06bb40 0x7fc37c1029b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fc370000c00 tx=0x7fc3700047b0 comp rx=0 tx=0).stop 2024-05-08T05:27:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.023+0000 7fc37bfff700 1 -- 172.21.15.59:0/443530063 reap_dead start 2024-05-08T05:27:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.023+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 shutdown_connections 2024-05-08T05:27:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.023+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 >> 172.21.15.59:0/443530063 conn(0x7fc37c079350 msgr2=0x7fc37c1010a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.023+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 shutdown_connections 2024-05-08T05:27:24.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.023+0000 7fc381d33700 1 -- 172.21.15.59:0/443530063 wait complete. 2024-05-08T05:27:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:25 smithi059 ceph-mon[30304]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:25 smithi143 ceph-mon[34337]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:25.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:25 smithi096 ceph-mon[34253]: pgmap v360: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:27.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.066+0000 7f2853a41700 1 -- 172.21.15.59:0/2513033696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f284c06b180 msgr2=0x7f284c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.066+0000 7f2853a41700 1 --2- 172.21.15.59:0/2513033696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f284c06b180 0x7f284c06b600 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f283c009990 tx=0x7f283c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.066+0000 7f2853a41700 1 -- 172.21.15.59:0/2513033696 shutdown_connections 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.066+0000 7f2853a41700 1 --2- 172.21.15.59:0/2513033696 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f284c06bb40 0x7f284c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.066+0000 7f2853a41700 1 --2- 172.21.15.59:0/2513033696 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f284c109510 0x7f284c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.066+0000 7f2853a41700 1 --2- 172.21.15.59:0/2513033696 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f284c06b180 0x7f284c06b600 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.066+0000 7f2853a41700 1 -- 172.21.15.59:0/2513033696 >> 172.21.15.59:0/2513033696 conn(0x7f284c079350 msgr2=0x7f284c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.066+0000 7f2853a41700 1 -- 172.21.15.59:0/2513033696 shutdown_connections 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.066+0000 7f2853a41700 1 -- 172.21.15.59:0/2513033696 wait complete. 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.067+0000 7f2853a41700 1 Processor -- start 2024-05-08T05:27:27.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.067+0000 7f2853a41700 1 -- start start 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.067+0000 7f2853a41700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f284c06b180 0x7f284c102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.067+0000 7f2853a41700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f284c06bb40 0x7f284c1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.067+0000 7f2853a41700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f284c109510 0x7f284c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.067+0000 7f2853a41700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f284c113620 con 0x7f284c109510 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.067+0000 7f2853a41700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1134a0 con 0x7f284c06b180 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.067+0000 7f2853a41700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f284c1137a0 con 0x7f284c06bb40 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2851fde700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f284c109510 0x7f284c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2851fde700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f284c109510 0x7f284c102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53964/0 (socket says 172.21.15.59:53964) 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2851fde700 1 -- 172.21.15.59:0/89860477 learned_addr learned my addr 172.21.15.59:0/89860477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f28517dd700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f284c06b180 0x7f284c102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2850fdc700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f284c06bb40 0x7f284c1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2851fde700 1 -- 172.21.15.59:0/89860477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f284c06bb40 msgr2=0x7f284c1029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2851fde700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f284c06bb40 0x7f284c1029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2851fde700 1 -- 172.21.15.59:0/89860477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f284c06b180 msgr2=0x7f284c102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2851fde700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f284c06b180 0x7f284c102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2851fde700 1 -- 172.21.15.59:0/89860477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f284c1a8000 con 0x7f284c109510 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2851fde700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f284c109510 0x7f284c102ef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2844007ee0 tx=0x7f284400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f28427fc700 1 -- 172.21.15.59:0/89860477 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844004bb0 con 0x7f284c109510 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f284c1a82f0 con 0x7f284c109510 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.068+0000 7f28427fc700 1 -- 172.21.15.59:0/89860477 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2844004d50 con 0x7f284c109510 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.069+0000 7f28427fc700 1 -- 172.21.15.59:0/89860477 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2844005920 con 0x7f284c109510 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.069+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f284c1a8800 con 0x7f284c109510 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.069+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f284c063bb0 con 0x7f284c109510 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.071+0000 7f28427fc700 1 -- 172.21.15.59:0/89860477 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f284401a020 con 0x7f284c109510 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.072+0000 7f28427fc700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28380b2910 0x7f28380b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.072+0000 7f28517dd700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28380b2910 0x7f28380b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.072+0000 7f28427fc700 1 -- 172.21.15.59:0/89860477 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f28440e28a0 con 0x7f284c109510 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.072+0000 7f28427fc700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] conn(0x7f28380bdb70 0x7f28380c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.072+0000 7f28427fc700 1 -- 172.21.15.59:0/89860477 --> [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f28380c06e0 con 0x7f28380bdb70 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.072+0000 7f28427fc700 1 -- 172.21.15.59:0/89860477 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f28440e2c50 con 0x7f284c109510 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.072+0000 7f28517dd700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28380b2910 0x7f28380b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f284800abd0 tx=0x7f284800a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.072+0000 7f2850fdc700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] conn(0x7f28380bdb70 0x7f28380c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.073+0000 7f2850fdc700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] conn(0x7f28380bdb70 0x7f28380c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.073+0000 7f28427fc700 1 -- 172.21.15.59:0/89860477 <== osd.7 v2:172.21.15.96:6816/2845324468 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f28380c06e0 con 0x7f28380bdb70 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.091+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 --> [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f284c063fe0 con 0x7f28380bdb70 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f28427fc700 1 -- 172.21.15.59:0/89860477 <== osd.7 v2:172.21.15.96:6816/2845324468 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f284c063fe0 con 0x7f28380bdb70 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 >> [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] conn(0x7f28380bdb70 msgr2=0x7f28380c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] conn(0x7f28380bdb70 0x7f28380c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28380b2910 msgr2=0x7f28380b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f28380b2910 0x7f28380b4dd0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f284800abd0 tx=0x7f284800a380 comp rx=0 tx=0).stop 2024-05-08T05:27:27.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f284c109510 msgr2=0x7f284c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 --2- 172.21.15.59:0/89860477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f284c109510 0x7f284c102ef0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f2844007ee0 tx=0x7f284400d3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2851fde700 1 -- 172.21.15.59:0/89860477 reap_dead start 2024-05-08T05:27:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 shutdown_connections 2024-05-08T05:27:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 >> 172.21.15.59:0/89860477 conn(0x7f284c079350 msgr2=0x7f284c100fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 shutdown_connections 2024-05-08T05:27:27.186 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.092+0000 7f2853a41700 1 -- 172.21.15.59:0/89860477 wait complete. 2024-05-08T05:27:27.186 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:27 smithi059 ceph-mon[30304]: pgmap v361: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:27 smithi143 ceph-mon[34337]: pgmap v361: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:27.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:27 smithi096 ceph-mon[34253]: pgmap v361: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.621+0000 7f8e5201b700 1 -- 172.21.15.59:0/2020242123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c102390 msgr2=0x7f8e4c102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.621+0000 7f8e5201b700 1 --2- 172.21.15.59:0/2020242123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c102390 0x7f8e4c102810 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f8e40009990 tx=0x7f8e40009ca0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.621+0000 7f8e5201b700 1 -- 172.21.15.59:0/2020242123 shutdown_connections 2024-05-08T05:27:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.621+0000 7f8e5201b700 1 --2- 172.21.15.59:0/2020242123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e4c102d50 0x7f8e4c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.622+0000 7f8e5201b700 1 --2- 172.21.15.59:0/2020242123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c108400 0x7f8e4c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.622+0000 7f8e5201b700 1 --2- 172.21.15.59:0/2020242123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c102390 0x7f8e4c102810 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.622+0000 7f8e5201b700 1 -- 172.21.15.59:0/2020242123 >> 172.21.15.59:0/2020242123 conn(0x7f8e4c079310 msgr2=0x7f8e4c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.622+0000 7f8e5201b700 1 -- 172.21.15.59:0/2020242123 shutdown_connections 2024-05-08T05:27:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.622+0000 7f8e5201b700 1 -- 172.21.15.59:0/2020242123 wait complete. 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.622+0000 7f8e5201b700 1 Processor -- start 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.622+0000 7f8e5201b700 1 -- start start 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e5201b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e4c102390 0x7f8e4c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e5201b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c102d50 0x7f8e4c19af00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e5201b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c108400 0x7f8e4c1a5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e5201b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c112520 con 0x7f8e4c108400 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e5201b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1123a0 con 0x7f8e4c102d50 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e5201b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8e4c1126a0 con 0x7f8e4c102390 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c108400 0x7f8e4c1a5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e4c102390 0x7f8e4c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e4c102390 0x7f8e4c19a9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:44432/0 (socket says 172.21.15.59:44432) 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c108400 0x7f8e4c1a5a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53976/0 (socket says 172.21.15.59:53976) 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4b7fe700 1 -- 172.21.15.59:0/1514705527 learned_addr learned my addr 172.21.15.59:0/1514705527 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4affd700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c102d50 0x7f8e4c19af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4bfff700 1 -- 172.21.15.59:0/1514705527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e4c102390 msgr2=0x7f8e4c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4bfff700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8e4c102390 0x7f8e4c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4bfff700 1 -- 172.21.15.59:0/1514705527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c102d50 msgr2=0x7f8e4c19af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4bfff700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8e4c102d50 0x7f8e4c19af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4bfff700 1 -- 172.21.15.59:0/1514705527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e4c1a5f80 con 0x7f8e4c108400 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.623+0000 7f8e4bfff700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c108400 0x7f8e4c1a5a40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8e44007f10 tx=0x7f8e4400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.624+0000 7f8e48ff9700 1 -- 172.21.15.59:0/1514705527 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44004db0 con 0x7f8e4c108400 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.624+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e4c1a61b0 con 0x7f8e4c108400 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.624+0000 7f8e48ff9700 1 -- 172.21.15.59:0/1514705527 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e44009e20 con 0x7f8e4c108400 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.624+0000 7f8e48ff9700 1 -- 172.21.15.59:0/1514705527 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e44015400 con 0x7f8e4c108400 2024-05-08T05:27:27.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.624+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e4c1a66c0 con 0x7f8e4c108400 2024-05-08T05:27:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.624+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e4c063b70 con 0x7f8e4c108400 2024-05-08T05:27:27.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.626+0000 7f8e48ff9700 1 -- 172.21.15.59:0/1514705527 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f8e44005260 con 0x7f8e4c108400 2024-05-08T05:27:27.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.627+0000 7f8e48ff9700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8e340b2990 0x7f8e340b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.627+0000 7f8e4b7fe700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8e340b2990 0x7f8e340b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.627+0000 7f8e48ff9700 1 -- 172.21.15.59:0/1514705527 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f8e440e2610 con 0x7f8e4c108400 2024-05-08T05:27:27.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.627+0000 7f8e48ff9700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] conn(0x7f8e340bdbf0 0x7f8e340c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.627+0000 7f8e48ff9700 1 -- 172.21.15.59:0/1514705527 --> [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e4400ae50 con 0x7f8e340bdbf0 2024-05-08T05:27:27.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.627+0000 7f8e48ff9700 1 -- 172.21.15.59:0/1514705527 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f8e440e29c0 con 0x7f8e4c108400 2024-05-08T05:27:27.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.628+0000 7f8e4b7fe700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8e340b2990 0x7f8e340b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c005950 tx=0x7f8e3c00b500 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:27.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.628+0000 7f8e4affd700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] conn(0x7f8e340bdbf0 0x7f8e340c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.628+0000 7f8e4affd700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] conn(0x7f8e340bdbf0 0x7f8e340c0090 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:27.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.629+0000 7f8e48ff9700 1 -- 172.21.15.59:0/1514705527 <== osd.8 v2:172.21.15.143:6816/2760087187 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e4400ae50 con 0x7f8e340bdbf0 2024-05-08T05:27:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.646+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 --> [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e4c064020 con 0x7f8e340bdbf0 2024-05-08T05:27:27.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.647+0000 7f8e48ff9700 1 -- 172.21.15.59:0/1514705527 <== osd.8 v2:172.21.15.143:6816/2760087187 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8e4c064020 con 0x7f8e340bdbf0 2024-05-08T05:27:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.647+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 >> [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] conn(0x7f8e340bdbf0 msgr2=0x7f8e340c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.647+0000 7f8e5201b700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] conn(0x7f8e340bdbf0 0x7f8e340c0090 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.647+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8e340b2990 msgr2=0x7f8e340b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.647+0000 7f8e5201b700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f8e340b2990 0x7f8e340b4e50 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f8e3c005950 tx=0x7f8e3c00b500 comp rx=0 tx=0).stop 2024-05-08T05:27:27.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.647+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c108400 msgr2=0x7f8e4c1a5a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.647+0000 7f8e5201b700 1 --2- 172.21.15.59:0/1514705527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8e4c108400 0x7f8e4c1a5a40 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f8e44007f10 tx=0x7f8e4400d3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.647+0000 7f8e4bfff700 1 -- 172.21.15.59:0/1514705527 reap_dead start 2024-05-08T05:27:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.648+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 shutdown_connections 2024-05-08T05:27:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.648+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 >> 172.21.15.59:0/1514705527 conn(0x7f8e4c079310 msgr2=0x7f8e4c101020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.648+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 shutdown_connections 2024-05-08T05:27:27.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.648+0000 7f8e5201b700 1 -- 172.21.15.59:0/1514705527 wait complete. 2024-05-08T05:27:27.726 INFO:teuthology.orchestra.run.smithi059.stdout:133143986252 2024-05-08T05:27:27.726 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.804+0000 7f46a6fc2700 1 -- 172.21.15.59:0/3543063019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a01023a0 msgr2=0x7f46a0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.804+0000 7f46a6fc2700 1 --2- 172.21.15.59:0/3543063019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a01023a0 0x7f46a0102820 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f468c009990 tx=0x7f468c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.805+0000 7f46a6fc2700 1 -- 172.21.15.59:0/3543063019 shutdown_connections 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.805+0000 7f46a6fc2700 1 --2- 172.21.15.59:0/3543063019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a0102d60 0x7f46a010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.805+0000 7f46a6fc2700 1 --2- 172.21.15.59:0/3543063019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a0108410 0x7f46a01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.805+0000 7f46a6fc2700 1 --2- 172.21.15.59:0/3543063019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a01023a0 0x7f46a0102820 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.805+0000 7f46a6fc2700 1 -- 172.21.15.59:0/3543063019 >> 172.21.15.59:0/3543063019 conn(0x7f46a0079350 msgr2=0x7f46a0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.805+0000 7f46a6fc2700 1 -- 172.21.15.59:0/3543063019 shutdown_connections 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.805+0000 7f46a6fc2700 1 -- 172.21.15.59:0/3543063019 wait complete. 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.805+0000 7f46a6fc2700 1 Processor -- start 2024-05-08T05:27:27.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.805+0000 7f46a6fc2700 1 -- start start 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a6fc2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a01023a0 0x7f46a01a0160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a6fc2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a0102d60 0x7f46a01a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a6fc2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0108410 0x7f46a01a4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a555f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0108410 0x7f46a01a4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a555f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0108410 0x7f46a01a4a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53994/0 (socket says 172.21.15.59:53994) 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f469ffff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a0102d60 0x7f46a01a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a555f700 1 -- 172.21.15.59:0/4260520708 learned_addr learned my addr 172.21.15.59:0/4260520708 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01114a0 con 0x7f46a0108410 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a4d5e700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a01023a0 0x7f46a01a0160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0111320 con 0x7f46a01023a0 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0111620 con 0x7f46a0102d60 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a555f700 1 -- 172.21.15.59:0/4260520708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a0102d60 msgr2=0x7f46a01a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.890 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a555f700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a0102d60 0x7f46a01a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a555f700 1 -- 172.21.15.59:0/4260520708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a01023a0 msgr2=0x7f46a01a0160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a555f700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a01023a0 0x7f46a01a0160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.806+0000 7f46a555f700 1 -- 172.21.15.59:0/4260520708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a01a5170 con 0x7f46a0108410 2024-05-08T05:27:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.807+0000 7f46a555f700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0108410 0x7f46a01a4a30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4698007ee0 tx=0x7f469800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.807+0000 7f469dffb700 1 -- 172.21.15.59:0/4260520708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698004bb0 con 0x7f46a0108410 2024-05-08T05:27:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.807+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a01a5460 con 0x7f46a0108410 2024-05-08T05:27:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.807+0000 7f469dffb700 1 -- 172.21.15.59:0/4260520708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4698004d50 con 0x7f46a0108410 2024-05-08T05:27:27.891 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.808+0000 7f469dffb700 1 -- 172.21.15.59:0/4260520708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698005920 con 0x7f46a0108410 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.808+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a01acce0 con 0x7f46a0108410 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.809+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f46a0063c30 con 0x7f46a0108410 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.811+0000 7f469dffb700 1 -- 172.21.15.59:0/4260520708 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f469801a020 con 0x7f46a0108410 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.812+0000 7f469dffb700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f46900b27d0 0x7f46900b4c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.812+0000 7f469dffb700 1 -- 172.21.15.59:0/4260520708 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f46980e15b0 con 0x7f46a0108410 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.812+0000 7f46a4d5e700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f46900b27d0 0x7f46900b4c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.812+0000 7f469dffb700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] conn(0x7f46900bda30 0x7f46900bfed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.812+0000 7f469dffb700 1 -- 172.21.15.59:0/4260520708 --> [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f469800fec0 con 0x7f46900bda30 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.812+0000 7f469dffb700 1 -- 172.21.15.59:0/4260520708 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f46980e8050 con 0x7f46a0108410 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.812+0000 7f469ffff700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] conn(0x7f46900bda30 0x7f46900bfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.812+0000 7f46a4d5e700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f46900b27d0 0x7f46900b4c90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4694005950 tx=0x7f469400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.813+0000 7f469ffff700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] conn(0x7f46900bda30 0x7f46900bfed0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.814+0000 7f469dffb700 1 -- 172.21.15.59:0/4260520708 <== osd.10 v2:172.21.15.96:6824/3187941389 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f469800fec0 con 0x7f46900bda30 2024-05-08T05:27:27.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.832+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 --> [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f46a00640e0 con 0x7f46900bda30 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.832+0000 7f469dffb700 1 -- 172.21.15.59:0/4260520708 <== osd.10 v2:172.21.15.96:6824/3187941389 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f46a00640e0 con 0x7f46900bda30 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 >> [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] conn(0x7f46900bda30 msgr2=0x7f46900bfed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] conn(0x7f46900bda30 0x7f46900bfed0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f46900b27d0 msgr2=0x7f46900b4c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f46900b27d0 0x7f46900b4c90 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f4694005950 tx=0x7f469400a300 comp rx=0 tx=0).stop 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0108410 msgr2=0x7f46a01a4a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 --2- 172.21.15.59:0/4260520708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0108410 0x7f46a01a4a30 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f4698007ee0 tx=0x7f469800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a555f700 1 -- 172.21.15.59:0/4260520708 reap_dead start 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 shutdown_connections 2024-05-08T05:27:27.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 >> 172.21.15.59:0/4260520708 conn(0x7f46a0079350 msgr2=0x7f46a01012b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:27.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 shutdown_connections 2024-05-08T05:27:27.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:27.833+0000 7f46a6fc2700 1 -- 172.21.15.59:0/4260520708 wait complete. 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.636+0000 7f1ad1cec700 1 -- 172.21.15.59:0/1066332159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1acc108850 msgr2=0x7f1acc1103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.636+0000 7f1ad1cec700 1 --2- 172.21.15.59:0/1066332159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1acc108850 0x7f1acc1103d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1ac400b0d0 tx=0x7f1ac400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.636+0000 7f1ad1cec700 1 -- 172.21.15.59:0/1066332159 shutdown_connections 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.636+0000 7f1ad1cec700 1 --2- 172.21.15.59:0/1066332159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1acc108850 0x7f1acc1103d0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.636+0000 7f1ad1cec700 1 --2- 172.21.15.59:0/1066332159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1acc103750 0x7f1acc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.636+0000 7f1ad1cec700 1 --2- 172.21.15.59:0/1066332159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1acc104100 0x7f1acc108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.636+0000 7f1ad1cec700 1 -- 172.21.15.59:0/1066332159 >> 172.21.15.59:0/1066332159 conn(0x7f1acc07a600 msgr2=0x7f1acc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.637+0000 7f1ad1cec700 1 -- 172.21.15.59:0/1066332159 shutdown_connections 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.637+0000 7f1ad1cec700 1 -- 172.21.15.59:0/1066332159 wait complete. 2024-05-08T05:27:28.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.637+0000 7f1ad1cec700 1 Processor -- start 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.637+0000 7f1ad1cec700 1 -- start start 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1ad1cec700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1acc103750 0x7f1acc1a1920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1ad1cec700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1acc104100 0x7f1acc1a1e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1ad1cec700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1acc108850 0x7f1acc19bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1ad1cec700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc113640 con 0x7f1acc103750 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1ad1cec700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1134c0 con 0x7f1acc104100 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1ad1cec700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1acc1137c0 con 0x7f1acc108850 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1acc103750 0x7f1acc1a1920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1acc103750 0x7f1acc1a1920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53946/0 (socket says 172.21.15.59:53946) 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acb7fe700 1 -- 172.21.15.59:0/2698231304 learned_addr learned my addr 172.21.15.59:0/2698231304 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acbfff700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1acc108850 0x7f1acc19bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acaffd700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1acc104100 0x7f1acc1a1e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:28.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acb7fe700 1 -- 172.21.15.59:0/2698231304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1acc108850 msgr2=0x7f1acc19bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acb7fe700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1acc108850 0x7f1acc19bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acb7fe700 1 -- 172.21.15.59:0/2698231304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1acc104100 msgr2=0x7f1acc1a1e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acb7fe700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1acc104100 0x7f1acc1a1e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acb7fe700 1 -- 172.21.15.59:0/2698231304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1acc19c240 con 0x7f1acc103750 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acbfff700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1acc108850 0x7f1acc19bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acaffd700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1acc104100 0x7f1acc1a1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.638+0000 7f1acb7fe700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1acc103750 0x7f1acc1a1920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00d950 tx=0x7f1abc00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.639+0000 7f1ac8ff9700 1 -- 172.21.15.59:0/2698231304 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc016070 con 0x7f1acc103750 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.639+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1acc19c530 con 0x7f1acc103750 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.639+0000 7f1ac8ff9700 1 -- 172.21.15.59:0/2698231304 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1abc010460 con 0x7f1acc103750 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.639+0000 7f1ac8ff9700 1 -- 172.21.15.59:0/2698231304 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1abc00f620 con 0x7f1acc103750 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.639+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1acc1a87f0 con 0x7f1acc103750 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.639+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f1acc076a30 con 0x7f1acc103750 2024-05-08T05:27:28.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.641+0000 7f1ac8ff9700 1 -- 172.21.15.59:0/2698231304 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f1abc019070 con 0x7f1acc103750 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.642+0000 7f1ac8ff9700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ab40b2910 0x7f1ab40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.642+0000 7f1acaffd700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ab40b2910 0x7f1ab40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.642+0000 7f1ac8ff9700 1 -- 172.21.15.59:0/2698231304 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1abc0e28f0 con 0x7f1acc103750 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.642+0000 7f1ac8ff9700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] conn(0x7f1ab40bdb70 0x7f1ab40c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.642+0000 7f1ac8ff9700 1 -- 172.21.15.59:0/2698231304 --> [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f1ab40c06e0 con 0x7f1ab40bdb70 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.642+0000 7f1ac8ff9700 1 -- 172.21.15.59:0/2698231304 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f1abc0e2d40 con 0x7f1acc103750 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.642+0000 7f1acaffd700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ab40b2910 0x7f1ab40b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005fd0 tx=0x7f1ac0005dc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.643+0000 7f1acbfff700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] conn(0x7f1ab40bdb70 0x7f1ab40c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.643+0000 7f1acbfff700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] conn(0x7f1ab40bdb70 0x7f1ab40c0010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.644+0000 7f1ac8ff9700 1 -- 172.21.15.59:0/2698231304 <== osd.11 v2:172.21.15.143:6824/1379436601 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f1ab40c06e0 con 0x7f1ab40bdb70 2024-05-08T05:27:28.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.661+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 --> [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f1acc104be0 con 0x7f1ab40bdb70 2024-05-08T05:27:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.661+0000 7f1ac8ff9700 1 -- 172.21.15.59:0/2698231304 <== osd.11 v2:172.21.15.143:6824/1379436601 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f1acc104be0 con 0x7f1ab40bdb70 2024-05-08T05:27:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 >> [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] conn(0x7f1ab40bdb70 msgr2=0x7f1ab40c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] conn(0x7f1ab40bdb70 0x7f1ab40c0010 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ab40b2910 msgr2=0x7f1ab40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ab40b2910 0x7f1ab40b4dd0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f1ac0005fd0 tx=0x7f1ac0005dc0 comp rx=0 tx=0).stop 2024-05-08T05:27:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1acc103750 msgr2=0x7f1acc1a1920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 --2- 172.21.15.59:0/2698231304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1acc103750 0x7f1acc1a1920 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1abc00d950 tx=0x7f1abc00dd10 comp rx=0 tx=0).stop 2024-05-08T05:27:28.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1acbfff700 1 -- 172.21.15.59:0/2698231304 reap_dead start 2024-05-08T05:27:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 shutdown_connections 2024-05-08T05:27:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 >> 172.21.15.59:0/2698231304 conn(0x7f1acc07a600 msgr2=0x7f1acc1018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 shutdown_connections 2024-05-08T05:27:28.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:24.662+0000 7f1ad1cec700 1 -- 172.21.15.59:0/2698231304 wait complete. 2024-05-08T05:27:28.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:28 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:27:28.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:28 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:27:28.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:28 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:27:29.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:29 smithi143 ceph-mon[34337]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:29.400 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:29 smithi059 ceph-mon[30304]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:29 smithi096 ceph-mon[34253]: pgmap v362: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:30.061 INFO:teuthology.orchestra.run.smithi059.stdout:201863462972 2024-05-08T05:27:30.062 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-05-08T05:27:30.764 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:30.764 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:30.765 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:30.765 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:30 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:30.765 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:30 smithi059 ceph-mon[30304]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:30 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:30.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:30 smithi143 ceph-mon[34337]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:30.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.994+0000 7f946662c700 1 -- 172.21.15.59:0/2877243547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9460102d50 msgr2=0x7f946010f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:30.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.994+0000 7f946662c700 1 --2- 172.21.15.59:0/2877243547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9460102d50 0x7f946010f240 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f945800b0a0 tx=0x7f945800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:30.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.994+0000 7f946662c700 1 -- 172.21.15.59:0/2877243547 shutdown_connections 2024-05-08T05:27:30.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.994+0000 7f946662c700 1 --2- 172.21.15.59:0/2877243547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9460102d50 0x7f946010f240 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:30.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.994+0000 7f946662c700 1 --2- 172.21.15.59:0/2877243547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9460108400 0x7f94601087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:30.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.994+0000 7f946662c700 1 --2- 172.21.15.59:0/2877243547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9460102390 0x7f9460102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:30.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.994+0000 7f946662c700 1 -- 172.21.15.59:0/2877243547 >> 172.21.15.59:0/2877243547 conn(0x7f9460079310 msgr2=0x7f9460079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:30.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.994+0000 7f946662c700 1 -- 172.21.15.59:0/2877243547 shutdown_connections 2024-05-08T05:27:30.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.994+0000 7f946662c700 1 -- 172.21.15.59:0/2877243547 wait complete. 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f946662c700 1 Processor -- start 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f946662c700 1 -- start start 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f946662c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9460102390 0x7f9460104070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f946662c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9460102d50 0x7f94601045b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f946662c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9460108400 0x7f9460104af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f946662c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9460112550 con 0x7f9460102390 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f946662c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f94601123d0 con 0x7f9460108400 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f946662c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f94601126d0 con 0x7f9460102d50 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f945ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9460102390 0x7f9460104070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f945ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9460102390 0x7f9460104070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34750/0 (socket says 172.21.15.59:34750) 2024-05-08T05:27:30.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f945f7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9460102d50 0x7f94601045b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.995+0000 7f945ffff700 1 -- 172.21.15.59:0/1736158944 learned_addr learned my addr 172.21.15.59:0/1736158944 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f9464bc9700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9460108400 0x7f9460104af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f945f7fe700 1 -- 172.21.15.59:0/1736158944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9460108400 msgr2=0x7f9460104af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f945f7fe700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9460108400 0x7f9460104af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f945f7fe700 1 -- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9460102390 msgr2=0x7f9460104070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f945f7fe700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9460102390 0x7f9460104070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f945f7fe700 1 -- 172.21.15.59:0/1736158944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94601a7fb0 con 0x7f9460102d50 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f945f7fe700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9460102d50 0x7f94601045b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f945400e9f0 tx=0x7f945400ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f945d7fa700 1 -- 172.21.15.59:0/1736158944 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f945400cbd0 con 0x7f9460102d50 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94601a82a0 con 0x7f9460102d50 2024-05-08T05:27:30.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.996+0000 7f945d7fa700 1 -- 172.21.15.59:0/1736158944 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94540044d0 con 0x7f9460102d50 2024-05-08T05:27:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.997+0000 7f945d7fa700 1 -- 172.21.15.59:0/1736158944 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9454005020 con 0x7f9460102d50 2024-05-08T05:27:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.997+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94601a8790 con 0x7f9460102d50 2024-05-08T05:27:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.998+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f946006bce0 con 0x7f9460102d50 2024-05-08T05:27:31.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:30.999+0000 7f945d7fa700 1 -- 172.21.15.59:0/1736158944 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f9454004750 con 0x7f9460102d50 2024-05-08T05:27:31.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.000+0000 7f945d7fa700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f94480b2910 0x7f94480b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:31.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.000+0000 7f945d7fa700 1 -- 172.21.15.59:0/1736158944 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f94540e2850 con 0x7f9460102d50 2024-05-08T05:27:31.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.000+0000 7f945ffff700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f94480b2910 0x7f94480b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:31.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.000+0000 7f945d7fa700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] conn(0x7f94480bdb60 0x7f94480c0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:31.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.000+0000 7f945d7fa700 1 -- 172.21.15.59:0/1736158944 --> [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94480c06d0 con 0x7f94480bdb60 2024-05-08T05:27:31.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.000+0000 7f945d7fa700 1 -- 172.21.15.59:0/1736158944 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f94540e2c00 con 0x7f9460102d50 2024-05-08T05:27:31.002 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.000+0000 7f9464bc9700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] conn(0x7f94480bdb60 0x7f94480c0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.001+0000 7f945ffff700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f94480b2910 0x7f94480b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f945000aca0 tx=0x7f9450005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.001+0000 7f9464bc9700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] conn(0x7f94480bdb60 0x7f94480c0000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:31.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.002+0000 7f945d7fa700 1 -- 172.21.15.59:0/1736158944 <== osd.9 v2:172.21.15.59:6826/440517197 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94480c06d0 con 0x7f94480bdb60 2024-05-08T05:27:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.020+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 --> [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9460103ab0 con 0x7f94480bdb60 2024-05-08T05:27:31.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.020+0000 7f945d7fa700 1 -- 172.21.15.59:0/1736158944 <== osd.9 v2:172.21.15.59:6826/440517197 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9460103ab0 con 0x7f94480bdb60 2024-05-08T05:27:31.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] conn(0x7f94480bdb60 msgr2=0x7f94480c0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] conn(0x7f94480bdb60 0x7f94480c0000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f94480b2910 msgr2=0x7f94480b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f94480b2910 0x7f94480b4dd0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f945000aca0 tx=0x7f9450005f90 comp rx=0 tx=0).stop 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9460102d50 msgr2=0x7f94601045b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 --2- 172.21.15.59:0/1736158944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9460102d50 0x7f94601045b0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f945400e9f0 tx=0x7f945400ed00 comp rx=0 tx=0).stop 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f9464bc9700 1 -- 172.21.15.59:0/1736158944 reap_dead start 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 shutdown_connections 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 >> 172.21.15.59:0/1736158944 conn(0x7f9460079310 msgr2=0x7f94601015e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 shutdown_connections 2024-05-08T05:27:31.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:31.021+0000 7f946662c700 1 -- 172.21.15.59:0/1736158944 wait complete. 2024-05-08T05:27:31.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:31.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:31.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:31.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:30 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:27:31.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:30 smithi096 ceph-mon[34253]: pgmap v363: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:31.554 INFO:teuthology.orchestra.run.smithi059.stdout:304942678055 2024-05-08T05:27:31.554 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-05-08T05:27:31.685 INFO:teuthology.orchestra.run.smithi059.stdout:34359738463 2024-05-08T05:27:31.685 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-05-08T05:27:32.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.040+0000 7fe7114ad700 1 -- 172.21.15.59:0/1450406934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c06c0b0 msgr2=0x7fe70c06c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:32.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.040+0000 7fe7114ad700 1 --2- 172.21.15.59:0/1450406934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c06c0b0 0x7fe70c06c510 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fe700009990 tx=0x7fe700009ca0 comp rx=0 tx=0).stop 2024-05-08T05:27:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.040+0000 7fe7114ad700 1 -- 172.21.15.59:0/1450406934 shutdown_connections 2024-05-08T05:27:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.040+0000 7fe7114ad700 1 --2- 172.21.15.59:0/1450406934 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe70c10fdf0 0x7fe70c1121e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.040+0000 7fe7114ad700 1 --2- 172.21.15.59:0/1450406934 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c1043b0 0x7fe70c06bb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.040+0000 7fe7114ad700 1 --2- 172.21.15.59:0/1450406934 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c06c0b0 0x7fe70c06c510 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.040+0000 7fe7114ad700 1 -- 172.21.15.59:0/1450406934 >> 172.21.15.59:0/1450406934 conn(0x7fe70c079350 msgr2=0x7fe70c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.040+0000 7fe7114ad700 1 -- 172.21.15.59:0/1450406934 shutdown_connections 2024-05-08T05:27:32.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.040+0000 7fe7114ad700 1 -- 172.21.15.59:0/1450406934 wait complete. 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe7114ad700 1 Processor -- start 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe7114ad700 1 -- start start 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe7114ad700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c06c0b0 0x7fe70c19bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe7114ad700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe70c1043b0 0x7fe70c19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe7114ad700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c10fdf0 0x7fe70c1a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe7114ad700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c115880 con 0x7fe70c06c0b0 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe7114ad700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c115700 con 0x7fe70c10fdf0 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe7114ad700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c115a00 con 0x7fe70c1043b0 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c06c0b0 0x7fe70c19bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c06c0b0 0x7fe70c19bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34774/0 (socket says 172.21.15.59:34774) 2024-05-08T05:27:32.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70affd700 1 -- 172.21.15.59:0/3032599809 learned_addr learned my addr 172.21.15.59:0/3032599809 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70b7fe700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c10fdf0 0x7fe70c1a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70a7fc700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe70c1043b0 0x7fe70c19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70affd700 1 -- 172.21.15.59:0/3032599809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe70c1043b0 msgr2=0x7fe70c19c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70affd700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe70c1043b0 0x7fe70c19c290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70affd700 1 -- 172.21.15.59:0/3032599809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c10fdf0 msgr2=0x7fe70c1a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70affd700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe70c10fdf0 0x7fe70c1a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.041+0000 7fe70affd700 1 -- 172.21.15.59:0/3032599809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c1a0d60 con 0x7fe70c06c0b0 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.042+0000 7fe70affd700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c06c0b0 0x7fe70c19bd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00e9a0 tx=0x7fe6fc00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.042+0000 7fe6f3fff700 1 -- 172.21.15.59:0/3032599809 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc00cbd0 con 0x7fe70c06c0b0 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.042+0000 7fe6f3fff700 1 -- 172.21.15.59:0/3032599809 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6fc00cd70 con 0x7fe70c06c0b0 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.042+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c1a1050 con 0x7fe70c06c0b0 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.042+0000 7fe6f3fff700 1 -- 172.21.15.59:0/3032599809 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6fc018680 con 0x7fe70c06c0b0 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.042+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1a88d0 con 0x7fe70c06c0b0 2024-05-08T05:27:32.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.043+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fe70c1021d0 con 0x7fe70c06c0b0 2024-05-08T05:27:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.044+0000 7fe6f3fff700 1 -- 172.21.15.59:0/3032599809 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fe6fc010970 con 0x7fe70c06c0b0 2024-05-08T05:27:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.045+0000 7fe6f3fff700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe6f40b2910 0x7fe6f40b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.045+0000 7fe70a7fc700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe6f40b2910 0x7fe6f40b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.045+0000 7fe6f3fff700 1 -- 172.21.15.59:0/3032599809 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe6fc0e32f0 con 0x7fe70c06c0b0 2024-05-08T05:27:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.045+0000 7fe6f3fff700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] conn(0x7fe6f40bdb70 0x7fe6f40c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.045+0000 7fe6f3fff700 1 -- 172.21.15.59:0/3032599809 --> [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fe6f40c06e0 con 0x7fe6f40bdb70 2024-05-08T05:27:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.045+0000 7fe6f3fff700 1 -- 172.21.15.59:0/3032599809 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fe6fc0e3740 con 0x7fe70c06c0b0 2024-05-08T05:27:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.046+0000 7fe70a7fc700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe6f40b2910 0x7fe6f40b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe700006010 tx=0x7fe70001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:32.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.046+0000 7fe70b7fe700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] conn(0x7fe6f40bdb70 0x7fe6f40c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.046+0000 7fe70b7fe700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] conn(0x7fe6f40bdb70 0x7fe6f40c0010 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:32.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.047+0000 7fe6f3fff700 1 -- 172.21.15.59:0/3032599809 <== osd.5 v2:172.21.15.143:6808/1078449086 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fe6f40c06e0 con 0x7fe6f40bdb70 2024-05-08T05:27:32.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.070+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 --> [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fe70c06b420 con 0x7fe6f40bdb70 2024-05-08T05:27:32.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe6f3fff700 1 -- 172.21.15.59:0/3032599809 <== osd.5 v2:172.21.15.143:6808/1078449086 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fe70c06b420 con 0x7fe6f40bdb70 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 >> [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] conn(0x7fe6f40bdb70 msgr2=0x7fe6f40c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] conn(0x7fe6f40bdb70 0x7fe6f40c0010 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe6f40b2910 msgr2=0x7fe6f40b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe6f40b2910 0x7fe6f40b4dd0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fe700006010 tx=0x7fe70001a040 comp rx=0 tx=0).stop 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c06c0b0 msgr2=0x7fe70c19bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 --2- 172.21.15.59:0/3032599809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe70c06c0b0 0x7fe70c19bd50 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc00e9a0 tx=0x7fe6fc00ed60 comp rx=0 tx=0).stop 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe70b7fe700 1 -- 172.21.15.59:0/3032599809 reap_dead start 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 shutdown_connections 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 >> 172.21.15.59:0/3032599809 conn(0x7fe70c079350 msgr2=0x7fe70c101770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 shutdown_connections 2024-05-08T05:27:32.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.071+0000 7fe7114ad700 1 -- 172.21.15.59:0/3032599809 wait complete. 2024-05-08T05:27:33.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:33 smithi059 ceph-mon[30304]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-05-08T05:27:33.397 INFO:teuthology.orchestra.run.smithi059.stdout:231928234041 2024-05-08T05:27:33.398 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-05-08T05:27:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:33 smithi143 ceph-mon[34337]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-05-08T05:27:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:33 smithi096 ceph-mon[34253]: pgmap v364: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 10 op/s 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.264+0000 7ffaa5159700 1 -- 172.21.15.59:0/114400500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0102d60 msgr2=0x7ffaa010f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.264+0000 7ffaa5159700 1 --2- 172.21.15.59:0/114400500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0102d60 0x7ffaa010f250 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800b0a0 tx=0x7ffa9800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.264+0000 7ffaa5159700 1 -- 172.21.15.59:0/114400500 shutdown_connections 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.264+0000 7ffaa5159700 1 --2- 172.21.15.59:0/114400500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0102d60 0x7ffaa010f250 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.264+0000 7ffaa5159700 1 --2- 172.21.15.59:0/114400500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffaa0108410 0x7ffaa01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.264+0000 7ffaa5159700 1 --2- 172.21.15.59:0/114400500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa01023a0 0x7ffaa0102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.264+0000 7ffaa5159700 1 -- 172.21.15.59:0/114400500 >> 172.21.15.59:0/114400500 conn(0x7ffaa0079350 msgr2=0x7ffaa0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.264+0000 7ffaa5159700 1 -- 172.21.15.59:0/114400500 shutdown_connections 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.264+0000 7ffaa5159700 1 -- 172.21.15.59:0/114400500 wait complete. 2024-05-08T05:27:33.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffaa5159700 1 Processor -- start 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffaa5159700 1 -- start start 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffaa5159700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffaa01023a0 0x7ffaa019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffaa5159700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa0102d60 0x7ffaa019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffaa5159700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0108410 0x7ffaa01998c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffaa5159700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa0112550 con 0x7ffaa0108410 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffaa5159700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01123d0 con 0x7ffaa0102d60 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffaa5159700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa01126d0 con 0x7ffaa01023a0 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffa9f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0108410 0x7ffaa01998c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffa9f59e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0108410 0x7ffaa01998c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34796/0 (socket says 172.21.15.59:34796) 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffa9f59e700 1 -- 172.21.15.59:0/2422664290 learned_addr learned my addr 172.21.15.59:0/2422664290 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:33.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffa9e59c700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa0102d60 0x7ffaa019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.265+0000 7ffa9ed9d700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffaa01023a0 0x7ffaa019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffa9f59e700 1 -- 172.21.15.59:0/2422664290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffaa01023a0 msgr2=0x7ffaa019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffa9f59e700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffaa01023a0 0x7ffaa019f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffa9f59e700 1 -- 172.21.15.59:0/2422664290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa0102d60 msgr2=0x7ffaa019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffa9f59e700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffaa0102d60 0x7ffaa019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffa9f59e700 1 -- 172.21.15.59:0/2422664290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa019a020 con 0x7ffaa0108410 2024-05-08T05:27:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffa9f59e700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0108410 0x7ffaa01998c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800b830 tx=0x7ffa98004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:33.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffa87fff700 1 -- 172.21.15.59:0/2422664290 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa98021070 con 0x7ffaa0108410 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa019a2b0 con 0x7ffaa0108410 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffa87fff700 1 -- 172.21.15.59:0/2422664290 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa98007e30 con 0x7ffaa0108410 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffa87fff700 1 -- 172.21.15.59:0/2422664290 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa980047d0 con 0x7ffaa0108410 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.266+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa0103fb0 con 0x7ffaa0108410 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.267+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffaa0063c30 con 0x7ffaa0108410 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.268+0000 7ffa87fff700 1 -- 172.21.15.59:0/2422664290 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7ffa98007980 con 0x7ffaa0108410 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.269+0000 7ffa87fff700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ffa880b2990 0x7ffa880b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.269+0000 7ffa9ed9d700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ffa880b2990 0x7ffa880b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.269+0000 7ffa87fff700 1 -- 172.21.15.59:0/2422664290 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ffa9800f020 con 0x7ffaa0108410 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.269+0000 7ffa87fff700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] conn(0x7ffa880bdbf0 0x7ffa880c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:33.700 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.269+0000 7ffa87fff700 1 -- 172.21.15.59:0/2422664290 --> [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa980040a0 con 0x7ffa880bdbf0 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.269+0000 7ffa87fff700 1 -- 172.21.15.59:0/2422664290 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7ffa980e4240 con 0x7ffaa0108410 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.270+0000 7ffa9e59c700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] conn(0x7ffa880bdbf0 0x7ffa880c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.270+0000 7ffa9ed9d700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ffa880b2990 0x7ffa880b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa90005e10 tx=0x7ffa90005d40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.270+0000 7ffa9e59c700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] conn(0x7ffa880bdbf0 0x7ffa880c0090 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.271+0000 7ffa87fff700 1 -- 172.21.15.59:0/2422664290 <== osd.6 v2:172.21.15.59:6818/126911389 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa980040a0 con 0x7ffa880bdbf0 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.288+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 --> [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffaa0064060 con 0x7ffa880bdbf0 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.288+0000 7ffa87fff700 1 -- 172.21.15.59:0/2422664290 <== osd.6 v2:172.21.15.59:6818/126911389 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffaa0064060 con 0x7ffa880bdbf0 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.288+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] conn(0x7ffa880bdbf0 msgr2=0x7ffa880c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.288+0000 7ffaa5159700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] conn(0x7ffa880bdbf0 0x7ffa880c0090 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.289+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ffa880b2990 msgr2=0x7ffa880b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.289+0000 7ffaa5159700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ffa880b2990 0x7ffa880b4e50 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ffa90005e10 tx=0x7ffa90005d40 comp rx=0 tx=0).stop 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.289+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0108410 msgr2=0x7ffaa01998c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.289+0000 7ffaa5159700 1 --2- 172.21.15.59:0/2422664290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffaa0108410 0x7ffaa01998c0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7ffa9800b830 tx=0x7ffa98004ce0 comp rx=0 tx=0).stop 2024-05-08T05:27:33.701 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.289+0000 7ffa9f59e700 1 -- 172.21.15.59:0/2422664290 reap_dead start 2024-05-08T05:27:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.289+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 shutdown_connections 2024-05-08T05:27:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.289+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 >> 172.21.15.59:0/2422664290 conn(0x7ffaa0079350 msgr2=0x7ffaa01015d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.289+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 shutdown_connections 2024-05-08T05:27:33.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:32.289+0000 7ffaa5159700 1 -- 172.21.15.59:0/2422664290 wait complete. 2024-05-08T05:27:34.349 INFO:teuthology.orchestra.run.smithi059.stdout:154618822728 2024-05-08T05:27:34.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-05-08T05:27:34.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.533+0000 7f6ce114d700 1 -- 172.21.15.59:0/646305080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cdc10ff50 msgr2=0x7f6cdc112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:34.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.533+0000 7f6ce114d700 1 --2- 172.21.15.59:0/646305080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cdc10ff50 0x7f6cdc112340 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f6cd400b3e0 tx=0x7f6cd400b6f0 comp rx=0 tx=0).stop 2024-05-08T05:27:34.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.533+0000 7f6ce114d700 1 -- 172.21.15.59:0/646305080 shutdown_connections 2024-05-08T05:27:34.535 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.533+0000 7f6ce114d700 1 --2- 172.21.15.59:0/646305080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cdc10ff50 0x7f6cdc112340 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:34.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.533+0000 7f6ce114d700 1 --2- 172.21.15.59:0/646305080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cdc06b1b0 0x7f6cdc06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:34.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.533+0000 7f6ce114d700 1 --2- 172.21.15.59:0/646305080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cdc06bad0 0x7f6cdc10fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:34.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.533+0000 7f6ce114d700 1 -- 172.21.15.59:0/646305080 >> 172.21.15.59:0/646305080 conn(0x7f6cdc079350 msgr2=0x7f6cdc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:34.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.533+0000 7f6ce114d700 1 -- 172.21.15.59:0/646305080 shutdown_connections 2024-05-08T05:27:34.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.533+0000 7f6ce114d700 1 -- 172.21.15.59:0/646305080 wait complete. 2024-05-08T05:27:34.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.534+0000 7f6ce114d700 1 Processor -- start 2024-05-08T05:27:34.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.535+0000 7f6ce114d700 1 -- start start 2024-05-08T05:27:34.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.535+0000 7f6ce114d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cdc06b1b0 0x7f6cdc10f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:34.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.535+0000 7f6ce114d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cdc06bad0 0x7f6cdc10d850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:34.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.535+0000 7f6cd259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cdc06bad0 0x7f6cdc10d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:34.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.535+0000 7f6cd259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cdc06bad0 0x7f6cdc10d850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34818/0 (socket says 172.21.15.59:34818) 2024-05-08T05:27:34.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.536+0000 7f6ce114d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cdc10ff50 0x7f6cdc10dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:34.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.536+0000 7f6cd259c700 1 -- 172.21.15.59:0/926983038 learned_addr learned my addr 172.21.15.59:0/926983038 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:34.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.536+0000 7f6cdad9d700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cdc06b1b0 0x7f6cdc10f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:34.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.536+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc104cd0 con 0x7f6cdc06bad0 2024-05-08T05:27:34.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.536+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc104b50 con 0x7f6cdc06b1b0 2024-05-08T05:27:34.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.536+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6cdc104e50 con 0x7f6cdc10ff50 2024-05-08T05:27:34.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cdb59e700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cdc10ff50 0x7f6cdc10dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:34.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cd259c700 1 -- 172.21.15.59:0/926983038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cdc10ff50 msgr2=0x7f6cdc10dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:34.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cd259c700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6cdc10ff50 0x7f6cdc10dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:34.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cd259c700 1 -- 172.21.15.59:0/926983038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cdc06b1b0 msgr2=0x7f6cdc10f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:34.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cd259c700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6cdc06b1b0 0x7f6cdc10f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:34.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cd259c700 1 -- 172.21.15.59:0/926983038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6cdc10e5f0 con 0x7f6cdc06bad0 2024-05-08T05:27:34.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cd259c700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cdc06bad0 0x7f6cdc10d850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400e9a0 tx=0x7f6cc400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:34.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cd8d99700 1 -- 172.21.15.59:0/926983038 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc400cbd0 con 0x7f6cdc06bad0 2024-05-08T05:27:34.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6cdc1ac7f0 con 0x7f6cdc06bad0 2024-05-08T05:27:34.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cd8d99700 1 -- 172.21.15.59:0/926983038 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cc400cd70 con 0x7f6cdc06bad0 2024-05-08T05:27:34.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6cd8d99700 1 -- 172.21.15.59:0/926983038 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6cc4018680 con 0x7f6cdc06bad0 2024-05-08T05:27:34.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.537+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6cdc1acd50 con 0x7f6cdc06bad0 2024-05-08T05:27:34.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.538+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6cdc063b30 con 0x7f6cdc06bad0 2024-05-08T05:27:34.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.541+0000 7f6cd8d99700 1 -- 172.21.15.59:0/926983038 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f6cc4010970 con 0x7f6cdc06bad0 2024-05-08T05:27:34.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.542+0000 7f6cd8d99700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6cbc0b2910 0x7f6cbc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:34.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.542+0000 7f6cdad9d700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6cbc0b2910 0x7f6cbc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:34.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.543+0000 7f6cd8d99700 1 -- 172.21.15.59:0/926983038 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6cc40e3240 con 0x7f6cdc06bad0 2024-05-08T05:27:34.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.543+0000 7f6cdad9d700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6cbc0b2910 0x7f6cbc0b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009680 tx=0x7f6ccc009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:34.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.543+0000 7f6cd8d99700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] conn(0x7f6cbc0bdb70 0x7f6cbc0c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:34.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.543+0000 7f6cd8d99700 1 -- 172.21.15.59:0/926983038 --> [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6cc401de50 con 0x7f6cbc0bdb70 2024-05-08T05:27:34.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.543+0000 7f6cd8d99700 1 -- 172.21.15.59:0/926983038 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f6cc40e3690 con 0x7f6cdc06bad0 2024-05-08T05:27:34.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.543+0000 7f6cdb59e700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] conn(0x7f6cbc0bdb70 0x7f6cbc0c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:34.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.544+0000 7f6cdb59e700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] conn(0x7f6cbc0bdb70 0x7f6cbc0c0010 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:34.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.545+0000 7f6cd8d99700 1 -- 172.21.15.59:0/926983038 <== osd.2 v2:172.21.15.143:6800/815254096 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6cc401de50 con 0x7f6cbc0bdb70 2024-05-08T05:27:34.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.562+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 --> [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6cdc063d80 con 0x7f6cbc0bdb70 2024-05-08T05:27:34.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.563+0000 7f6cd8d99700 1 -- 172.21.15.59:0/926983038 <== osd.2 v2:172.21.15.143:6800/815254096 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6cdc063d80 con 0x7f6cbc0bdb70 2024-05-08T05:27:34.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 >> [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] conn(0x7f6cbc0bdb70 msgr2=0x7f6cbc0c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:34.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] conn(0x7f6cbc0bdb70 0x7f6cbc0c0010 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:34.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6cbc0b2910 msgr2=0x7f6cbc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:34.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6cbc0b2910 0x7f6cbc0b4dd0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f6ccc009680 tx=0x7f6ccc009260 comp rx=0 tx=0).stop 2024-05-08T05:27:34.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cdc06bad0 msgr2=0x7f6cdc10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:34.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 --2- 172.21.15.59:0/926983038 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6cdc06bad0 0x7f6cdc10d850 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f6cc400e9a0 tx=0x7f6cc400ed60 comp rx=0 tx=0).stop 2024-05-08T05:27:34.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6cdb59e700 1 -- 172.21.15.59:0/926983038 reap_dead start 2024-05-08T05:27:34.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 shutdown_connections 2024-05-08T05:27:34.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 >> 172.21.15.59:0/926983038 conn(0x7f6cdc079350 msgr2=0x7f6cdc111460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:34.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 shutdown_connections 2024-05-08T05:27:34.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:34.564+0000 7f6ce114d700 1 -- 172.21.15.59:0/926983038 wait complete. 2024-05-08T05:27:34.993 INFO:teuthology.orchestra.run.smithi059.stdout:257698037812 2024-05-08T05:27:34.994 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-05-08T05:27:35.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:35 smithi059 ceph-mon[30304]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:35.403 INFO:teuthology.orchestra.run.smithi059.stdout:279172874285 2024-05-08T05:27:35.404 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-05-08T05:27:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:35 smithi143 ceph-mon[34337]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:35.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:35 smithi096 ceph-mon[34253]: pgmap v365: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:36.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.039+0000 7f7667767700 1 -- 172.21.15.59:0/3229863166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f766006bb40 msgr2=0x7f7660110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.039+0000 7f7667767700 1 --2- 172.21.15.59:0/3229863166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f766006bb40 0x7f7660110330 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f765800b0a0 tx=0x7f765800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.039+0000 7f7667767700 1 -- 172.21.15.59:0/3229863166 shutdown_connections 2024-05-08T05:27:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.039+0000 7f7667767700 1 --2- 172.21.15.59:0/3229863166 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f766006bb40 0x7f7660110330 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.039+0000 7f7667767700 1 --2- 172.21.15.59:0/3229863166 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7660109510 0x7f76601098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.039+0000 7f7667767700 1 --2- 172.21.15.59:0/3229863166 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f766006b180 0x7f766006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.039+0000 7f7667767700 1 -- 172.21.15.59:0/3229863166 >> 172.21.15.59:0/3229863166 conn(0x7f7660079350 msgr2=0x7f7660079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.039+0000 7f7667767700 1 -- 172.21.15.59:0/3229863166 shutdown_connections 2024-05-08T05:27:36.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.039+0000 7f7667767700 1 -- 172.21.15.59:0/3229863166 wait complete. 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7667767700 1 Processor -- start 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7667767700 1 -- start start 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7667767700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766006b180 0x7f7660102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7667767700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f766006bb40 0x7f76601029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7667767700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660109510 0x7f7660102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7667767700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7660113690 con 0x7f766006bb40 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7667767700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7660113510 con 0x7f766006b180 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7667767700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7660113810 con 0x7f7660109510 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7664d02700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f766006bb40 0x7f76601029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7664d02700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f766006bb40 0x7f76601029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34828/0 (socket says 172.21.15.59:34828) 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7664d02700 1 -- 172.21.15.59:0/4084648837 learned_addr learned my addr 172.21.15.59:0/4084648837 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7665503700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766006b180 0x7f7660102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7665d04700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660109510 0x7f7660102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7664d02700 1 -- 172.21.15.59:0/4084648837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660109510 msgr2=0x7f7660102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7664d02700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7660109510 0x7f7660102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7664d02700 1 -- 172.21.15.59:0/4084648837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766006b180 msgr2=0x7f7660102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7664d02700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f766006b180 0x7f7660102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.040+0000 7f7664d02700 1 -- 172.21.15.59:0/4084648837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76601a8090 con 0x7f766006bb40 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.041+0000 7f7664d02700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f766006bb40 0x7f76601029b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f765000d980 tx=0x7f765000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.041+0000 7f76567fc700 1 -- 172.21.15.59:0/4084648837 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7650017070 con 0x7f766006bb40 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.041+0000 7f76567fc700 1 -- 172.21.15.59:0/4084648837 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7650005020 con 0x7f766006bb40 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.041+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76601a8380 con 0x7f766006bb40 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.041+0000 7f76567fc700 1 -- 172.21.15.59:0/4084648837 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f765000fa10 con 0x7f766006bb40 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.041+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76601a8870 con 0x7f766006bb40 2024-05-08T05:27:36.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.042+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f7660063bb0 con 0x7f766006bb40 2024-05-08T05:27:36.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.043+0000 7f76567fc700 1 -- 172.21.15.59:0/4084648837 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f7650004a30 con 0x7f766006bb40 2024-05-08T05:27:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.044+0000 7f76567fc700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f764c0b2910 0x7f764c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.044+0000 7f7665503700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f764c0b2910 0x7f764c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.044+0000 7f76567fc700 1 -- 172.21.15.59:0/4084648837 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f76500e2b50 con 0x7f766006bb40 2024-05-08T05:27:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.044+0000 7f76567fc700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] conn(0x7f764c0bdb70 0x7f764c0c0010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.044+0000 7f76567fc700 1 -- 172.21.15.59:0/4084648837 --> [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f764c0c06e0 con 0x7f764c0bdb70 2024-05-08T05:27:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.044+0000 7f76567fc700 1 -- 172.21.15.59:0/4084648837 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f76500e2fa0 con 0x7f766006bb40 2024-05-08T05:27:36.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.044+0000 7f7665503700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f764c0b2910 0x7f764c0b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f765c005950 tx=0x7f765c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.045+0000 7f7665d04700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] conn(0x7f764c0bdb70 0x7f764c0c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.045+0000 7f7665d04700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] conn(0x7f764c0bdb70 0x7f764c0c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:36.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.046+0000 7f76567fc700 1 -- 172.21.15.59:0/4084648837 <== osd.1 v2:172.21.15.96:6800/2776039768 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f764c0c06e0 con 0x7f764c0bdb70 2024-05-08T05:27:36.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.063+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 --> [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f7660064030 con 0x7f764c0bdb70 2024-05-08T05:27:36.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.063+0000 7f76567fc700 1 -- 172.21.15.59:0/4084648837 <== osd.1 v2:172.21.15.96:6800/2776039768 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f7660064030 con 0x7f764c0bdb70 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 >> [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] conn(0x7f764c0bdb70 msgr2=0x7f764c0c0010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] conn(0x7f764c0bdb70 0x7f764c0c0010 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f764c0b2910 msgr2=0x7f764c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f764c0b2910 0x7f764c0b4dd0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f765c005950 tx=0x7f765c00b410 comp rx=0 tx=0).stop 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f766006bb40 msgr2=0x7f76601029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 --2- 172.21.15.59:0/4084648837 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f766006bb40 0x7f76601029b0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f765000d980 tx=0x7f765000dc90 comp rx=0 tx=0).stop 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7665d04700 1 -- 172.21.15.59:0/4084648837 reap_dead start 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 shutdown_connections 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 >> 172.21.15.59:0/4084648837 conn(0x7f7660079350 msgr2=0x7f7660101040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 shutdown_connections 2024-05-08T05:27:36.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.064+0000 7f7667767700 1 -- 172.21.15.59:0/4084648837 wait complete. 2024-05-08T05:27:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.246+0000 7f39b638b700 1 -- 172.21.15.59:0/3279601021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b0108850 msgr2=0x7f39b01103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.246+0000 7f39b638b700 1 --2- 172.21.15.59:0/3279601021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b0108850 0x7f39b01103d0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f39a800b0d0 tx=0x7f39a800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.246+0000 7f39b638b700 1 -- 172.21.15.59:0/3279601021 shutdown_connections 2024-05-08T05:27:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.246+0000 7f39b638b700 1 --2- 172.21.15.59:0/3279601021 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b0108850 0x7f39b01103d0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.246+0000 7f39b638b700 1 --2- 172.21.15.59:0/3279601021 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39b0103750 0x7f39b0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.246+0000 7f39b638b700 1 --2- 172.21.15.59:0/3279601021 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39b0104100 0x7f39b0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.246+0000 7f39b638b700 1 -- 172.21.15.59:0/3279601021 >> 172.21.15.59:0/3279601021 conn(0x7f39b007a600 msgr2=0x7f39b007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.247+0000 7f39b638b700 1 -- 172.21.15.59:0/3279601021 shutdown_connections 2024-05-08T05:27:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.247+0000 7f39b638b700 1 -- 172.21.15.59:0/3279601021 wait complete. 2024-05-08T05:27:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.247+0000 7f39b638b700 1 Processor -- start 2024-05-08T05:27:36.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.247+0000 7f39b638b700 1 -- start start 2024-05-08T05:27:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b638b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39b0103750 0x7f39b019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b638b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39b0104100 0x7f39b019fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b638b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b0108850 0x7f39b0199880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b638b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39b0113620 con 0x7f39b0108850 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b638b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01134a0 con 0x7f39b0103750 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b638b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f39b01137a0 con 0x7f39b0104100 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b4928700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b0108850 0x7f39b0199880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b4928700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b0108850 0x7f39b0199880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34850/0 (socket says 172.21.15.59:34850) 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b4928700 1 -- 172.21.15.59:0/212648898 learned_addr learned my addr 172.21.15.59:0/212648898 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39af7fe700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39b0104100 0x7f39b019fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b4928700 1 -- 172.21.15.59:0/212648898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39b0104100 msgr2=0x7f39b019fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b4928700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39b0104100 0x7f39b019fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b4928700 1 -- 172.21.15.59:0/212648898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39b0103750 msgr2=0x7f39b019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39affff700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39b0103750 0x7f39b019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b4928700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39b0103750 0x7f39b019f6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b4928700 1 -- 172.21.15.59:0/212648898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b0199fe0 con 0x7f39b0108850 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39af7fe700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39b0104100 0x7f39b019fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.248+0000 7f39b4928700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b0108850 0x7f39b0199880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f39a80095c0 tx=0x7f39a8007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:36.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.249+0000 7f39ad7fa700 1 -- 172.21.15.59:0/212648898 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a801f070 con 0x7f39b0108850 2024-05-08T05:27:36.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.249+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b019a270 con 0x7f39b0108850 2024-05-08T05:27:36.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.249+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b01a6590 con 0x7f39b0108850 2024-05-08T05:27:36.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.249+0000 7f39ad7fa700 1 -- 172.21.15.59:0/212648898 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a8003c00 con 0x7f39b0108850 2024-05-08T05:27:36.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.249+0000 7f39ad7fa700 1 -- 172.21.15.59:0/212648898 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a801b6a0 con 0x7f39b0108850 2024-05-08T05:27:36.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.250+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f399c001030 con 0x7f39b0108850 2024-05-08T05:27:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.251+0000 7f39ad7fa700 1 -- 172.21.15.59:0/212648898 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f39a8022070 con 0x7f39b0108850 2024-05-08T05:27:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.252+0000 7f39ad7fa700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f39980b2990 0x7f39980b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.252+0000 7f39affff700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f39980b2990 0x7f39980b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.252+0000 7f39ad7fa700 1 -- 172.21.15.59:0/212648898 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f39a800f020 con 0x7f39b0108850 2024-05-08T05:27:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.252+0000 7f39ad7fa700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] conn(0x7f39980bdbf0 0x7f39980c0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.252+0000 7f39ad7fa700 1 -- 172.21.15.59:0/212648898 --> [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f39a8021ec0 con 0x7f39980bdbf0 2024-05-08T05:27:36.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.252+0000 7f39ad7fa700 1 -- 172.21.15.59:0/212648898 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7f39a80e4190 con 0x7f39b0108850 2024-05-08T05:27:36.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.253+0000 7f39af7fe700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] conn(0x7f39980bdbf0 0x7f39980c0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:36.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.253+0000 7f39affff700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f39980b2990 0x7f39980b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f39a0005950 tx=0x7f39a00058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:36.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.253+0000 7f39af7fe700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] conn(0x7f39980bdbf0 0x7f39980c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:36.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.254+0000 7f39ad7fa700 1 -- 172.21.15.59:0/212648898 <== osd.3 v2:172.21.15.59:6810/3135413737 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f39a8021ec0 con 0x7f39980bdbf0 2024-05-08T05:27:36.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.272+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 --> [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f399c002e60 con 0x7f39980bdbf0 2024-05-08T05:27:36.273 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.272+0000 7f39ad7fa700 1 -- 172.21.15.59:0/212648898 <== osd.3 v2:172.21.15.59:6810/3135413737 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f399c002e60 con 0x7f39980bdbf0 2024-05-08T05:27:36.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.272+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] conn(0x7f39980bdbf0 msgr2=0x7f39980c0090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.274 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.272+0000 7f39b638b700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] conn(0x7f39980bdbf0 0x7f39980c0090 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.273+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f39980b2990 msgr2=0x7f39980b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.273+0000 7f39b638b700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f39980b2990 0x7f39980b4e50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f39a0005950 tx=0x7f39a00058e0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.273+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b0108850 msgr2=0x7f39b0199880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.273+0000 7f39b638b700 1 --2- 172.21.15.59:0/212648898 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39b0108850 0x7f39b0199880 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f39a80095c0 tx=0x7f39a8007ad0 comp rx=0 tx=0).stop 2024-05-08T05:27:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.273+0000 7f39b4928700 1 -- 172.21.15.59:0/212648898 reap_dead start 2024-05-08T05:27:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.273+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 shutdown_connections 2024-05-08T05:27:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.273+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 >> 172.21.15.59:0/212648898 conn(0x7f39b007a600 msgr2=0x7f39b0101760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.273+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 shutdown_connections 2024-05-08T05:27:36.275 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:36.273+0000 7f39b638b700 1 -- 172.21.15.59:0/212648898 wait complete. 2024-05-08T05:27:37.233 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:37 smithi059 ceph-mon[30304]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:37.263 INFO:teuthology.orchestra.run.smithi059.stdout:180388626500 2024-05-08T05:27:37.264 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-05-08T05:27:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:37 smithi143 ceph-mon[34337]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:37 smithi096 ceph-mon[34253]: pgmap v366: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:37.702 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:37.731 INFO:teuthology.orchestra.run.smithi059.stdout:55834574941 2024-05-08T05:27:37.731 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-05-08T05:27:38.900 INFO:teuthology.orchestra.run.smithi059.stdout:107374182483 2024-05-08T05:27:38.900 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-05-08T05:27:39.174 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:39 smithi059 ceph-mon[30304]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:39.228 INFO:teuthology.orchestra.run.smithi059.stdout:77309411416 2024-05-08T05:27:39.228 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-05-08T05:27:39.232 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:39 smithi143 ceph-mon[34337]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:39 smithi096 ceph-mon[34253]: pgmap v367: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:40.821 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.820+0000 7f82f6e06700 1 -- 172.21.15.59:0/330350633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 msgr2=0x7f82f0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.820+0000 7f82f6e06700 1 --2- 172.21.15.59:0/330350633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 0x7f82f0110330 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f82e800b0a0 tx=0x7f82e800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.820+0000 7f82f6e06700 1 -- 172.21.15.59:0/330350633 shutdown_connections 2024-05-08T05:27:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.820+0000 7f82f6e06700 1 --2- 172.21.15.59:0/330350633 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 0x7f82f0110330 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.820+0000 7f82f6e06700 1 --2- 172.21.15.59:0/330350633 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f0109510 0x7f82f01098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.820+0000 7f82f6e06700 1 --2- 172.21.15.59:0/330350633 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82f006bbd0 0x7f82f006c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.820+0000 7f82f6e06700 1 -- 172.21.15.59:0/330350633 >> 172.21.15.59:0/330350633 conn(0x7f82f0079350 msgr2=0x7f82f0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.820+0000 7f82f6e06700 1 -- 172.21.15.59:0/330350633 shutdown_connections 2024-05-08T05:27:40.822 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.820+0000 7f82f6e06700 1 -- 172.21.15.59:0/330350633 wait complete. 2024-05-08T05:27:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f6e06700 1 Processor -- start 2024-05-08T05:27:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f6e06700 1 -- start start 2024-05-08T05:27:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f6e06700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82f006bbd0 0x7f82f019f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f6e06700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 0x7f82f019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f6e06700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f0109510 0x7f82f0199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f6e06700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f82f0113640 con 0x7f82f006c590 2024-05-08T05:27:40.824 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f6e06700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01134c0 con 0x7f82f0109510 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f6e06700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f82f01137c0 con 0x7f82f006bbd0 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 0x7f82f019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 0x7f82f019fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50888/0 (socket says 172.21.15.59:50888) 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82effff700 1 -- 172.21.15.59:0/4191171356 learned_addr learned my addr 172.21.15.59:0/4191171356 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f4ba2700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82f006bbd0 0x7f82f019f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.821+0000 7f82f53a3700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f0109510 0x7f82f0199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82effff700 1 -- 172.21.15.59:0/4191171356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82f006bbd0 msgr2=0x7f82f019f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82effff700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82f006bbd0 0x7f82f019f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82effff700 1 -- 172.21.15.59:0/4191171356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f0109510 msgr2=0x7f82f0199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82effff700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f0109510 0x7f82f0199900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82effff700 1 -- 172.21.15.59:0/4191171356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82f019a060 con 0x7f82f006c590 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82effff700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 0x7f82f019fc80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00d980 tx=0x7f82dc00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82edffb700 1 -- 172.21.15.59:0/4191171356 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc017070 con 0x7f82f006c590 2024-05-08T05:27:40.825 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82f019a350 con 0x7f82f006c590 2024-05-08T05:27:40.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82edffb700 1 -- 172.21.15.59:0/4191171356 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82dc005020 con 0x7f82f006c590 2024-05-08T05:27:40.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82edffb700 1 -- 172.21.15.59:0/4191171356 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82dc00fa10 con 0x7f82f006c590 2024-05-08T05:27:40.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.822+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82f01a6610 con 0x7f82f006c590 2024-05-08T05:27:40.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.823+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82f006a190 con 0x7f82f006c590 2024-05-08T05:27:40.826 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.824+0000 7f82edffb700 1 -- 172.21.15.59:0/4191171356 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f82dc004a30 con 0x7f82f006c590 2024-05-08T05:27:40.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.825+0000 7f82edffb700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82e00b2910 0x7f82e00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:40.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.825+0000 7f82f4ba2700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82e00b2910 0x7f82e00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:40.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.825+0000 7f82edffb700 1 -- 172.21.15.59:0/4191171356 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f82dc0e2650 con 0x7f82f006c590 2024-05-08T05:27:40.827 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.826+0000 7f82f4ba2700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82e00b2910 0x7f82e00b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82f019aa40 tx=0x7f82e400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:40.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:40.830+0000 7f82edffb700 1 -- 172.21.15.59:0/4191171356 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82dc020020 con 0x7f82f006c590 2024-05-08T05:27:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.061+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f82f002e510 con 0x7f82f006c590 2024-05-08T05:27:41.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.061+0000 7f82edffb700 1 -- 172.21.15.59:0/4191171356 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f82dc0a9e20 con 0x7f82f006c590 2024-05-08T05:27:41.063 INFO:teuthology.orchestra.run.smithi059.stdout:133143986255 2024-05-08T05:27:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82e00b2910 msgr2=0x7f82e00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82e00b2910 0x7f82e00b4dd0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f82f019aa40 tx=0x7f82e400b410 comp rx=0 tx=0).stop 2024-05-08T05:27:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 msgr2=0x7f82f019fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:41.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 0x7f82f019fc80 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f82dc00d980 tx=0x7f82dc00dc90 comp rx=0 tx=0).stop 2024-05-08T05:27:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 shutdown_connections 2024-05-08T05:27:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82e00b2910 0x7f82e00b4dd0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f82f0109510 0x7f82f0199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f82f006bbd0 0x7f82f019f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 --2- 172.21.15.59:0/4191171356 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f82f006c590 0x7f82f019fc80 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 >> 172.21.15.59:0/4191171356 conn(0x7f82f0079350 msgr2=0x7f82f0100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 shutdown_connections 2024-05-08T05:27:41.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:41.064+0000 7f82f6e06700 1 -- 172.21.15.59:0/4191171356 wait complete. 2024-05-08T05:27:41.282 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:41 smithi059 ceph-mon[30304]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:41.283 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:41 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/4191171356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T05:27:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:41 smithi143 ceph-mon[34337]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:41 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/4191171356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T05:27:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:41 smithi096 ceph-mon[34253]: pgmap v368: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:41.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:41 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/4191171356' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-05-08T05:27:41.798 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:41.799 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:41.799 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:41.799 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:41.802 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:41.803 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:41.804 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:42.204 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:42.237 INFO:tasks.cephadm.ceph_manager.ceph:need seq 133143986252 got 133143986255 for osd.4 2024-05-08T05:27:42.238 DEBUG:teuthology.parallel:result is None 2024-05-08T05:27:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:43 smithi059 ceph-mon[30304]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:27:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:43 smithi143 ceph-mon[34337]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:27:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:43 smithi096 ceph-mon[34253]: pgmap v369: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:27:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.094+0000 7f0e4994c700 1 -- 172.21.15.59:0/4076221757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406bbf0 msgr2=0x7f0e4406c070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.094+0000 7f0e4994c700 1 --2- 172.21.15.59:0/4076221757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406bbf0 0x7f0e4406c070 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c009990 tx=0x7f0e2c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.095+0000 7f0e4994c700 1 -- 172.21.15.59:0/4076221757 shutdown_connections 2024-05-08T05:27:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.095+0000 7f0e4994c700 1 --2- 172.21.15.59:0/4076221757 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e4406c5b0 0x7f0e44110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.095+0000 7f0e4994c700 1 --2- 172.21.15.59:0/4076221757 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e44109510 0x7f0e441098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.095+0000 7f0e4994c700 1 --2- 172.21.15.59:0/4076221757 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406bbf0 0x7f0e4406c070 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.095+0000 7f0e4994c700 1 -- 172.21.15.59:0/4076221757 >> 172.21.15.59:0/4076221757 conn(0x7f0e44079370 msgr2=0x7f0e44079780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.095+0000 7f0e4994c700 1 -- 172.21.15.59:0/4076221757 shutdown_connections 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.095+0000 7f0e4994c700 1 -- 172.21.15.59:0/4076221757 wait complete. 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.095+0000 7f0e4994c700 1 Processor -- start 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.095+0000 7f0e4994c700 1 -- start start 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e4994c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4406bbf0 0x7f0e4419be10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e4994c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406c5b0 0x7f0e4419c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e4994c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e44109510 0x7f0e441a06e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e4994c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0e44113640 con 0x7f0e4406c5b0 2024-05-08T05:27:44.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e4994c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441134c0 con 0x7f0e4406bbf0 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e4994c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0e441137c0 con 0x7f0e44109510 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e3a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406c5b0 0x7f0e4419c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e3a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406c5b0 0x7f0e4419c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50906/0 (socket says 172.21.15.59:50906) 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e3a7fc700 1 -- 172.21.15.59:0/29996378 learned_addr learned my addr 172.21.15.59:0/29996378 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e42ffd700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4406bbf0 0x7f0e4419be10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.096+0000 7f0e437fe700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e44109510 0x7f0e441a06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e3a7fc700 1 -- 172.21.15.59:0/29996378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e44109510 msgr2=0x7f0e441a06e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e3a7fc700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e44109510 0x7f0e441a06e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e3a7fc700 1 -- 172.21.15.59:0/29996378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4406bbf0 msgr2=0x7f0e4419be10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e3a7fc700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4406bbf0 0x7f0e4419be10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e3a7fc700 1 -- 172.21.15.59:0/29996378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e441a0e20 con 0x7f0e4406c5b0 2024-05-08T05:27:44.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e3a7fc700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406c5b0 0x7f0e4419c350 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c000c00 tx=0x7f0e2c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e4894a700 1 -- 172.21.15.59:0/29996378 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c027070 con 0x7f0e4406c5b0 2024-05-08T05:27:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e4894a700 1 -- 172.21.15.59:0/29996378 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e2c003f20 con 0x7f0e4406c5b0 2024-05-08T05:27:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e4894a700 1 -- 172.21.15.59:0/29996378 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e2c022ca0 con 0x7f0e4406c5b0 2024-05-08T05:27:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e441a10b0 con 0x7f0e4406c5b0 2024-05-08T05:27:44.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.097+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e441a8990 con 0x7f0e4406c5b0 2024-05-08T05:27:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.098+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0e44104bd0 con 0x7f0e4406c5b0 2024-05-08T05:27:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.100+0000 7f0e4894a700 1 -- 172.21.15.59:0/29996378 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f0e2c02a070 con 0x7f0e4406c5b0 2024-05-08T05:27:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.101+0000 7f0e4894a700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e300b2990 0x7f0e300b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.101+0000 7f0e42ffd700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e300b2990 0x7f0e300b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.101+0000 7f0e4894a700 1 -- 172.21.15.59:0/29996378 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f0e2c0e3500 con 0x7f0e4406c5b0 2024-05-08T05:27:44.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.101+0000 7f0e42ffd700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e300b2990 0x7f0e300b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009550 tx=0x7f0e340091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:44.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.105+0000 7f0e4894a700 1 -- 172.21.15.59:0/29996378 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0e2c05fdf0 con 0x7f0e4406c5b0 2024-05-08T05:27:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.326+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f0e441a8b60 con 0x7f0e4406c5b0 2024-05-08T05:27:44.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.326+0000 7f0e4894a700 1 -- 172.21.15.59:0/29996378 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f0e2c013070 con 0x7f0e4406c5b0 2024-05-08T05:27:44.328 INFO:teuthology.orchestra.run.smithi059.stdout:304942678059 2024-05-08T05:27:44.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e300b2990 msgr2=0x7f0e300b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e300b2990 0x7f0e300b4e50 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f0e34009550 tx=0x7f0e340091b0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406c5b0 msgr2=0x7f0e4419c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406c5b0 0x7f0e4419c350 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0e2c000c00 tx=0x7f0e2c0047b0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 shutdown_connections 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0e44109510 0x7f0e441a06e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f0e300b2990 0x7f0e300b4e50 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0e4406bbf0 0x7f0e4419be10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 --2- 172.21.15.59:0/29996378 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0e4406c5b0 0x7f0e4419c350 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 >> 172.21.15.59:0/29996378 conn(0x7f0e44079370 msgr2=0x7f0e4410e660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 shutdown_connections 2024-05-08T05:27:44.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:44.329+0000 7f0e4994c700 1 -- 172.21.15.59:0/29996378 wait complete. 2024-05-08T05:27:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:45 smithi059 ceph-mon[30304]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-08T05:27:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:45 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/29996378' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T05:27:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:45 smithi143 ceph-mon[34337]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-08T05:27:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:45 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/29996378' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T05:27:45.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:45 smithi096 ceph-mon[34253]: pgmap v370: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-05-08T05:27:45.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:45 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/29996378' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-05-08T05:27:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:47 smithi059 ceph-mon[30304]: pgmap v371: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:47 smithi143 ceph-mon[34337]: pgmap v371: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:47 smithi096 ceph-mon[34253]: pgmap v371: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:47.699 INFO:tasks.cephadm.ceph_manager.ceph:need seq 304942678055 got 304942678059 for osd.11 2024-05-08T05:27:47.700 DEBUG:teuthology.parallel:result is None 2024-05-08T05:27:49.309 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:49 smithi059 ceph-mon[30304]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:49 smithi143 ceph-mon[34337]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:49 smithi096 ceph-mon[34253]: pgmap v372: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:50.133 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:50.134 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:27:51.398 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:51 smithi059 ceph-mon[30304]: pgmap v373: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:51.398 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:51 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2174703365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T05:27:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:51 smithi143 ceph-mon[34337]: pgmap v373: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:51 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2174703365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T05:27:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:51 smithi096 ceph-mon[34253]: pgmap v373: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:51 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2174703365' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-05-08T05:27:53.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 -- 172.21.15.59:0/2530637950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0103750 msgr2=0x7fa4f0103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2530637950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0103750 0x7fa4f0103b30 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00099c0 tx=0x7fa4e0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 -- 172.21.15.59:0/2530637950 shutdown_connections 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2530637950 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f0108850 0x7fa4f01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2530637950 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0103750 0x7fa4f0103b30 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2530637950 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f0104100 0x7fa4f0108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 -- 172.21.15.59:0/2530637950 >> 172.21.15.59:0/2530637950 conn(0x7fa4f007a600 msgr2=0x7fa4f007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 -- 172.21.15.59:0/2530637950 shutdown_connections 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 -- 172.21.15.59:0/2530637950 wait complete. 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 Processor -- start 2024-05-08T05:27:53.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.585+0000 7fa4f6822700 1 -- start start 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4f6822700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f0103750 0x7fa4f019f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4f6822700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0104100 0x7fa4f019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4f6822700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f0108850 0x7fa4f0199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4f6822700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f0113620 con 0x7fa4f0104100 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4f6822700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01134a0 con 0x7fa4f0108850 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4f6822700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa4f01137a0 con 0x7fa4f0103750 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4ef7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0104100 0x7fa4f019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4ef7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0104100 0x7fa4f019fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56070/0 (socket says 172.21.15.59:56070) 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4ef7fe700 1 -- 172.21.15.59:0/2174703365 learned_addr learned my addr 172.21.15.59:0/2174703365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4effff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f0103750 0x7fa4f019f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:53.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4f4dbf700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f0108850 0x7fa4f0199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:53.020 INFO:teuthology.orchestra.run.smithi059.stdout:180388626503 2024-05-08T05:27:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4ef7fe700 1 -- 172.21.15.59:0/2174703365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f0103750 msgr2=0x7fa4f019f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4ef7fe700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f0103750 0x7fa4f019f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4ef7fe700 1 -- 172.21.15.59:0/2174703365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f0108850 msgr2=0x7fa4f0199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4ef7fe700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f0108850 0x7fa4f0199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.586+0000 7fa4ef7fe700 1 -- 172.21.15.59:0/2174703365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4f019a0c0 con 0x7fa4f0104100 2024-05-08T05:27:53.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.587+0000 7fa4ef7fe700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0104100 0x7fa4f019fce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400c920 tx=0x7fa4e400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.587+0000 7fa4ed7fa700 1 -- 172.21.15.59:0/2174703365 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4004a30 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.587+0000 7fa4ed7fa700 1 -- 172.21.15.59:0/2174703365 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa4e40054d0 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.587+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa4f019a3b0 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.587+0000 7fa4ed7fa700 1 -- 172.21.15.59:0/2174703365 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4007c70 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.587+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4f01a6670 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.588+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa4f010ba90 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.589+0000 7fa4ed7fa700 1 -- 172.21.15.59:0/2174703365 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fa4e4004c60 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.590+0000 7fa4ed7fa700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa4d80b2910 0x7fa4d80b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.590+0000 7fa4effff700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa4d80b2910 0x7fa4d80b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.590+0000 7fa4ed7fa700 1 -- 172.21.15.59:0/2174703365 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa4e40e26b0 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.591+0000 7fa4effff700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa4d80b2910 0x7fa4d80b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa4f019aaa0 tx=0x7fa4e0005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.594+0000 7fa4ed7fa700 1 -- 172.21.15.59:0/2174703365 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa4e4014030 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.814+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7fa4f010b470 con 0x7fa4f0104100 2024-05-08T05:27:53.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.814+0000 7fa4ed7fa700 1 -- 172.21.15.59:0/2174703365 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa4e40a9e80 con 0x7fa4f0104100 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.816+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa4d80b2910 msgr2=0x7fa4d80b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.816+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa4d80b2910 0x7fa4d80b4dd0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa4f019aaa0 tx=0x7fa4e0005c40 comp rx=0 tx=0).stop 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.816+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0104100 msgr2=0x7fa4f019fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.816+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0104100 0x7fa4f019fce0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400c920 tx=0x7fa4e400cc30 comp rx=0 tx=0).stop 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.817+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 shutdown_connections 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.817+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa4d80b2910 0x7fa4d80b4dd0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.817+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4f0108850 0x7fa4f0199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.817+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa4f0103750 0x7fa4f019f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.817+0000 7fa4f6822700 1 --2- 172.21.15.59:0/2174703365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4f0104100 0x7fa4f019fce0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.817+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 >> 172.21.15.59:0/2174703365 conn(0x7fa4f007a600 msgr2=0x7fa4f0101830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.817+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 shutdown_connections 2024-05-08T05:27:53.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:50.817+0000 7fa4f6822700 1 -- 172.21.15.59:0/2174703365 wait complete. 2024-05-08T05:27:53.313 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:53 smithi059 ceph-mon[30304]: pgmap v374: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:27:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:53 smithi143 ceph-mon[34337]: pgmap v374: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:27:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:53 smithi096 ceph-mon[34253]: pgmap v374: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:27:53.624 INFO:tasks.cephadm.ceph_manager.ceph:need seq 180388626500 got 180388626503 for osd.6 2024-05-08T05:27:53.624 DEBUG:teuthology.parallel:result is None 2024-05-08T05:27:54.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.331+0000 7fadd8ecd700 1 -- 172.21.15.59:0/1505194065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd4102d60 msgr2=0x7fadd410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.331+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/1505194065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd4102d60 0x7fadd410f250 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fadcc00b0a0 tx=0x7fadcc00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.332+0000 7fadd8ecd700 1 -- 172.21.15.59:0/1505194065 shutdown_connections 2024-05-08T05:27:54.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.332+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/1505194065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd4102d60 0x7fadd410f250 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.332+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/1505194065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd4108410 0x7fadd41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.372 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.332+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/1505194065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadd41023a0 0x7fadd4102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.332+0000 7fadd8ecd700 1 -- 172.21.15.59:0/1505194065 >> 172.21.15.59:0/1505194065 conn(0x7fadd4079350 msgr2=0x7fadd4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.332+0000 7fadd8ecd700 1 -- 172.21.15.59:0/1505194065 shutdown_connections 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.332+0000 7fadd8ecd700 1 -- 172.21.15.59:0/1505194065 wait complete. 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.332+0000 7fadd8ecd700 1 Processor -- start 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.332+0000 7fadd8ecd700 1 -- start start 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd8ecd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd41023a0 0x7fadd419f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd8ecd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd4102d60 0x7fadd4197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd8ecd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadd4108410 0x7fadd4197f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd8ecd700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fadd4112560 con 0x7fadd41023a0 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd8ecd700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41123e0 con 0x7fadd4102d60 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd8ecd700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fadd41126e0 con 0x7fadd4108410 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd41023a0 0x7fadd419f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd41023a0 0x7fadd419f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56116/0 (socket says 172.21.15.59:56116) 2024-05-08T05:27:54.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd259c700 1 -- 172.21.15.59:0/47584067 learned_addr learned my addr 172.21.15.59:0/47584067 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd2d9d700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadd4108410 0x7fadd4197f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd1d9b700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd4102d60 0x7fadd4197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd259c700 1 -- 172.21.15.59:0/47584067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadd4108410 msgr2=0x7fadd4197f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd259c700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadd4108410 0x7fadd4197f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd259c700 1 -- 172.21.15.59:0/47584067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd4102d60 msgr2=0x7fadd4197a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd259c700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd4102d60 0x7fadd4197a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd259c700 1 -- 172.21.15.59:0/47584067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadd41984b0 con 0x7fadd41023a0 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.333+0000 7fadd259c700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd41023a0 0x7fadd419f800 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fadc400e9a0 tx=0x7fadc400ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.334+0000 7fadc37fe700 1 -- 172.21.15.59:0/47584067 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc400ca10 con 0x7fadd41023a0 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.334+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadd41987a0 con 0x7fadd41023a0 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.334+0000 7fadc37fe700 1 -- 172.21.15.59:0/47584067 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadc400cbb0 con 0x7fadd41023a0 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.334+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadd41b54a0 con 0x7fadd41023a0 2024-05-08T05:27:54.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.334+0000 7fadc37fe700 1 -- 172.21.15.59:0/47584067 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadc4018680 con 0x7fadd41023a0 2024-05-08T05:27:54.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.335+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadd4103ac0 con 0x7fadd41023a0 2024-05-08T05:27:54.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.337+0000 7fadc37fe700 1 -- 172.21.15.59:0/47584067 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fadc4010430 con 0x7fadd41023a0 2024-05-08T05:27:54.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.338+0000 7fadc37fe700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fadbc0b2910 0x7fadbc0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:54.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.338+0000 7fadd1d9b700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fadbc0b2910 0x7fadbc0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:54.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.338+0000 7fadc37fe700 1 -- 172.21.15.59:0/47584067 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fadc40e2b40 con 0x7fadd41023a0 2024-05-08T05:27:54.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.339+0000 7fadd1d9b700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fadbc0b2910 0x7fadbc0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fadd410f670 tx=0x7fadc8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:54.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.341+0000 7fadc37fe700 1 -- 172.21.15.59:0/47584067 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadc40aa370 con 0x7fadd41023a0 2024-05-08T05:27:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.565+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7fadd4198c90 con 0x7fadd41023a0 2024-05-08T05:27:54.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.566+0000 7fadc37fe700 1 -- 172.21.15.59:0/47584067 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fadc40b23f0 con 0x7fadd41023a0 2024-05-08T05:27:54.567 INFO:teuthology.orchestra.run.smithi059.stdout:154618822732 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fadbc0b2910 msgr2=0x7fadbc0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fadbc0b2910 0x7fadbc0b4dd0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fadd410f670 tx=0x7fadc8009260 comp rx=0 tx=0).stop 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd41023a0 msgr2=0x7fadd419f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd41023a0 0x7fadd419f800 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fadc400e9a0 tx=0x7fadc400ecb0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 shutdown_connections 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadd4108410 0x7fadd4197f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadd41023a0 0x7fadd419f800 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fadbc0b2910 0x7fadbc0b4dd0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 --2- 172.21.15.59:0/47584067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadd4102d60 0x7fadd4197a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 >> 172.21.15.59:0/47584067 conn(0x7fadd4079350 msgr2=0x7fadd406b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 shutdown_connections 2024-05-08T05:27:54.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.570+0000 7fadd8ecd700 1 -- 172.21.15.59:0/47584067 wait complete. 2024-05-08T05:27:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.596+0000 7f42580cb700 1 -- 172.21.15.59:0/1794659982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006c3d0 msgr2=0x7f4250110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.596+0000 7f42580cb700 1 --2- 172.21.15.59:0/1794659982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006c3d0 0x7f4250110320 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f424800b0a0 tx=0x7f424800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.596+0000 7f42580cb700 1 -- 172.21.15.59:0/1794659982 shutdown_connections 2024-05-08T05:27:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.596+0000 7f42580cb700 1 --2- 172.21.15.59:0/1794659982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006c3d0 0x7f4250110320 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.596+0000 7f42580cb700 1 --2- 172.21.15.59:0/1794659982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4250109500 0x7f42501098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.596+0000 7f42580cb700 1 --2- 172.21.15.59:0/1794659982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f425006ba10 0x7f425006be90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.596+0000 7f42580cb700 1 -- 172.21.15.59:0/1794659982 >> 172.21.15.59:0/1794659982 conn(0x7f4250079310 msgr2=0x7f4250079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.596+0000 7f42580cb700 1 -- 172.21.15.59:0/1794659982 shutdown_connections 2024-05-08T05:27:54.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.596+0000 7f42580cb700 1 -- 172.21.15.59:0/1794659982 wait complete. 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f42580cb700 1 Processor -- start 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f42580cb700 1 -- start start 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f42580cb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006ba10 0x7f425019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f42580cb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f425006c3d0 0x7f425019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f42580cb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4250109500 0x7f4250199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f42580cb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f4250113610 con 0x7f425006ba10 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f42580cb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4250113490 con 0x7f4250109500 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f42580cb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4250113790 con 0x7f425006c3d0 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f4255e67700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006ba10 0x7f425019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f4255e67700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006ba10 0x7f425019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56136/0 (socket says 172.21.15.59:56136) 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f4255e67700 1 -- 172.21.15.59:0/3474594170 learned_addr learned my addr 172.21.15.59:0/3474594170 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:54.599 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f4256668700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4250109500 0x7f4250199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.597+0000 7f4255666700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f425006c3d0 0x7f425019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4255e67700 1 -- 172.21.15.59:0/3474594170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f425006c3d0 msgr2=0x7f425019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4255e67700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f425006c3d0 0x7f425019fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4255e67700 1 -- 172.21.15.59:0/3474594170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4250109500 msgr2=0x7f4250199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4255e67700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4250109500 0x7f4250199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4255e67700 1 -- 172.21.15.59:0/3474594170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f425019a090 con 0x7f425006ba10 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4256668700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4250109500 0x7f4250199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4255e67700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006ba10 0x7f425019f720 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f424c00d950 tx=0x7f424c00dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4246ffd700 1 -- 172.21.15.59:0/3474594170 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c016070 con 0x7f425006ba10 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f425019a380 con 0x7f425006ba10 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4246ffd700 1 -- 172.21.15.59:0/3474594170 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f424c010460 con 0x7f425006ba10 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f4246ffd700 1 -- 172.21.15.59:0/3474594170 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f424c00f620 con 0x7f425006ba10 2024-05-08T05:27:54.600 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.598+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42501a65f0 con 0x7f425006ba10 2024-05-08T05:27:54.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.599+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f425006a190 con 0x7f425006ba10 2024-05-08T05:27:54.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.601+0000 7f4246ffd700 1 -- 172.21.15.59:0/3474594170 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f424c019070 con 0x7f425006ba10 2024-05-08T05:27:54.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.603+0000 7f4246ffd700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f423c0b2910 0x7f423c0b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:54.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.603+0000 7f4246ffd700 1 -- 172.21.15.59:0/3474594170 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f424c0e22f0 con 0x7f425006ba10 2024-05-08T05:27:54.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.603+0000 7f4255666700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f423c0b2910 0x7f423c0b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:54.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.604+0000 7f4255666700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f423c0b2910 0x7f423c0b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4250079980 tx=0x7f4240009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:54.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.607+0000 7f4246ffd700 1 -- 172.21.15.59:0/3474594170 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f424c0a9b20 con 0x7f425006ba10 2024-05-08T05:27:54.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.826+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f425019ae10 con 0x7f425006ba10 2024-05-08T05:27:54.830 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.827+0000 7f4246ffd700 1 -- 172.21.15.59:0/3474594170 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f424c0a9420 con 0x7f425006ba10 2024-05-08T05:27:54.830 INFO:teuthology.orchestra.run.smithi059.stdout:34359738469 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.829+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f423c0b2910 msgr2=0x7f423c0b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.829+0000 7f42580cb700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f423c0b2910 0x7f423c0b4dd0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f4250079980 tx=0x7f4240009260 comp rx=0 tx=0).stop 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.829+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006ba10 msgr2=0x7f425019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.829+0000 7f42580cb700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006ba10 0x7f425019f720 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f424c00d950 tx=0x7f424c00dd10 comp rx=0 tx=0).stop 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.830+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 shutdown_connections 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.830+0000 7f42580cb700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f423c0b2910 0x7f423c0b4dd0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.830+0000 7f42580cb700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4250109500 0x7f4250199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.830+0000 7f42580cb700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f425006ba10 0x7f425019f720 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.830+0000 7f42580cb700 1 --2- 172.21.15.59:0/3474594170 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f425006c3d0 0x7f425019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:54.831 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.830+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 >> 172.21.15.59:0/3474594170 conn(0x7f4250079310 msgr2=0x7f4250101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:54.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.830+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 shutdown_connections 2024-05-08T05:27:54.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.830+0000 7f42580cb700 1 -- 172.21.15.59:0/3474594170 wait complete. 2024-05-08T05:27:55.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:55 smithi059 ceph-mon[30304]: pgmap v375: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:55.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:55 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3373336676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T05:27:55.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:55 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/47584067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T05:27:55.416 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:55 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3474594170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T05:27:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:55 smithi143 ceph-mon[34337]: pgmap v375: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:55 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3373336676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T05:27:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:55 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/47584067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T05:27:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:55 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3474594170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T05:27:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:55 smithi096 ceph-mon[34253]: pgmap v375: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:27:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:55 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3373336676' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-05-08T05:27:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:55 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/47584067' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-05-08T05:27:55.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:55 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3474594170' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.860+0000 7fb7d94de700 1 -- 172.21.15.59:0/3618949430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d406c630 msgr2=0x7fb7d4110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.860+0000 7fb7d94de700 1 --2- 172.21.15.59:0/3618949430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d406c630 0x7fb7d4110360 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc0099c0 tx=0x7fb7cc009cd0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.860+0000 7fb7d94de700 1 -- 172.21.15.59:0/3618949430 shutdown_connections 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.860+0000 7fb7d94de700 1 --2- 172.21.15.59:0/3618949430 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d406c630 0x7fb7d4110360 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.860+0000 7fb7d94de700 1 --2- 172.21.15.59:0/3618949430 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d4109590 0x7fb7d4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.860+0000 7fb7d94de700 1 --2- 172.21.15.59:0/3618949430 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d406bc70 0x7fb7d406c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.860+0000 7fb7d94de700 1 -- 172.21.15.59:0/3618949430 >> 172.21.15.59:0/3618949430 conn(0x7fb7d4079440 msgr2=0x7fb7d4079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.861+0000 7fb7d94de700 1 -- 172.21.15.59:0/3618949430 shutdown_connections 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.861+0000 7fb7d94de700 1 -- 172.21.15.59:0/3618949430 wait complete. 2024-05-08T05:27:56.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.861+0000 7fb7d94de700 1 Processor -- start 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.861+0000 7fb7d94de700 1 -- start start 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d94de700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d406bc70 0x7fb7d419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d94de700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d406c630 0x7fb7d419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d94de700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d4109590 0x7fb7d4199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d94de700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4113680 con 0x7fb7d4109590 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d94de700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4113500 con 0x7fb7d406bc70 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d94de700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d4113800 con 0x7fb7d406c630 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d4109590 0x7fb7d4199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d4109590 0x7fb7d4199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56150/0 (socket says 172.21.15.59:56150) 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d37fe700 1 -- 172.21.15.59:0/352049701 learned_addr learned my addr 172.21.15.59:0/352049701 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7ca7fc700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d406c630 0x7fb7d419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d2ffd700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d406bc70 0x7fb7d419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d37fe700 1 -- 172.21.15.59:0/352049701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d406c630 msgr2=0x7fb7d419fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:56.266 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d37fe700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d406c630 0x7fb7d419fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d37fe700 1 -- 172.21.15.59:0/352049701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d406bc70 msgr2=0x7fb7d419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d37fe700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d406bc70 0x7fb7d419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.862+0000 7fb7d37fe700 1 -- 172.21.15.59:0/352049701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d419a060 con 0x7fb7d4109590 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.863+0000 7fb7d37fe700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d4109590 0x7fb7d4199900 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc005910 tx=0x7fb7cc009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.863+0000 7fb7d0ff9700 1 -- 172.21.15.59:0/352049701 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc027070 con 0x7fb7d4109590 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.863+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d419a2f0 con 0x7fb7d4109590 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.863+0000 7fb7d0ff9700 1 -- 172.21.15.59:0/352049701 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7cc003f20 con 0x7fb7d4109590 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.863+0000 7fb7d0ff9700 1 -- 172.21.15.59:0/352049701 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7cc022ba0 con 0x7fb7d4109590 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.863+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d41a6610 con 0x7fb7d4109590 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.865+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d4104c00 con 0x7fb7d4109590 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.867+0000 7fb7d0ff9700 1 -- 172.21.15.59:0/352049701 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb7cc02a070 con 0x7fb7d4109590 2024-05-08T05:27:56.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.868+0000 7fb7d0ff9700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7c00b2910 0x7fb7c00b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.868+0000 7fb7d2ffd700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7c00b2910 0x7fb7c00b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.868+0000 7fb7d0ff9700 1 -- 172.21.15.59:0/352049701 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb7cc0e3260 con 0x7fb7d4109590 2024-05-08T05:27:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.868+0000 7fb7d2ffd700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7c00b2910 0x7fb7c00b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4007fd0 tx=0x7fb7c400c040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.873+0000 7fb7d0ff9700 1 -- 172.21.15.59:0/352049701 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb7cc013070 con 0x7fb7d4109590 2024-05-08T05:27:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.095+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7fb7d402e510 con 0x7fb7d4109590 2024-05-08T05:27:56.268 INFO:teuthology.orchestra.run.smithi059.stdout:257698037817 2024-05-08T05:27:56.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.095+0000 7fb7d0ff9700 1 -- 172.21.15.59:0/352049701 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb7cc0aab20 con 0x7fb7d4109590 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7c00b2910 msgr2=0x7fb7c00b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7c00b2910 0x7fb7c00b4dd0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fb7c4007fd0 tx=0x7fb7c400c040 comp rx=0 tx=0).stop 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d4109590 msgr2=0x7fb7d4199900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d4109590 0x7fb7d4199900 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fb7cc005910 tx=0x7fb7cc009580 comp rx=0 tx=0).stop 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 shutdown_connections 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb7d4109590 0x7fb7d4199900 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb7c00b2910 0x7fb7c00b4dd0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb7d406bc70 0x7fb7d419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 --2- 172.21.15.59:0/352049701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb7d406c630 0x7fb7d419fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 >> 172.21.15.59:0/352049701 conn(0x7fb7d4079440 msgr2=0x7fb7d4100ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.097+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 shutdown_connections 2024-05-08T05:27:56.269 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:56.098+0000 7fb7d94de700 1 -- 172.21.15.59:0/352049701 wait complete. 2024-05-08T05:27:56.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:56 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3616501982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T05:27:56.472 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:56 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/352049701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T05:27:56.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:56 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3616501982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T05:27:56.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:56 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/352049701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T05:27:56.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:56 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3616501982' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-05-08T05:27:56.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:56 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/352049701' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-05-08T05:27:57.165 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738463 got 34359738469 for osd.0 2024-05-08T05:27:57.165 DEBUG:teuthology.parallel:result is None 2024-05-08T05:27:57.363 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:57 smithi059 ceph-mon[30304]: pgmap v376: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:57 smithi143 ceph-mon[34337]: pgmap v376: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:57.500 INFO:teuthology.orchestra.run.smithi059.stdout:231928234046 2024-05-08T05:27:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.217+0000 7f2fa485d700 1 -- 172.21.15.59:0/3172136453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0104100 msgr2=0x7f2fa0108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.217+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3172136453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0104100 0x7f2fa0108120 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2f940099c0 tx=0x7f2f94009cd0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.217+0000 7f2fa485d700 1 -- 172.21.15.59:0/3172136453 shutdown_connections 2024-05-08T05:27:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.217+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3172136453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa0108850 0x7f2fa01103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.217+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3172136453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa0103750 0x7f2fa0103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.217+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3172136453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0104100 0x7f2fa0108120 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.217+0000 7f2fa485d700 1 -- 172.21.15.59:0/3172136453 >> 172.21.15.59:0/3172136453 conn(0x7f2fa007a600 msgr2=0x7f2fa007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:57.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.217+0000 7f2fa485d700 1 -- 172.21.15.59:0/3172136453 shutdown_connections 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.217+0000 7f2fa485d700 1 -- 172.21.15.59:0/3172136453 wait complete. 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.218+0000 7f2fa485d700 1 Processor -- start 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.218+0000 7f2fa485d700 1 -- start start 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.218+0000 7f2fa485d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0103750 0x7f2fa019f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.218+0000 7f2fa485d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa0104100 0x7f2fa019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.218+0000 7f2fa485d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa0108850 0x7f2fa01998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.218+0000 7f2fa485d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0113630 con 0x7f2fa0103750 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.218+0000 7f2fa485d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa01134b0 con 0x7f2fa0108850 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.218+0000 7f2fa485d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa01137b0 con 0x7f2fa0104100 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0103750 0x7f2fa019f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9e59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0103750 0x7f2fa019f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56098/0 (socket says 172.21.15.59:56098) 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9e59c700 1 -- 172.21.15.59:0/3373336676 learned_addr learned my addr 172.21.15.59:0/3373336676 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9dd9b700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa0104100 0x7f2fa019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:57.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9ed9d700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa0108850 0x7f2fa01998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9e59c700 1 -- 172.21.15.59:0/3373336676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa0104100 msgr2=0x7f2fa019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9e59c700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa0104100 0x7f2fa019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9e59c700 1 -- 172.21.15.59:0/3373336676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa0108850 msgr2=0x7f2fa01998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9e59c700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa0108850 0x7f2fa01998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9e59c700 1 -- 172.21.15.59:0/3373336676 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa019a050 con 0x7f2fa0103750 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f9e59c700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0103750 0x7f2fa019f730 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000d980 tx=0x7f2f9000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f8f7fe700 1 -- 172.21.15.59:0/3373336676 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90016070 con 0x7f2fa0103750 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa019a340 con 0x7f2fa0103750 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.219+0000 7f2f8f7fe700 1 -- 172.21.15.59:0/3373336676 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f90004830 con 0x7f2fa0103750 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.220+0000 7f2f8f7fe700 1 -- 172.21.15.59:0/3373336676 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f90005540 con 0x7f2fa0103750 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.220+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa01a6600 con 0x7f2fa0103750 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.221+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa006a190 con 0x7f2fa0103750 2024-05-08T05:27:57.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.223+0000 7f2f8f7fe700 1 -- 172.21.15.59:0/3373336676 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f2f90004220 con 0x7f2fa0103750 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.224+0000 7f2f8f7fe700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f880b2910 0x7f2f880b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.225+0000 7f2f9dd9b700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f880b2910 0x7f2f880b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.225+0000 7f2f8f7fe700 1 -- 172.21.15.59:0/3373336676 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f2f900e25a0 con 0x7f2fa0103750 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.225+0000 7f2f9dd9b700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f880b2910 0x7f2f880b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009990 tx=0x7f2f94009650 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.229+0000 7f2f8f7fe700 1 -- 172.21.15.59:0/3373336676 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2f9001f020 con 0x7f2fa0103750 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.451+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7f2fa002e510 con 0x7f2fa0103750 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.451+0000 7f2f8f7fe700 1 -- 172.21.15.59:0/3373336676 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f2f900a9d70 con 0x7f2fa0103750 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f880b2910 msgr2=0x7f2f880b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f880b2910 0x7f2f880b4dd0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f2f94009990 tx=0x7f2f94009650 comp rx=0 tx=0).stop 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0103750 msgr2=0x7f2fa019f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0103750 0x7f2fa019f730 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000d980 tx=0x7f2f9000dc90 comp rx=0 tx=0).stop 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 shutdown_connections 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f2f880b2910 0x7f2f880b4dd0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2fa0108850 0x7f2fa01998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2fa0103750 0x7f2fa019f730 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 --2- 172.21.15.59:0/3373336676 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2fa0104100 0x7f2fa019fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:57.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 >> 172.21.15.59:0/3373336676 conn(0x7f2fa007a600 msgr2=0x7f2fa0101840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:57.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 shutdown_connections 2024-05-08T05:27:57.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:54.454+0000 7f2fa485d700 1 -- 172.21.15.59:0/3373336676 wait complete. 2024-05-08T05:27:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:57 smithi096 ceph-mon[34253]: pgmap v376: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:27:58.138 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234041 got 231928234046 for osd.8 2024-05-08T05:27:58.138 DEBUG:teuthology.parallel:result is None 2024-05-08T05:27:58.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.330+0000 7fb07ffa3700 1 -- 172.21.15.59:0/4042532002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 msgr2=0x7fb078110770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.330+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/4042532002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 0x7fb078110770 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb074009990 tx=0x7fb074009ca0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.330+0000 7fb07ffa3700 1 -- 172.21.15.59:0/4042532002 shutdown_connections 2024-05-08T05:27:58.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.330+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/4042532002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07810b6f0 0x7fb0781171d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.330+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/4042532002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 0x7fb078110770 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.330+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/4042532002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb07810ad30 0x7fb07810b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.330+0000 7fb07ffa3700 1 -- 172.21.15.59:0/4042532002 >> 172.21.15.59:0/4042532002 conn(0x7fb07806e9f0 msgr2=0x7fb07806ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:58.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.331+0000 7fb07ffa3700 1 -- 172.21.15.59:0/4042532002 shutdown_connections 2024-05-08T05:27:58.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.331+0000 7fb07ffa3700 1 -- 172.21.15.59:0/4042532002 wait complete. 2024-05-08T05:27:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07ffa3700 1 Processor -- start 2024-05-08T05:27:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07ffa3700 1 -- start start 2024-05-08T05:27:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07ffa3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07810ad30 0x7fb0781a0150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07ffa3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb07810b6f0 0x7fb0781a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07ffa3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 0x7fb0781a4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07ffa3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb078119cc0 con 0x7fb078110390 2024-05-08T05:27:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07ffa3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb078119b40 con 0x7fb07810ad30 2024-05-08T05:27:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07ffa3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb078119e40 con 0x7fb07810b6f0 2024-05-08T05:27:58.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07e540700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 0x7fb0781a4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:58.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07e540700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 0x7fb0781a4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56168/0 (socket says 172.21.15.59:56168) 2024-05-08T05:27:58.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.332+0000 7fb07e540700 1 -- 172.21.15.59:0/1739676793 learned_addr learned my addr 172.21.15.59:0/1739676793 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:58.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07d53e700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb07810b6f0 0x7fb0781a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:58.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07dd3f700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07810ad30 0x7fb0781a0150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:58.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07e540700 1 -- 172.21.15.59:0/1739676793 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb07810b6f0 msgr2=0x7fb0781a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07e540700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb07810b6f0 0x7fb0781a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07e540700 1 -- 172.21.15.59:0/1739676793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07810ad30 msgr2=0x7fb0781a0150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07e540700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07810ad30 0x7fb0781a0150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07e540700 1 -- 172.21.15.59:0/1739676793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0781a5160 con 0x7fb078110390 2024-05-08T05:27:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07e540700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 0x7fb0781a4a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb070007ee0 tx=0x7fb07000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb06effd700 1 -- 172.21.15.59:0/1739676793 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb07000dbd0 con 0x7fb078110390 2024-05-08T05:27:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0781a5450 con 0x7fb078110390 2024-05-08T05:27:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb06effd700 1 -- 172.21.15.59:0/1739676793 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb07000dd70 con 0x7fb078110390 2024-05-08T05:27:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb06effd700 1 -- 172.21.15.59:0/1739676793 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0700058d0 con 0x7fb078110390 2024-05-08T05:27:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.333+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0781accd0 con 0x7fb078110390 2024-05-08T05:27:58.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.335+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb07806a190 con 0x7fb078110390 2024-05-08T05:27:58.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.336+0000 7fb06effd700 1 -- 172.21.15.59:0/1739676793 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fb07001a020 con 0x7fb078110390 2024-05-08T05:27:58.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.337+0000 7fb06effd700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb0640b4b90 0x7fb0640b7050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:58.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.337+0000 7fb07dd3f700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb0640b4b90 0x7fb0640b7050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:58.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.337+0000 7fb06effd700 1 -- 172.21.15.59:0/1739676793 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb0700e1d90 con 0x7fb078110390 2024-05-08T05:27:58.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.337+0000 7fb07dd3f700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb0640b4b90 0x7fb0640b7050 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b5c0 tx=0x7fb074005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:58.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.341+0000 7fb06effd700 1 -- 172.21.15.59:0/1739676793 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb0700e5030 con 0x7fb078110390 2024-05-08T05:27:58.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.559+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb07802e510 con 0x7fb078110390 2024-05-08T05:27:58.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.560+0000 7fb06effd700 1 -- 172.21.15.59:0/1739676793 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb070010070 con 0x7fb078110390 2024-05-08T05:27:58.560 INFO:teuthology.orchestra.run.smithi059.stdout:201863462978 2024-05-08T05:27:58.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.562+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb0640b4b90 msgr2=0x7fb0640b7050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.562+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb0640b4b90 0x7fb0640b7050 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fb07400b5c0 tx=0x7fb074005fd0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.562+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 msgr2=0x7fb0781a4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.562+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 0x7fb0781a4a20 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb070007ee0 tx=0x7fb07000d3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.562+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 shutdown_connections 2024-05-08T05:27:58.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.562+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb078110390 0x7fb0781a4a20 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.562+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb0640b4b90 0x7fb0640b7050 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.562+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb07810ad30 0x7fb0781a0150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.562+0000 7fb07ffa3700 1 --2- 172.21.15.59:0/1739676793 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb07810b6f0 0x7fb0781a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.563+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 >> 172.21.15.59:0/1739676793 conn(0x7fb07806e9f0 msgr2=0x7fb07810e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:58.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.563+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 shutdown_connections 2024-05-08T05:27:58.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:58.563+0000 7fb07ffa3700 1 -- 172.21.15.59:0/1739676793 wait complete. 2024-05-08T05:27:58.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.524+0000 7fcf2c77a700 1 -- 172.21.15.59:0/2496020194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24102d50 msgr2=0x7fcf2410f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.524+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/2496020194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24102d50 0x7fcf2410f240 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b0a0 tx=0x7fcf1c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.524+0000 7fcf2c77a700 1 -- 172.21.15.59:0/2496020194 shutdown_connections 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.524+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/2496020194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24102d50 0x7fcf2410f240 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.524+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/2496020194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf24108400 0x7fcf241087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.524+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/2496020194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf24102390 0x7fcf24102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.524+0000 7fcf2c77a700 1 -- 172.21.15.59:0/2496020194 >> 172.21.15.59:0/2496020194 conn(0x7fcf24079310 msgr2=0x7fcf24079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.524+0000 7fcf2c77a700 1 -- 172.21.15.59:0/2496020194 shutdown_connections 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.524+0000 7fcf2c77a700 1 -- 172.21.15.59:0/2496020194 wait complete. 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.525+0000 7fcf2c77a700 1 Processor -- start 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.525+0000 7fcf2c77a700 1 -- start start 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.525+0000 7fcf2c77a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf24102390 0x7fcf241a1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.525+0000 7fcf2c77a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf24102d50 0x7fcf241a1f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.525+0000 7fcf2c77a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24108400 0x7fcf2419bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.525+0000 7fcf2c77a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcf24112540 con 0x7fcf24108400 2024-05-08T05:27:58.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.525+0000 7fcf2c77a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241123c0 con 0x7fcf24102d50 2024-05-08T05:27:58.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.525+0000 7fcf2c77a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcf241126c0 con 0x7fcf24102390 2024-05-08T05:27:58.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2ad17700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24108400 0x7fcf2419bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:58.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2ad17700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24108400 0x7fcf2419bb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56146/0 (socket says 172.21.15.59:56146) 2024-05-08T05:27:58.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2ad17700 1 -- 172.21.15.59:0/3616501982 learned_addr learned my addr 172.21.15.59:0/3616501982 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:27:58.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf29d15700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf24102d50 0x7fcf241a1f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:58.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2a516700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf24102390 0x7fcf241a1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:58.853 INFO:teuthology.orchestra.run.smithi059.stdout:107374182486 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2ad17700 1 -- 172.21.15.59:0/3616501982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf24102390 msgr2=0x7fcf241a1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2ad17700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf24102390 0x7fcf241a1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2ad17700 1 -- 172.21.15.59:0/3616501982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf24102d50 msgr2=0x7fcf241a1f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2ad17700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf24102d50 0x7fcf241a1f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2ad17700 1 -- 172.21.15.59:0/3616501982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2419c400 con 0x7fcf24108400 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2ad17700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24108400 0x7fcf2419bb20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b830 tx=0x7fcf1c004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf1b7fe700 1 -- 172.21.15.59:0/3616501982 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c021070 con 0x7fcf24108400 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf1b7fe700 1 -- 172.21.15.59:0/3616501982 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf1c007e30 con 0x7fcf24108400 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2419c690 con 0x7fcf24108400 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf1b7fe700 1 -- 172.21.15.59:0/3616501982 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf1c0047d0 con 0x7fcf24108400 2024-05-08T05:27:58.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.526+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf241a89d0 con 0x7fcf24108400 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.527+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2419ca00 con 0x7fcf24108400 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.529+0000 7fcf1b7fe700 1 -- 172.21.15.59:0/3616501982 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7fcf1c007980 con 0x7fcf24108400 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.530+0000 7fcf1b7fe700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcf100b2910 0x7fcf100b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.530+0000 7fcf2a516700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcf100b2910 0x7fcf100b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.530+0000 7fcf1b7fe700 1 -- 172.21.15.59:0/3616501982 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fcf1c00f020 con 0x7fcf24108400 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.530+0000 7fcf2a516700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcf100b2910 0x7fcf100b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf20005fd0 tx=0x7fcf20005d90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.533+0000 7fcf1b7fe700 1 -- 172.21.15.59:0/3616501982 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fcf2419ca00 con 0x7fcf24108400 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.760+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fcf241a8ba0 con 0x7fcf24108400 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.760+0000 7fcf1b7fe700 1 -- 172.21.15.59:0/3616501982 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fcf1c02a020 con 0x7fcf24108400 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcf100b2910 msgr2=0x7fcf100b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcf100b2910 0x7fcf100b4dd0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fcf20005fd0 tx=0x7fcf20005d90 comp rx=0 tx=0).stop 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24108400 msgr2=0x7fcf2419bb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24108400 0x7fcf2419bb20 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c00b830 tx=0x7fcf1c004ce0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 shutdown_connections 2024-05-08T05:27:58.855 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcf24108400 0x7fcf2419bb20 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcf24102390 0x7fcf241a1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fcf100b2910 0x7fcf100b4dd0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 --2- 172.21.15.59:0/3616501982 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcf24102d50 0x7fcf241a1f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:27:58.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.762+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 >> 172.21.15.59:0/3616501982 conn(0x7fcf24079310 msgr2=0x7fcf2406ba50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:27:58.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.763+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 shutdown_connections 2024-05-08T05:27:58.856 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:27:55.763+0000 7fcf2c77a700 1 -- 172.21.15.59:0/3616501982 wait complete. 2024-05-08T05:27:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:59 smithi143 ceph-mon[34337]: pgmap v377: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:27:59 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1739676793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T05:27:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:59 smithi096 ceph-mon[34253]: pgmap v377: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:27:59 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1739676793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T05:27:59.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:59 smithi059 ceph-mon[30304]: pgmap v377: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:27:59.548 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:27:59 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1739676793' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-05-08T05:27:59.584 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182483 got 107374182486 for osd.3 2024-05-08T05:27:59.584 DEBUG:teuthology.parallel:result is None 2024-05-08T05:28:00.414 INFO:tasks.cephadm.ceph_manager.ceph:need seq 257698037812 got 257698037817 for osd.9 2024-05-08T05:28:00.414 DEBUG:teuthology.parallel:result is None 2024-05-08T05:28:01.014 INFO:tasks.cephadm.ceph_manager.ceph:need seq 154618822728 got 154618822732 for osd.5 2024-05-08T05:28:01.014 DEBUG:teuthology.parallel:result is None 2024-05-08T05:28:01.313 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:01 smithi059 ceph-mon[30304]: pgmap v378: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:01 smithi143 ceph-mon[34337]: pgmap v378: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:01 smithi096 ceph-mon[34253]: pgmap v378: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:02.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 -- 172.21.15.59:0/2200501266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0103ee0 msgr2=0x7f53d0110370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:02.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 --2- 172.21.15.59:0/2200501266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0103ee0 0x7f53d0110370 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b0a0 tx=0x7f53c800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 -- 172.21.15.59:0/2200501266 shutdown_connections 2024-05-08T05:28:02.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 --2- 172.21.15.59:0/2200501266 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0103ee0 0x7f53d0110370 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 --2- 172.21.15.59:0/2200501266 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d0109530 0x7f53d0109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 --2- 172.21.15.59:0/2200501266 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f53d0103520 0x7f53d01039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 -- 172.21.15.59:0/2200501266 >> 172.21.15.59:0/2200501266 conn(0x7f53d007a570 msgr2=0x7f53d007a980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:02.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 -- 172.21.15.59:0/2200501266 shutdown_connections 2024-05-08T05:28:02.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 -- 172.21.15.59:0/2200501266 wait complete. 2024-05-08T05:28:02.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.114+0000 7f53d59dc700 1 Processor -- start 2024-05-08T05:28:02.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53d59dc700 1 -- start start 2024-05-08T05:28:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53d59dc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f53d0103520 0x7f53d010cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53d59dc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d0103ee0 0x7f53d010d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53d59dc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0109530 0x7f53d01a5c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53d59dc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0076b00 con 0x7f53d0109530 2024-05-08T05:28:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53d59dc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0076980 con 0x7f53d0103ee0 2024-05-08T05:28:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53d59dc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f53d0076c80 con 0x7f53d0103520 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53cf7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0109530 0x7f53d01a5c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53cf7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0109530 0x7f53d01a5c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37822/0 (socket says 172.21.15.59:37822) 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53cf7fe700 1 -- 172.21.15.59:0/3907760003 learned_addr learned my addr 172.21.15.59:0/3907760003 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53ceffd700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f53d0103520 0x7f53d010cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.115+0000 7f53ce7fc700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d0103ee0 0x7f53d010d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53cf7fe700 1 -- 172.21.15.59:0/3907760003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f53d0103520 msgr2=0x7f53d010cdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53cf7fe700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f53d0103520 0x7f53d010cdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53cf7fe700 1 -- 172.21.15.59:0/3907760003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d0103ee0 msgr2=0x7f53d010d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53cf7fe700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d0103ee0 0x7f53d010d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53cf7fe700 1 -- 172.21.15.59:0/3907760003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53d01a6350 con 0x7f53d0109530 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53cf7fe700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0109530 0x7f53d01a5c50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b070 tx=0x7f53c8009c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53d49da700 1 -- 172.21.15.59:0/3907760003 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8021070 con 0x7f53d0109530 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53d01a65e0 con 0x7f53d0109530 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53d49da700 1 -- 172.21.15.59:0/3907760003 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53c8004e40 con 0x7f53d0109530 2024-05-08T05:28:02.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53d49da700 1 -- 172.21.15.59:0/3907760003 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53c8004630 con 0x7f53d0109530 2024-05-08T05:28:02.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.116+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53d01acd00 con 0x7f53d0109530 2024-05-08T05:28:02.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.117+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f53d006a190 con 0x7f53d0109530 2024-05-08T05:28:02.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.119+0000 7f53d49da700 1 -- 172.21.15.59:0/3907760003 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 17) ==== 149703+0+0 (secure 0 0 0) 0x7f53c8007980 con 0x7f53d0109530 2024-05-08T05:28:02.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.119+0000 7f53d49da700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53b80b2990 0x7f53b80b4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:02.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.120+0000 7f53ceffd700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53b80b2990 0x7f53b80b4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:02.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.120+0000 7f53d49da700 1 -- 172.21.15.59:0/3907760003 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f53c800f020 con 0x7f53d0109530 2024-05-08T05:28:02.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.120+0000 7f53ceffd700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53b80b2990 0x7f53b80b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009be0 tx=0x7f53c0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:02.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.124+0000 7f53d49da700 1 -- 172.21.15.59:0/3907760003 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f53c802a020 con 0x7f53d0109530 2024-05-08T05:28:02.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.340+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f53d01aced0 con 0x7f53d0109530 2024-05-08T05:28:02.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.340+0000 7f53d49da700 1 -- 172.21.15.59:0/3907760003 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f53c80ab2b0 con 0x7f53d0109530 2024-05-08T05:28:02.341 INFO:teuthology.orchestra.run.smithi059.stdout:77309411422 2024-05-08T05:28:02.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53b80b2990 msgr2=0x7f53b80b4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:02.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53b80b2990 0x7f53b80b4e50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f53c0009be0 tx=0x7f53c0009260 comp rx=0 tx=0).stop 2024-05-08T05:28:02.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0109530 msgr2=0x7f53d01a5c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:02.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0109530 0x7f53d01a5c50 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f53c800b070 tx=0x7f53c8009c90 comp rx=0 tx=0).stop 2024-05-08T05:28:02.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 shutdown_connections 2024-05-08T05:28:02.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53d0109530 0x7f53d01a5c50 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f53d0103520 0x7f53d010cdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f53b80b2990 0x7f53b80b4e50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 --2- 172.21.15.59:0/3907760003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f53d0103ee0 0x7f53d010d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:02.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 >> 172.21.15.59:0/3907760003 conn(0x7f53d007a570 msgr2=0x7f53d01070a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:02.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 shutdown_connections 2024-05-08T05:28:02.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:02.343+0000 7f53d59dc700 1 -- 172.21.15.59:0/3907760003 wait complete. 2024-05-08T05:28:03.079 INFO:tasks.cephadm.ceph_manager.ceph:need seq 201863462972 got 201863462978 for osd.7 2024-05-08T05:28:03.079 DEBUG:teuthology.parallel:result is None 2024-05-08T05:28:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:03 smithi059 ceph-mon[30304]: pgmap v379: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:03 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3907760003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T05:28:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:03 smithi143 ceph-mon[34337]: pgmap v379: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:03 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3907760003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T05:28:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:03 smithi096 ceph-mon[34253]: pgmap v379: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:03 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3907760003' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-05-08T05:28:03.776 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411416 got 77309411422 for osd.2 2024-05-08T05:28:03.776 DEBUG:teuthology.parallel:result is None 2024-05-08T05:28:04.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.339+0000 7f605d2d8700 1 -- 172.21.15.59:0/2530750840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058102d50 msgr2=0x7f605810f240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:04.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.339+0000 7f605d2d8700 1 --2- 172.21.15.59:0/2530750840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058102d50 0x7f605810f240 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6050009990 tx=0x7f6050009ca0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.339+0000 7f605d2d8700 1 -- 172.21.15.59:0/2530750840 shutdown_connections 2024-05-08T05:28:04.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.339+0000 7f605d2d8700 1 --2- 172.21.15.59:0/2530750840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058102d50 0x7f605810f240 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.339+0000 7f605d2d8700 1 --2- 172.21.15.59:0/2530750840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6058108400 0x7f60581087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.339+0000 7f605d2d8700 1 --2- 172.21.15.59:0/2530750840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6058102390 0x7f6058102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.339+0000 7f605d2d8700 1 -- 172.21.15.59:0/2530750840 >> 172.21.15.59:0/2530750840 conn(0x7f6058079310 msgr2=0x7f6058079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:04.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.339+0000 7f605d2d8700 1 -- 172.21.15.59:0/2530750840 shutdown_connections 2024-05-08T05:28:04.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.339+0000 7f605d2d8700 1 -- 172.21.15.59:0/2530750840 wait complete. 2024-05-08T05:28:04.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f605d2d8700 1 Processor -- start 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f605d2d8700 1 -- start start 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f605d2d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6058102390 0x7f605819f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f605d2d8700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6058102d50 0x7f605819fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f605d2d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058108400 0x7f6058199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f605d2d8700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6058112540 con 0x7f6058108400 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f605d2d8700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60581123c0 con 0x7f6058102d50 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f605d2d8700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f60581126c0 con 0x7f6058102390 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f60577fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058108400 0x7f6058199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f60577fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058108400 0x7f6058199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37828/0 (socket says 172.21.15.59:37828) 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.340+0000 7f60577fe700 1 -- 172.21.15.59:0/1260003856 learned_addr learned my addr 172.21.15.59:0/1260003856 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f6056ffd700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6058102390 0x7f605819f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f60567fc700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6058102d50 0x7f605819fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f60577fe700 1 -- 172.21.15.59:0/1260003856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6058102390 msgr2=0x7f605819f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:04.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f60577fe700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6058102390 0x7f605819f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f60577fe700 1 -- 172.21.15.59:0/1260003856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6058102d50 msgr2=0x7f605819fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f60577fe700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6058102d50 0x7f605819fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f60577fe700 1 -- 172.21.15.59:0/1260003856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6058199fd0 con 0x7f6058108400 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f6056ffd700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6058102390 0x7f605819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f60577fe700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058108400 0x7f6058199870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f60500047b0 tx=0x7f6050009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f603ffff700 1 -- 172.21.15.59:0/1260003856 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050004830 con 0x7f6058108400 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f603ffff700 1 -- 172.21.15.59:0/1260003856 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6050005020 con 0x7f6058108400 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f605819a260 con 0x7f6058108400 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f603ffff700 1 -- 172.21.15.59:0/1260003856 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6050003df0 con 0x7f6058108400 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.341+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6058103fa0 con 0x7f6058108400 2024-05-08T05:28:04.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.342+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6058103ab0 con 0x7f6058108400 2024-05-08T05:28:04.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.344+0000 7f603ffff700 1 -- 172.21.15.59:0/1260003856 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f60500054d0 con 0x7f6058108400 2024-05-08T05:28:04.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.345+0000 7f603ffff700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f60400b2900 0x7f60400b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:04.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.345+0000 7f6056ffd700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f60400b2900 0x7f60400b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:04.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.345+0000 7f603ffff700 1 -- 172.21.15.59:0/1260003856 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f60500e3850 con 0x7f6058108400 2024-05-08T05:28:04.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.346+0000 7f6056ffd700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f60400b2900 0x7f60400b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f605819a9b0 tx=0x7f6048009040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:04.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.349+0000 7f603ffff700 1 -- 172.21.15.59:0/1260003856 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6050013070 con 0x7f6058108400 2024-05-08T05:28:04.578 INFO:teuthology.orchestra.run.smithi059.stdout:55834574947 2024-05-08T05:28:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.576+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f60580640b0 con 0x7f6058108400 2024-05-08T05:28:04.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.577+0000 7f603ffff700 1 -- 172.21.15.59:0/1260003856 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f60500ab270 con 0x7f6058108400 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f60400b2900 msgr2=0x7f60400b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f60400b2900 0x7f60400b4dc0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f605819a9b0 tx=0x7f6048009040 comp rx=0 tx=0).stop 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058108400 msgr2=0x7f6058199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058108400 0x7f6058199870 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f60500047b0 tx=0x7f6050009ee0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 shutdown_connections 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6058108400 0x7f6058199870 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6058102390 0x7f605819f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f60400b2900 0x7f60400b4dc0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 --2- 172.21.15.59:0/1260003856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6058102d50 0x7f605819fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:04.580 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.579+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 >> 172.21.15.59:0/1260003856 conn(0x7f6058079310 msgr2=0x7f6058101590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.580+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 shutdown_connections 2024-05-08T05:28:04.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.580+0000 7f605d2d8700 1 -- 172.21.15.59:0/1260003856 wait complete. 2024-05-08T05:28:05.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:05 smithi059 ceph-mon[30304]: pgmap v380: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:05.206 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:05 smithi059 ceph-mon[30304]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T05:28:05.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:28:05.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:05 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:28:05.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:05 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1260003856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T05:28:05.207 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:05 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2053132721' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T05:28:05.235 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574941 got 55834574947 for osd.1 2024-05-08T05:28:05.236 DEBUG:teuthology.parallel:result is None 2024-05-08T05:28:05.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 -- 172.21.15.59:0/3256094675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006b180 msgr2=0x7fec5006b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:05.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 --2- 172.21.15.59:0/3256094675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006b180 0x7fec5006b600 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fec40009990 tx=0x7fec40009ca0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 -- 172.21.15.59:0/3256094675 shutdown_connections 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 --2- 172.21.15.59:0/3256094675 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec5006bb40 0x7fec50110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 --2- 172.21.15.59:0/3256094675 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec50109510 0x7fec501098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 --2- 172.21.15.59:0/3256094675 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006b180 0x7fec5006b600 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 -- 172.21.15.59:0/3256094675 >> 172.21.15.59:0/3256094675 conn(0x7fec50079350 msgr2=0x7fec50079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 -- 172.21.15.59:0/3256094675 shutdown_connections 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 -- 172.21.15.59:0/3256094675 wait complete. 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 Processor -- start 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.704+0000 7fec58441700 1 -- start start 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec58441700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec5006b180 0x7fec5019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec58441700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006bb40 0x7fec5019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec58441700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec50109510 0x7fec501999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec58441700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec50113640 con 0x7fec5006bb40 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec58441700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec501134c0 con 0x7fec50109510 2024-05-08T05:28:05.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec58441700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fec501137c0 con 0x7fec5006b180 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec559dc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006bb40 0x7fec5019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec559dc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006bb40 0x7fec5019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37858/0 (socket says 172.21.15.59:37858) 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec559dc700 1 -- 172.21.15.59:0/2053132721 learned_addr learned my addr 172.21.15.59:0/2053132721 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec561dd700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec5006b180 0x7fec5019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec569de700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec50109510 0x7fec501999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec559dc700 1 -- 172.21.15.59:0/2053132721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec5006b180 msgr2=0x7fec5019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec559dc700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec5006b180 0x7fec5019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec559dc700 1 -- 172.21.15.59:0/2053132721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec50109510 msgr2=0x7fec501999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec559dc700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec50109510 0x7fec501999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.705+0000 7fec559dc700 1 -- 172.21.15.59:0/2053132721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec5019a100 con 0x7fec5006bb40 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.706+0000 7fec559dc700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006bb40 0x7fec5019fcd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fec40000c00 tx=0x7fec400047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.706+0000 7fec477fe700 1 -- 172.21.15.59:0/2053132721 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40027070 con 0x7fec5006bb40 2024-05-08T05:28:05.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.706+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec5019a390 con 0x7fec5006bb40 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.706+0000 7fec477fe700 1 -- 172.21.15.59:0/2053132721 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec40003f20 con 0x7fec5006bb40 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.706+0000 7fec477fe700 1 -- 172.21.15.59:0/2053132721 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec40022ca0 con 0x7fec5006bb40 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.706+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec501a6660 con 0x7fec5006bb40 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.707+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec5006a190 con 0x7fec5006bb40 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.708+0000 7fec477fe700 1 -- 172.21.15.59:0/2053132721 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fec4002a070 con 0x7fec5006bb40 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.709+0000 7fec477fe700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fec3c0b2880 0x7fec3c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.710+0000 7fec561dd700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fec3c0b2880 0x7fec3c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.710+0000 7fec477fe700 1 -- 172.21.15.59:0/2053132721 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fec400e33a0 con 0x7fec5006bb40 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.710+0000 7fec561dd700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fec3c0b2880 0x7fec3c0b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005950 tx=0x7fec4c00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:05.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.714+0000 7fec477fe700 1 -- 172.21.15.59:0/2053132721 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec40013070 con 0x7fec5006bb40 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stdout:279172874293 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.929+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7fec5002e510 con 0x7fec5006bb40 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.929+0000 7fec477fe700 1 -- 172.21.15.59:0/2053132721 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7fec400aadc0 con 0x7fec5006bb40 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fec3c0b2880 msgr2=0x7fec3c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fec3c0b2880 0x7fec3c0b4d40 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fec4c005950 tx=0x7fec4c00a300 comp rx=0 tx=0).stop 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006bb40 msgr2=0x7fec5019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006bb40 0x7fec5019fcd0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fec40000c00 tx=0x7fec400047b0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 shutdown_connections 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fec3c0b2880 0x7fec3c0b4d40 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec50109510 0x7fec501999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec5006b180 0x7fec5019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 --2- 172.21.15.59:0/2053132721 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec5006bb40 0x7fec5019fcd0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:05.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.931+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 >> 172.21.15.59:0/2053132721 conn(0x7fec50079350 msgr2=0x7fec50101060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:05.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.932+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 shutdown_connections 2024-05-08T05:28:05.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:04.932+0000 7fec58441700 1 -- 172.21.15.59:0/2053132721 wait complete. 2024-05-08T05:28:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:05 smithi143 ceph-mon[34337]: pgmap v380: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:05 smithi143 ceph-mon[34337]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T05:28:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:28:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:05 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:28:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:05 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1260003856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T05:28:05.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:05 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2053132721' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T05:28:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:05 smithi096 ceph-mon[34253]: pgmap v380: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:05 smithi096 ceph-mon[34253]: mgrmap e18: x(active, since 10m), standbys: z, y 2024-05-08T05:28:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:28:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:05 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:28:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:05 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1260003856' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-05-08T05:28:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:05 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2053132721' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-05-08T05:28:06.125 INFO:tasks.cephadm.ceph_manager.ceph:need seq 279172874285 got 279172874293 for osd.10 2024-05-08T05:28:06.125 DEBUG:teuthology.parallel:result is None 2024-05-08T05:28:06.125 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T05:28:06.125 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:28:06.379 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:28:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:28:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:28:06.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:28:06.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:28:06.496 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:28:06.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:28:06.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:28:06.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:28:07.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:07 smithi143 ceph-mon[34337]: pgmap v381: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-08T05:28:07.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:07 smithi096 ceph-mon[34253]: pgmap v381: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-08T05:28:07.807 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:07 smithi059 ceph-mon[30304]: pgmap v381: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 341 B/s wr, 10 op/s 2024-05-08T05:28:08.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.074+0000 7fa83c4df700 1 -- 172.21.15.59:0/2385795892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa834102d60 msgr2=0x7fa83410f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:08.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.074+0000 7fa83c4df700 1 --2- 172.21.15.59:0/2385795892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa834102d60 0x7fa83410f250 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00b0a0 tx=0x7fa82c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.075+0000 7fa83c4df700 1 -- 172.21.15.59:0/2385795892 shutdown_connections 2024-05-08T05:28:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.075+0000 7fa83c4df700 1 --2- 172.21.15.59:0/2385795892 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa834102d60 0x7fa83410f250 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.075+0000 7fa83c4df700 1 --2- 172.21.15.59:0/2385795892 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa834108410 0x7fa8341087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.075+0000 7fa83c4df700 1 --2- 172.21.15.59:0/2385795892 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa8341023a0 0x7fa834102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.075+0000 7fa83c4df700 1 -- 172.21.15.59:0/2385795892 >> 172.21.15.59:0/2385795892 conn(0x7fa834079350 msgr2=0x7fa834079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.075+0000 7fa83c4df700 1 -- 172.21.15.59:0/2385795892 shutdown_connections 2024-05-08T05:28:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.075+0000 7fa83c4df700 1 -- 172.21.15.59:0/2385795892 wait complete. 2024-05-08T05:28:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.075+0000 7fa83c4df700 1 Processor -- start 2024-05-08T05:28:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.075+0000 7fa83c4df700 1 -- start start 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83c4df700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8341023a0 0x7fa83419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83c4df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa834102d60 0x7fa83419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83c4df700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa834108410 0x7fa8341998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83c4df700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa834112530 con 0x7fa8341023a0 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83c4df700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341123b0 con 0x7fa834102d60 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83c4df700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa8341126b0 con 0x7fa834108410 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83a27b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8341023a0 0x7fa83419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83a27b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8341023a0 0x7fa83419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37870/0 (socket says 172.21.15.59:37870) 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83a27b700 1 -- 172.21.15.59:0/188489348 learned_addr learned my addr 172.21.15.59:0/188489348 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83aa7c700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa834108410 0x7fa8341998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa839a7a700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa834102d60 0x7fa83419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83a27b700 1 -- 172.21.15.59:0/188489348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa834108410 msgr2=0x7fa8341998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83a27b700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa834108410 0x7fa8341998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83a27b700 1 -- 172.21.15.59:0/188489348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa834102d60 msgr2=0x7fa83419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83a27b700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa834102d60 0x7fa83419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83a27b700 1 -- 172.21.15.59:0/188489348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83419a050 con 0x7fa8341023a0 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa839a7a700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa834102d60 0x7fa83419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83a27b700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8341023a0 0x7fa83419f730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa83000d950 tx=0x7fa83000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa82b7fe700 1 -- 172.21.15.59:0/188489348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa830016070 con 0x7fa8341023a0 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.076+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83419a340 con 0x7fa8341023a0 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.077+0000 7fa82b7fe700 1 -- 172.21.15.59:0/188489348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa830009df0 con 0x7fa8341023a0 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.077+0000 7fa82b7fe700 1 -- 172.21.15.59:0/188489348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa83000f3c0 con 0x7fa8341023a0 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.077+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8341a6600 con 0x7fa8341023a0 2024-05-08T05:28:08.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.078+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa83406a190 con 0x7fa8341023a0 2024-05-08T05:28:08.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.079+0000 7fa82b7fe700 1 -- 172.21.15.59:0/188489348 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa830019070 con 0x7fa8341023a0 2024-05-08T05:28:08.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.080+0000 7fa82b7fe700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa8200b2880 0x7fa8200b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:08.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.080+0000 7fa839a7a700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa8200b2880 0x7fa8200b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:08.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.080+0000 7fa82b7fe700 1 -- 172.21.15.59:0/188489348 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa8300e2150 con 0x7fa8341023a0 2024-05-08T05:28:08.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.080+0000 7fa839a7a700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa8200b2880 0x7fa8200b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa824005950 tx=0x7fa8240058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:08.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.083+0000 7fa82b7fe700 1 -- 172.21.15.59:0/188489348 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa8300a9a80 con 0x7fa8341023a0 2024-05-08T05:28:08.293 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.293+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa8341061c0 con 0x7fa8200b2880 2024-05-08T05:28:08.303 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.302+0000 7fa82b7fe700 1 -- 172.21.15.59:0/188489348 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340900 (secure 0 0 0) 0x7fa8341061c0 con 0x7fa8200b2880 2024-05-08T05:28:08.308 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:08.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa8200b2880 msgr2=0x7fa8200b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa8200b2880 0x7fa8200b4d40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fa824005950 tx=0x7fa8240058e0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8341023a0 msgr2=0x7fa83419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8341023a0 0x7fa83419f730 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7fa83000d950 tx=0x7fa83000dc60 comp rx=0 tx=0).stop 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 shutdown_connections 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa834108410 0x7fa8341998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa8341023a0 0x7fa83419f730 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa8200b2880 0x7fa8200b4d40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 --2- 172.21.15.59:0/188489348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa834102d60 0x7fa83419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 >> 172.21.15.59:0/188489348 conn(0x7fa834079350 msgr2=0x7fa8341016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 shutdown_connections 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:08.306+0000 7fa83c4df700 1 -- 172.21.15.59:0/188489348 wait complete. 2024-05-08T05:28:08.312 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T05:28:08.965 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":382,"stamp":"2024-05-08T05:28:08.023335+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":663797,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1092,"num_read_kb":653,"num_write":381,"num_write_kb":2303,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951840,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":761,"ondisk_log_size":761,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361436,"kb_used_data":12260,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580388,"statfs":{"total":1151940427776,"available":1151570317312,"internally_reserved":0,"allocated":12554240,"data_stored":8212598,"data_compressed":496944,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":564,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":59,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003273"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018215+0000","last_change":"2024-05-08T05:25:25.667895+0000","last_active":"2024-05-08T05:28:04.018215+0000","last_peered":"2024-05-08T05:28:04.018215+0000","last_clean":"2024-05-08T05:28:04.018215+0000","last_became_active":"2024-05-08T05:25:25.667563+0000","last_became_peered":"2024-05-08T05:25:25.667563+0000","last_unstale":"2024-05-08T05:28:04.018215+0000","last_undegraded":"2024-05-08T05:28:04.018215+0000","last_fullsized":"2024-05-08T05:28:04.018215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:47.677082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018936+0000","last_change":"2024-05-08T05:25:25.673346+0000","last_active":"2024-05-08T05:28:04.018936+0000","last_peered":"2024-05-08T05:28:04.018936+0000","last_clean":"2024-05-08T05:28:04.018936+0000","last_became_active":"2024-05-08T05:25:25.673011+0000","last_became_peered":"2024-05-08T05:25:25.673011+0000","last_unstale":"2024-05-08T05:28:04.018936+0000","last_undegraded":"2024-05-08T05:28:04.018936+0000","last_fullsized":"2024-05-08T05:28:04.018936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:21:05.091177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.013105+0000","last_change":"2024-05-08T05:25:25.670469+0000","last_active":"2024-05-08T05:28:04.013105+0000","last_peered":"2024-05-08T05:28:04.013105+0000","last_clean":"2024-05-08T05:28:04.013105+0000","last_became_active":"2024-05-08T05:25:25.669700+0000","last_became_peered":"2024-05-08T05:25:25.669700+0000","last_unstale":"2024-05-08T05:28:04.013105+0000","last_undegraded":"2024-05-08T05:28:04.013105+0000","last_fullsized":"2024-05-08T05:28:04.013105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:06.046633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.009536+0000","last_change":"2024-05-08T05:25:25.659937+0000","last_active":"2024-05-08T05:28:04.009536+0000","last_peered":"2024-05-08T05:28:04.009536+0000","last_clean":"2024-05-08T05:28:04.009536+0000","last_became_active":"2024-05-08T05:25:25.658892+0000","last_became_peered":"2024-05-08T05:25:25.658892+0000","last_unstale":"2024-05-08T05:28:04.009536+0000","last_undegraded":"2024-05-08T05:28:04.009536+0000","last_fullsized":"2024-05-08T05:28:04.009536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:10.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.006988+0000","last_change":"2024-05-08T05:25:25.669132+0000","last_active":"2024-05-08T05:28:04.006988+0000","last_peered":"2024-05-08T05:28:04.006988+0000","last_clean":"2024-05-08T05:28:04.006988+0000","last_became_active":"2024-05-08T05:25:25.668556+0000","last_became_peered":"2024-05-08T05:25:25.668556+0000","last_unstale":"2024-05-08T05:28:04.006988+0000","last_undegraded":"2024-05-08T05:28:04.006988+0000","last_fullsized":"2024-05-08T05:28:04.006988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:28.443763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765653+0000","last_change":"2024-05-08T05:25:25.668978+0000","last_active":"2024-05-08T05:27:05.765653+0000","last_peered":"2024-05-08T05:27:05.765653+0000","last_clean":"2024-05-08T05:27:05.765653+0000","last_became_active":"2024-05-08T05:25:25.668647+0000","last_became_peered":"2024-05-08T05:25:25.668647+0000","last_unstale":"2024-05-08T05:27:05.765653+0000","last_undegraded":"2024-05-08T05:27:05.765653+0000","last_fullsized":"2024-05-08T05:27:05.765653+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:07:00.962620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.014161+0000","last_change":"2024-05-08T05:25:25.664784+0000","last_active":"2024-05-08T05:28:04.014161+0000","last_peered":"2024-05-08T05:28:04.014161+0000","last_clean":"2024-05-08T05:28:04.014161+0000","last_became_active":"2024-05-08T05:25:25.664390+0000","last_became_peered":"2024-05-08T05:25:25.664390+0000","last_unstale":"2024-05-08T05:28:04.014161+0000","last_undegraded":"2024-05-08T05:28:04.014161+0000","last_fullsized":"2024-05-08T05:28:04.014161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:08:35.358796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.021523+0000","last_change":"2024-05-08T05:25:25.677139+0000","last_active":"2024-05-08T05:28:04.021523+0000","last_peered":"2024-05-08T05:28:04.021523+0000","last_clean":"2024-05-08T05:28:04.021523+0000","last_became_active":"2024-05-08T05:25:25.676444+0000","last_became_peered":"2024-05-08T05:25:25.676444+0000","last_unstale":"2024-05-08T05:28:04.021523+0000","last_undegraded":"2024-05-08T05:28:04.021523+0000","last_fullsized":"2024-05-08T05:28:04.021523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:56.704152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.016183+0000","last_change":"2024-05-08T05:25:25.675593+0000","last_active":"2024-05-08T05:28:04.016183+0000","last_peered":"2024-05-08T05:28:04.016183+0000","last_clean":"2024-05-08T05:28:04.016183+0000","last_became_active":"2024-05-08T05:25:25.675136+0000","last_became_peered":"2024-05-08T05:25:25.675136+0000","last_unstale":"2024-05-08T05:28:04.016183+0000","last_undegraded":"2024-05-08T05:28:04.016183+0000","last_fullsized":"2024-05-08T05:28:04.016183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:49.042956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.766418+0000","last_change":"2024-05-08T05:25:25.669384+0000","last_active":"2024-05-08T05:27:05.766418+0000","last_peered":"2024-05-08T05:27:05.766418+0000","last_clean":"2024-05-08T05:27:05.766418+0000","last_became_active":"2024-05-08T05:25:25.668861+0000","last_became_peered":"2024-05-08T05:25:25.668861+0000","last_unstale":"2024-05-08T05:27:05.766418+0000","last_undegraded":"2024-05-08T05:27:05.766418+0000","last_fullsized":"2024-05-08T05:27:05.766418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:57.491242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.014903+0000","last_change":"2024-05-08T05:25:25.669463+0000","last_active":"2024-05-08T05:28:04.014903+0000","last_peered":"2024-05-08T05:28:04.014903+0000","last_clean":"2024-05-08T05:28:04.014903+0000","last_became_active":"2024-05-08T05:25:25.668672+0000","last_became_peered":"2024-05-08T05:25:25.668672+0000","last_unstale":"2024-05-08T05:28:04.014903+0000","last_undegraded":"2024-05-08T05:28:04.014903+0000","last_fullsized":"2024-05-08T05:28:04.014903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:30.779473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.005727+0000","last_change":"2024-05-08T05:25:25.670378+0000","last_active":"2024-05-08T05:28:04.005727+0000","last_peered":"2024-05-08T05:28:04.005727+0000","last_clean":"2024-05-08T05:28:04.005727+0000","last_became_active":"2024-05-08T05:25:25.669801+0000","last_became_peered":"2024-05-08T05:25:25.669801+0000","last_unstale":"2024-05-08T05:28:04.005727+0000","last_undegraded":"2024-05-08T05:28:04.005727+0000","last_fullsized":"2024-05-08T05:28:04.005727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:49.614163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.012174+0000","last_change":"2024-05-08T05:25:25.670874+0000","last_active":"2024-05-08T05:28:04.012174+0000","last_peered":"2024-05-08T05:28:04.012174+0000","last_clean":"2024-05-08T05:28:04.012174+0000","last_became_active":"2024-05-08T05:25:25.670557+0000","last_became_peered":"2024-05-08T05:25:25.670557+0000","last_unstale":"2024-05-08T05:28:04.012174+0000","last_undegraded":"2024-05-08T05:28:04.012174+0000","last_fullsized":"2024-05-08T05:28:04.012174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:58.594223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.010134+0000","last_change":"2024-05-08T05:25:25.670015+0000","last_active":"2024-05-08T05:28:04.010134+0000","last_peered":"2024-05-08T05:28:04.010134+0000","last_clean":"2024-05-08T05:28:04.010134+0000","last_became_active":"2024-05-08T05:25:25.669493+0000","last_became_peered":"2024-05-08T05:25:25.669493+0000","last_unstale":"2024-05-08T05:28:04.010134+0000","last_undegraded":"2024-05-08T05:28:04.010134+0000","last_fullsized":"2024-05-08T05:28:04.010134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:49.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.013746+0000","last_change":"2024-05-08T05:25:25.659651+0000","last_active":"2024-05-08T05:28:04.013746+0000","last_peered":"2024-05-08T05:28:04.013746+0000","last_clean":"2024-05-08T05:28:04.013746+0000","last_became_active":"2024-05-08T05:25:25.659308+0000","last_became_peered":"2024-05-08T05:25:25.659308+0000","last_unstale":"2024-05-08T05:28:04.013746+0000","last_undegraded":"2024-05-08T05:28:04.013746+0000","last_fullsized":"2024-05-08T05:28:04.013746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:49.945474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.017418+0000","last_change":"2024-05-08T05:25:25.663696+0000","last_active":"2024-05-08T05:28:04.017418+0000","last_peered":"2024-05-08T05:28:04.017418+0000","last_clean":"2024-05-08T05:28:04.017418+0000","last_became_active":"2024-05-08T05:25:25.662590+0000","last_became_peered":"2024-05-08T05:25:25.662590+0000","last_unstale":"2024-05-08T05:28:04.017418+0000","last_undegraded":"2024-05-08T05:28:04.017418+0000","last_fullsized":"2024-05-08T05:28:04.017418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:29:21.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018348+0000","last_change":"2024-05-08T05:25:25.668552+0000","last_active":"2024-05-08T05:28:04.018348+0000","last_peered":"2024-05-08T05:28:04.018348+0000","last_clean":"2024-05-08T05:28:04.018348+0000","last_became_active":"2024-05-08T05:25:25.668185+0000","last_became_peered":"2024-05-08T05:25:25.668185+0000","last_unstale":"2024-05-08T05:28:04.018348+0000","last_undegraded":"2024-05-08T05:28:04.018348+0000","last_fullsized":"2024-05-08T05:28:04.018348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:25.762909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764585+0000","last_change":"2024-05-08T05:25:25.668259+0000","last_active":"2024-05-08T05:27:05.764585+0000","last_peered":"2024-05-08T05:27:05.764585+0000","last_clean":"2024-05-08T05:27:05.764585+0000","last_became_active":"2024-05-08T05:25:25.667767+0000","last_became_peered":"2024-05-08T05:25:25.667767+0000","last_unstale":"2024-05-08T05:27:05.764585+0000","last_undegraded":"2024-05-08T05:27:05.764585+0000","last_fullsized":"2024-05-08T05:27:05.764585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:36:34.984139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.017148+0000","last_change":"2024-05-08T05:25:25.663381+0000","last_active":"2024-05-08T05:28:04.017148+0000","last_peered":"2024-05-08T05:28:04.017148+0000","last_clean":"2024-05-08T05:28:04.017148+0000","last_became_active":"2024-05-08T05:25:25.663019+0000","last_became_peered":"2024-05-08T05:25:25.663019+0000","last_unstale":"2024-05-08T05:28:04.017148+0000","last_undegraded":"2024-05-08T05:28:04.017148+0000","last_fullsized":"2024-05-08T05:28:04.017148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:58.440446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564094+0000","last_change":"2024-05-08T05:25:37.052656+0000","last_active":"2024-05-08T05:27:05.564094+0000","last_peered":"2024-05-08T05:27:05.564094+0000","last_clean":"2024-05-08T05:27:05.564094+0000","last_became_active":"2024-05-08T05:25:37.052237+0000","last_became_peered":"2024-05-08T05:25:37.052237+0000","last_unstale":"2024-05-08T05:27:05.564094+0000","last_undegraded":"2024-05-08T05:27:05.564094+0000","last_fullsized":"2024-05-08T05:27:05.564094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:54.354488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"88'13","reported_seq":154,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:02.551596+0000","last_change":"2024-05-08T05:25:22.649492+0000","last_active":"2024-05-08T05:28:02.551596+0000","last_peered":"2024-05-08T05:28:02.551596+0000","last_clean":"2024-05-08T05:28:02.551596+0000","last_became_active":"2024-05-08T05:25:22.649132+0000","last_became_peered":"2024-05-08T05:25:22.649132+0000","last_unstale":"2024-05-08T05:28:02.551596+0000","last_undegraded":"2024-05-08T05:28:02.551596+0000","last_fullsized":"2024-05-08T05:28:02.551596+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:24.474848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":97,"num_read_kb":97,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.013390+0000","last_change":"2024-05-08T05:25:25.667811+0000","last_active":"2024-05-08T05:28:04.013390+0000","last_peered":"2024-05-08T05:28:04.013390+0000","last_clean":"2024-05-08T05:28:04.013390+0000","last_became_active":"2024-05-08T05:25:25.667449+0000","last_became_peered":"2024-05-08T05:25:25.667449+0000","last_unstale":"2024-05-08T05:28:04.013390+0000","last_undegraded":"2024-05-08T05:28:04.013390+0000","last_fullsized":"2024-05-08T05:28:04.013390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:29.293702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764379+0000","last_change":"2024-05-08T05:26:05.308251+0000","last_active":"2024-05-08T05:27:05.764379+0000","last_peered":"2024-05-08T05:27:05.764379+0000","last_clean":"2024-05-08T05:27:05.764379+0000","last_became_active":"2024-05-08T05:26:05.307622+0000","last_became_peered":"2024-05-08T05:26:05.307622+0000","last_unstale":"2024-05-08T05:27:05.764379+0000","last_undegraded":"2024-05-08T05:27:05.764379+0000","last_fullsized":"2024-05-08T05:27:05.764379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:51.116094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563675+0000","last_change":"2024-05-08T05:25:22.748130+0000","last_active":"2024-05-08T05:27:05.563675+0000","last_peered":"2024-05-08T05:27:05.563675+0000","last_clean":"2024-05-08T05:27:05.563675+0000","last_became_active":"2024-05-08T05:25:22.747769+0000","last_became_peered":"2024-05-08T05:25:22.747769+0000","last_unstale":"2024-05-08T05:27:05.563675+0000","last_undegraded":"2024-05-08T05:27:05.563675+0000","last_fullsized":"2024-05-08T05:27:05.563675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:04.086851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.016074+0000","last_change":"2024-05-08T05:25:25.672114+0000","last_active":"2024-05-08T05:28:04.016074+0000","last_peered":"2024-05-08T05:28:04.016074+0000","last_clean":"2024-05-08T05:28:04.016074+0000","last_became_active":"2024-05-08T05:25:25.671761+0000","last_became_peered":"2024-05-08T05:25:25.671761+0000","last_unstale":"2024-05-08T05:28:04.016074+0000","last_undegraded":"2024-05-08T05:28:04.016074+0000","last_fullsized":"2024-05-08T05:28:04.016074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:40.684565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765356+0000","last_change":"2024-05-08T05:25:37.058375+0000","last_active":"2024-05-08T05:27:05.765356+0000","last_peered":"2024-05-08T05:27:05.765356+0000","last_clean":"2024-05-08T05:27:05.765356+0000","last_became_active":"2024-05-08T05:25:37.057844+0000","last_became_peered":"2024-05-08T05:25:37.057844+0000","last_unstale":"2024-05-08T05:27:05.765356+0000","last_undegraded":"2024-05-08T05:27:05.765356+0000","last_fullsized":"2024-05-08T05:27:05.765356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:51:42.881830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.576781+0000","last_change":"2024-05-08T05:27:05.576781+0000","last_active":"2024-05-08T05:27:05.576781+0000","last_peered":"2024-05-08T05:27:05.576781+0000","last_clean":"2024-05-08T05:27:05.576781+0000","last_became_active":"2024-05-08T05:27:05.566420+0000","last_became_peered":"2024-05-08T05:27:05.566420+0000","last_unstale":"2024-05-08T05:27:05.576781+0000","last_undegraded":"2024-05-08T05:27:05.576781+0000","last_fullsized":"2024-05-08T05:27:05.576781+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:09.385135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.019679+0000","last_change":"2024-05-08T05:25:25.673300+0000","last_active":"2024-05-08T05:28:04.019679+0000","last_peered":"2024-05-08T05:28:04.019679+0000","last_clean":"2024-05-08T05:28:04.019679+0000","last_became_active":"2024-05-08T05:25:25.672881+0000","last_became_peered":"2024-05-08T05:25:25.672881+0000","last_unstale":"2024-05-08T05:28:04.019679+0000","last_undegraded":"2024-05-08T05:28:04.019679+0000","last_fullsized":"2024-05-08T05:28:04.019679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:16.111775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764689+0000","last_change":"2024-05-08T05:25:37.058903+0000","last_active":"2024-05-08T05:27:05.764689+0000","last_peered":"2024-05-08T05:27:05.764689+0000","last_clean":"2024-05-08T05:27:05.764689+0000","last_became_active":"2024-05-08T05:25:37.058508+0000","last_became_peered":"2024-05-08T05:25:37.058508+0000","last_unstale":"2024-05-08T05:27:05.764689+0000","last_undegraded":"2024-05-08T05:27:05.764689+0000","last_fullsized":"2024-05-08T05:27:05.764689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:15:16.605748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.567090+0000","last_change":"2024-05-08T05:25:22.646191+0000","last_active":"2024-05-08T05:27:05.567090+0000","last_peered":"2024-05-08T05:27:05.567090+0000","last_clean":"2024-05-08T05:27:05.567090+0000","last_became_active":"2024-05-08T05:25:22.645187+0000","last_became_peered":"2024-05-08T05:25:22.645187+0000","last_unstale":"2024-05-08T05:27:05.567090+0000","last_undegraded":"2024-05-08T05:27:05.567090+0000","last_fullsized":"2024-05-08T05:27:05.567090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:13.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008387+0000","last_change":"2024-05-08T05:26:05.308149+0000","last_active":"2024-05-08T05:28:04.008387+0000","last_peered":"2024-05-08T05:28:04.008387+0000","last_clean":"2024-05-08T05:28:04.008387+0000","last_became_active":"2024-05-08T05:26:05.307602+0000","last_became_peered":"2024-05-08T05:26:05.307602+0000","last_unstale":"2024-05-08T05:28:04.008387+0000","last_undegraded":"2024-05-08T05:28:04.008387+0000","last_fullsized":"2024-05-08T05:28:04.008387+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:02.791894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564446+0000","last_change":"2024-05-08T05:25:37.059052+0000","last_active":"2024-05-08T05:27:05.564446+0000","last_peered":"2024-05-08T05:27:05.564446+0000","last_clean":"2024-05-08T05:27:05.564446+0000","last_became_active":"2024-05-08T05:25:37.058545+0000","last_became_peered":"2024-05-08T05:25:37.058545+0000","last_unstale":"2024-05-08T05:27:05.564446+0000","last_undegraded":"2024-05-08T05:27:05.564446+0000","last_fullsized":"2024-05-08T05:27:05.564446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:31.521284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.555379+0000","last_change":"2024-05-08T05:25:22.648874+0000","last_active":"2024-05-08T05:28:04.555379+0000","last_peered":"2024-05-08T05:28:04.555379+0000","last_clean":"2024-05-08T05:28:04.555379+0000","last_became_active":"2024-05-08T05:25:22.648477+0000","last_became_peered":"2024-05-08T05:25:22.648477+0000","last_unstale":"2024-05-08T05:28:04.555379+0000","last_undegraded":"2024-05-08T05:28:04.555379+0000","last_fullsized":"2024-05-08T05:28:04.555379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:08.075904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.003459+0000","last_change":"2024-05-08T05:25:25.662449+0000","last_active":"2024-05-08T05:28:04.003459+0000","last_peered":"2024-05-08T05:28:04.003459+0000","last_clean":"2024-05-08T05:28:04.003459+0000","last_became_active":"2024-05-08T05:25:25.662089+0000","last_became_peered":"2024-05-08T05:25:25.662089+0000","last_unstale":"2024-05-08T05:28:04.003459+0000","last_undegraded":"2024-05-08T05:28:04.003459+0000","last_fullsized":"2024-05-08T05:28:04.003459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:31.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563983+0000","last_change":"2024-05-08T05:25:37.055548+0000","last_active":"2024-05-08T05:27:05.563983+0000","last_peered":"2024-05-08T05:27:05.563983+0000","last_clean":"2024-05-08T05:27:05.563983+0000","last_became_active":"2024-05-08T05:25:37.055138+0000","last_became_peered":"2024-05-08T05:25:37.055138+0000","last_unstale":"2024-05-08T05:27:05.563983+0000","last_undegraded":"2024-05-08T05:27:05.563983+0000","last_fullsized":"2024-05-08T05:27:05.563983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:45.697831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765546+0000","last_change":"2024-05-08T05:25:22.648072+0000","last_active":"2024-05-08T05:27:05.765546+0000","last_peered":"2024-05-08T05:27:05.765546+0000","last_clean":"2024-05-08T05:27:05.765546+0000","last_became_active":"2024-05-08T05:25:22.647457+0000","last_became_peered":"2024-05-08T05:25:22.647457+0000","last_unstale":"2024-05-08T05:27:05.765546+0000","last_undegraded":"2024-05-08T05:27:05.765546+0000","last_fullsized":"2024-05-08T05:27:05.765546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:17.476479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.010562+0000","last_change":"2024-05-08T05:25:25.662698+0000","last_active":"2024-05-08T05:28:04.010562+0000","last_peered":"2024-05-08T05:28:04.010562+0000","last_clean":"2024-05-08T05:28:04.010562+0000","last_became_active":"2024-05-08T05:25:25.662368+0000","last_became_peered":"2024-05-08T05:25:25.662368+0000","last_unstale":"2024-05-08T05:28:04.010562+0000","last_undegraded":"2024-05-08T05:28:04.010562+0000","last_fullsized":"2024-05-08T05:28:04.010562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:38.511037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.565170+0000","last_change":"2024-05-08T05:25:37.054407+0000","last_active":"2024-05-08T05:27:05.565170+0000","last_peered":"2024-05-08T05:27:05.565170+0000","last_clean":"2024-05-08T05:27:05.565170+0000","last_became_active":"2024-05-08T05:25:37.053999+0000","last_became_peered":"2024-05-08T05:25:37.053999+0000","last_unstale":"2024-05-08T05:27:05.565170+0000","last_undegraded":"2024-05-08T05:27:05.565170+0000","last_fullsized":"2024-05-08T05:27:05.565170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:16.942601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.560574+0000","last_change":"2024-05-08T05:25:22.643654+0000","last_active":"2024-05-08T05:27:05.560574+0000","last_peered":"2024-05-08T05:27:05.560574+0000","last_clean":"2024-05-08T05:27:05.560574+0000","last_became_active":"2024-05-08T05:25:22.643315+0000","last_became_peered":"2024-05-08T05:25:22.643315+0000","last_unstale":"2024-05-08T05:27:05.560574+0000","last_undegraded":"2024-05-08T05:27:05.560574+0000","last_fullsized":"2024-05-08T05:27:05.560574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:53.514629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020446+0000","last_change":"2024-05-08T05:25:25.663148+0000","last_active":"2024-05-08T05:28:04.020446+0000","last_peered":"2024-05-08T05:28:04.020446+0000","last_clean":"2024-05-08T05:28:04.020446+0000","last_became_active":"2024-05-08T05:25:25.662770+0000","last_became_peered":"2024-05-08T05:25:25.662770+0000","last_unstale":"2024-05-08T05:28:04.020446+0000","last_undegraded":"2024-05-08T05:28:04.020446+0000","last_fullsized":"2024-05-08T05:28:04.020446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:46.458233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.723878+0000","last_change":"2024-05-08T05:25:37.054265+0000","last_active":"2024-05-08T05:27:05.723878+0000","last_peered":"2024-05-08T05:27:05.723878+0000","last_clean":"2024-05-08T05:27:05.723878+0000","last_became_active":"2024-05-08T05:25:37.053579+0000","last_became_peered":"2024-05-08T05:25:37.053579+0000","last_unstale":"2024-05-08T05:27:05.723878+0000","last_undegraded":"2024-05-08T05:27:05.723878+0000","last_fullsized":"2024-05-08T05:27:05.723878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:21.748715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561931+0000","last_change":"2024-05-08T05:25:22.648083+0000","last_active":"2024-05-08T05:27:05.561931+0000","last_peered":"2024-05-08T05:27:05.561931+0000","last_clean":"2024-05-08T05:27:05.561931+0000","last_became_active":"2024-05-08T05:25:22.647650+0000","last_became_peered":"2024-05-08T05:25:22.647650+0000","last_unstale":"2024-05-08T05:27:05.561931+0000","last_undegraded":"2024-05-08T05:27:05.561931+0000","last_fullsized":"2024-05-08T05:27:05.561931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:56.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764300+0000","last_change":"2024-05-08T05:25:25.668290+0000","last_active":"2024-05-08T05:27:05.764300+0000","last_peered":"2024-05-08T05:27:05.764300+0000","last_clean":"2024-05-08T05:27:05.764300+0000","last_became_active":"2024-05-08T05:25:25.667888+0000","last_became_peered":"2024-05-08T05:25:25.667888+0000","last_unstale":"2024-05-08T05:27:05.764300+0000","last_undegraded":"2024-05-08T05:27:05.764300+0000","last_fullsized":"2024-05-08T05:27:05.764300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:19.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563156+0000","last_change":"2024-05-08T05:25:37.057639+0000","last_active":"2024-05-08T05:27:05.563156+0000","last_peered":"2024-05-08T05:27:05.563156+0000","last_clean":"2024-05-08T05:27:05.563156+0000","last_became_active":"2024-05-08T05:25:37.056848+0000","last_became_peered":"2024-05-08T05:25:37.056848+0000","last_unstale":"2024-05-08T05:27:05.563156+0000","last_undegraded":"2024-05-08T05:27:05.563156+0000","last_fullsized":"2024-05-08T05:27:05.563156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:37.137224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"88'4","reported_seq":238,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:02.551995+0000","last_change":"2024-05-08T05:25:22.649658+0000","last_active":"2024-05-08T05:28:02.551995+0000","last_peered":"2024-05-08T05:28:02.551995+0000","last_clean":"2024-05-08T05:28:02.551995+0000","last_became_active":"2024-05-08T05:25:22.648771+0000","last_became_peered":"2024-05-08T05:25:22.648771+0000","last_unstale":"2024-05-08T05:28:02.551995+0000","last_undegraded":"2024-05-08T05:28:02.551995+0000","last_fullsized":"2024-05-08T05:28:02.551995+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:54.412710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":190,"num_read_kb":190,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018564+0000","last_change":"2024-05-08T05:25:25.673931+0000","last_active":"2024-05-08T05:28:04.018564+0000","last_peered":"2024-05-08T05:28:04.018564+0000","last_clean":"2024-05-08T05:28:04.018564+0000","last_became_active":"2024-05-08T05:25:25.673585+0000","last_became_peered":"2024-05-08T05:25:25.673585+0000","last_unstale":"2024-05-08T05:28:04.018564+0000","last_undegraded":"2024-05-08T05:28:04.018564+0000","last_fullsized":"2024-05-08T05:28:04.018564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:27.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764218+0000","last_change":"2024-05-08T05:25:37.047651+0000","last_active":"2024-05-08T05:27:05.764218+0000","last_peered":"2024-05-08T05:27:05.764218+0000","last_clean":"2024-05-08T05:27:05.764218+0000","last_became_active":"2024-05-08T05:25:37.047246+0000","last_became_peered":"2024-05-08T05:25:37.047246+0000","last_unstale":"2024-05-08T05:27:05.764218+0000","last_undegraded":"2024-05-08T05:27:05.764218+0000","last_fullsized":"2024-05-08T05:27:05.764218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:22.300076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561924+0000","last_change":"2024-05-08T05:25:22.646973+0000","last_active":"2024-05-08T05:27:05.561924+0000","last_peered":"2024-05-08T05:27:05.561924+0000","last_clean":"2024-05-08T05:27:05.561924+0000","last_became_active":"2024-05-08T05:25:22.646232+0000","last_became_peered":"2024-05-08T05:25:22.646232+0000","last_unstale":"2024-05-08T05:27:05.561924+0000","last_undegraded":"2024-05-08T05:27:05.561924+0000","last_fullsized":"2024-05-08T05:27:05.561924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:38.147470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.016976+0000","last_change":"2024-05-08T05:25:25.661373+0000","last_active":"2024-05-08T05:28:04.016976+0000","last_peered":"2024-05-08T05:28:04.016976+0000","last_clean":"2024-05-08T05:28:04.016976+0000","last_became_active":"2024-05-08T05:25:25.661080+0000","last_became_peered":"2024-05-08T05:25:25.661080+0000","last_unstale":"2024-05-08T05:28:04.016976+0000","last_undegraded":"2024-05-08T05:28:04.016976+0000","last_fullsized":"2024-05-08T05:28:04.016976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:18.718610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765639+0000","last_change":"2024-05-08T05:25:37.051797+0000","last_active":"2024-05-08T05:27:05.765639+0000","last_peered":"2024-05-08T05:27:05.765639+0000","last_clean":"2024-05-08T05:27:05.765639+0000","last_became_active":"2024-05-08T05:25:37.051381+0000","last_became_peered":"2024-05-08T05:25:37.051381+0000","last_unstale":"2024-05-08T05:27:05.765639+0000","last_undegraded":"2024-05-08T05:27:05.765639+0000","last_fullsized":"2024-05-08T05:27:05.765639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:34.337512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562222+0000","last_change":"2024-05-08T05:25:22.748510+0000","last_active":"2024-05-08T05:27:05.562222+0000","last_peered":"2024-05-08T05:27:05.562222+0000","last_clean":"2024-05-08T05:27:05.562222+0000","last_became_active":"2024-05-08T05:25:22.748140+0000","last_became_peered":"2024-05-08T05:25:22.748140+0000","last_unstale":"2024-05-08T05:27:05.562222+0000","last_undegraded":"2024-05-08T05:27:05.562222+0000","last_fullsized":"2024-05-08T05:27:05.562222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:32.325653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.017787+0000","last_change":"2024-05-08T05:25:25.671261+0000","last_active":"2024-05-08T05:28:04.017787+0000","last_peered":"2024-05-08T05:28:04.017787+0000","last_clean":"2024-05-08T05:28:04.017787+0000","last_became_active":"2024-05-08T05:25:25.670937+0000","last_became_peered":"2024-05-08T05:25:25.670937+0000","last_unstale":"2024-05-08T05:28:04.017787+0000","last_undegraded":"2024-05-08T05:28:04.017787+0000","last_fullsized":"2024-05-08T05:28:04.017787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:41.375908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765231+0000","last_change":"2024-05-08T05:26:05.150754+0000","last_active":"2024-05-08T05:27:05.765231+0000","last_peered":"2024-05-08T05:27:05.765231+0000","last_clean":"2024-05-08T05:27:05.765231+0000","last_became_active":"2024-05-08T05:26:05.150331+0000","last_became_peered":"2024-05-08T05:26:05.150331+0000","last_unstale":"2024-05-08T05:27:05.765231+0000","last_undegraded":"2024-05-08T05:27:05.765231+0000","last_fullsized":"2024-05-08T05:27:05.765231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:23.309103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"88'35","reported_seq":278,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:02.568463+0000","last_change":"2024-05-08T05:25:22.645627+0000","last_active":"2024-05-08T05:28:02.568463+0000","last_peered":"2024-05-08T05:28:02.568463+0000","last_clean":"2024-05-08T05:28:02.568463+0000","last_became_active":"2024-05-08T05:25:22.645243+0000","last_became_peered":"2024-05-08T05:25:22.645243+0000","last_unstale":"2024-05-08T05:28:02.568463+0000","last_undegraded":"2024-05-08T05:28:02.568463+0000","last_fullsized":"2024-05-08T05:28:02.568463+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:23.256903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64645,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":199,"num_read_kb":163,"num_write":37,"num_write_kb":78,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.002021+0000","last_change":"2024-05-08T05:25:25.661258+0000","last_active":"2024-05-08T05:28:04.002021+0000","last_peered":"2024-05-08T05:28:04.002021+0000","last_clean":"2024-05-08T05:28:04.002021+0000","last_became_active":"2024-05-08T05:25:25.660925+0000","last_became_peered":"2024-05-08T05:25:25.660925+0000","last_unstale":"2024-05-08T05:28:04.002021+0000","last_undegraded":"2024-05-08T05:28:04.002021+0000","last_fullsized":"2024-05-08T05:28:04.002021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:58.764540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562882+0000","last_change":"2024-05-08T05:25:37.053539+0000","last_active":"2024-05-08T05:27:05.562882+0000","last_peered":"2024-05-08T05:27:05.562882+0000","last_clean":"2024-05-08T05:27:05.562882+0000","last_became_active":"2024-05-08T05:25:37.053102+0000","last_became_peered":"2024-05-08T05:25:37.053102+0000","last_unstale":"2024-05-08T05:27:05.562882+0000","last_undegraded":"2024-05-08T05:27:05.562882+0000","last_fullsized":"2024-05-08T05:27:05.562882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:48.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564138+0000","last_change":"2024-05-08T05:25:22.646651+0000","last_active":"2024-05-08T05:27:05.564138+0000","last_peered":"2024-05-08T05:27:05.564138+0000","last_clean":"2024-05-08T05:27:05.564138+0000","last_became_active":"2024-05-08T05:25:22.646285+0000","last_became_peered":"2024-05-08T05:25:22.646285+0000","last_unstale":"2024-05-08T05:27:05.564138+0000","last_undegraded":"2024-05-08T05:27:05.564138+0000","last_fullsized":"2024-05-08T05:27:05.564138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.843858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020203+0000","last_change":"2024-05-08T05:25:25.666018+0000","last_active":"2024-05-08T05:28:04.020203+0000","last_peered":"2024-05-08T05:28:04.020203+0000","last_clean":"2024-05-08T05:28:04.020203+0000","last_became_active":"2024-05-08T05:25:25.663609+0000","last_became_peered":"2024-05-08T05:25:25.663609+0000","last_unstale":"2024-05-08T05:28:04.020203+0000","last_undegraded":"2024-05-08T05:28:04.020203+0000","last_fullsized":"2024-05-08T05:28:04.020203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:36.223121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564512+0000","last_change":"2024-05-08T05:25:37.052907+0000","last_active":"2024-05-08T05:27:05.564512+0000","last_peered":"2024-05-08T05:27:05.564512+0000","last_clean":"2024-05-08T05:27:05.564512+0000","last_became_active":"2024-05-08T05:25:37.052506+0000","last_became_peered":"2024-05-08T05:25:37.052506+0000","last_unstale":"2024-05-08T05:27:05.564512+0000","last_undegraded":"2024-05-08T05:27:05.564512+0000","last_fullsized":"2024-05-08T05:27:05.564512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:41.356996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562180+0000","last_change":"2024-05-08T05:25:22.646671+0000","last_active":"2024-05-08T05:27:05.562180+0000","last_peered":"2024-05-08T05:27:05.562180+0000","last_clean":"2024-05-08T05:27:05.562180+0000","last_became_active":"2024-05-08T05:25:22.646202+0000","last_became_peered":"2024-05-08T05:25:22.646202+0000","last_unstale":"2024-05-08T05:27:05.562180+0000","last_undegraded":"2024-05-08T05:27:05.562180+0000","last_fullsized":"2024-05-08T05:27:05.562180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:59.224960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.016014+0000","last_change":"2024-05-08T05:25:25.666969+0000","last_active":"2024-05-08T05:28:04.016014+0000","last_peered":"2024-05-08T05:28:04.016014+0000","last_clean":"2024-05-08T05:28:04.016014+0000","last_became_active":"2024-05-08T05:25:25.666482+0000","last_became_peered":"2024-05-08T05:25:25.666482+0000","last_unstale":"2024-05-08T05:28:04.016014+0000","last_undegraded":"2024-05-08T05:28:04.016014+0000","last_fullsized":"2024-05-08T05:28:04.016014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:07.819116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764513+0000","last_change":"2024-05-08T05:25:37.055807+0000","last_active":"2024-05-08T05:27:05.764513+0000","last_peered":"2024-05-08T05:27:05.764513+0000","last_clean":"2024-05-08T05:27:05.764513+0000","last_became_active":"2024-05-08T05:25:37.054778+0000","last_became_peered":"2024-05-08T05:25:37.054778+0000","last_unstale":"2024-05-08T05:27:05.764513+0000","last_undegraded":"2024-05-08T05:27:05.764513+0000","last_fullsized":"2024-05-08T05:27:05.764513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:33.767885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.556573+0000","last_change":"2024-05-08T05:25:22.647093+0000","last_active":"2024-05-08T05:28:04.556573+0000","last_peered":"2024-05-08T05:28:04.556573+0000","last_clean":"2024-05-08T05:28:04.556573+0000","last_became_active":"2024-05-08T05:25:22.646733+0000","last_became_peered":"2024-05-08T05:25:22.646733+0000","last_unstale":"2024-05-08T05:28:04.556573+0000","last_undegraded":"2024-05-08T05:28:04.556573+0000","last_fullsized":"2024-05-08T05:28:04.556573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:04.346568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.001297+0000","last_change":"2024-05-08T05:25:25.666565+0000","last_active":"2024-05-08T05:28:04.001297+0000","last_peered":"2024-05-08T05:28:04.001297+0000","last_clean":"2024-05-08T05:28:04.001297+0000","last_became_active":"2024-05-08T05:25:25.666138+0000","last_became_peered":"2024-05-08T05:25:25.666138+0000","last_unstale":"2024-05-08T05:28:04.001297+0000","last_undegraded":"2024-05-08T05:28:04.001297+0000","last_fullsized":"2024-05-08T05:28:04.001297+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:05.396530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561191+0000","last_change":"2024-05-08T05:25:37.060933+0000","last_active":"2024-05-08T05:27:05.561191+0000","last_peered":"2024-05-08T05:27:05.561191+0000","last_clean":"2024-05-08T05:27:05.561191+0000","last_became_active":"2024-05-08T05:25:37.060388+0000","last_became_peered":"2024-05-08T05:25:37.060388+0000","last_unstale":"2024-05-08T05:27:05.561191+0000","last_undegraded":"2024-05-08T05:27:05.561191+0000","last_fullsized":"2024-05-08T05:27:05.561191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:09.776051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.566531+0000","last_change":"2024-05-08T05:26:05.156863+0000","last_active":"2024-05-08T05:27:05.566531+0000","last_peered":"2024-05-08T05:27:05.566531+0000","last_clean":"2024-05-08T05:27:05.566531+0000","last_became_active":"2024-05-08T05:26:05.149008+0000","last_became_peered":"2024-05-08T05:26:05.149008+0000","last_unstale":"2024-05-08T05:27:05.566531+0000","last_undegraded":"2024-05-08T05:27:05.566531+0000","last_fullsized":"2024-05-08T05:27:05.566531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:56.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008664+0000","last_change":"2024-05-08T05:25:25.670415+0000","last_active":"2024-05-08T05:28:04.008664+0000","last_peered":"2024-05-08T05:28:04.008664+0000","last_clean":"2024-05-08T05:28:04.008664+0000","last_became_active":"2024-05-08T05:25:25.670095+0000","last_became_peered":"2024-05-08T05:25:25.670095+0000","last_unstale":"2024-05-08T05:28:04.008664+0000","last_undegraded":"2024-05-08T05:28:04.008664+0000","last_fullsized":"2024-05-08T05:28:04.008664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.489203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.566513+0000","last_change":"2024-05-08T05:26:05.307721+0000","last_active":"2024-05-08T05:27:05.566513+0000","last_peered":"2024-05-08T05:27:05.566513+0000","last_clean":"2024-05-08T05:27:05.566513+0000","last_became_active":"2024-05-08T05:26:05.307279+0000","last_became_peered":"2024-05-08T05:26:05.307279+0000","last_unstale":"2024-05-08T05:27:05.566513+0000","last_undegraded":"2024-05-08T05:27:05.566513+0000","last_fullsized":"2024-05-08T05:27:05.566513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:08.857770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.567101+0000","last_change":"2024-05-08T05:25:22.644279+0000","last_active":"2024-05-08T05:27:05.567101+0000","last_peered":"2024-05-08T05:27:05.567101+0000","last_clean":"2024-05-08T05:27:05.567101+0000","last_became_active":"2024-05-08T05:25:22.643900+0000","last_became_peered":"2024-05-08T05:25:22.643900+0000","last_unstale":"2024-05-08T05:27:05.567101+0000","last_undegraded":"2024-05-08T05:27:05.567101+0000","last_fullsized":"2024-05-08T05:27:05.567101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:32.435843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.006132+0000","last_change":"2024-05-08T05:25:25.670130+0000","last_active":"2024-05-08T05:28:04.006132+0000","last_peered":"2024-05-08T05:28:04.006132+0000","last_clean":"2024-05-08T05:28:04.006132+0000","last_became_active":"2024-05-08T05:25:25.669784+0000","last_became_peered":"2024-05-08T05:25:25.669784+0000","last_unstale":"2024-05-08T05:28:04.006132+0000","last_undegraded":"2024-05-08T05:28:04.006132+0000","last_fullsized":"2024-05-08T05:28:04.006132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:07.188883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.566659+0000","last_change":"2024-05-08T05:26:05.308804+0000","last_active":"2024-05-08T05:27:05.566659+0000","last_peered":"2024-05-08T05:27:05.566659+0000","last_clean":"2024-05-08T05:27:05.566659+0000","last_became_active":"2024-05-08T05:26:05.308226+0000","last_became_peered":"2024-05-08T05:26:05.308226+0000","last_unstale":"2024-05-08T05:27:05.566659+0000","last_undegraded":"2024-05-08T05:27:05.566659+0000","last_fullsized":"2024-05-08T05:27:05.566659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:33.442023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764614+0000","last_change":"2024-05-08T05:25:22.748409+0000","last_active":"2024-05-08T05:27:05.764614+0000","last_peered":"2024-05-08T05:27:05.764614+0000","last_clean":"2024-05-08T05:27:05.764614+0000","last_became_active":"2024-05-08T05:25:22.748062+0000","last_became_peered":"2024-05-08T05:25:22.748062+0000","last_unstale":"2024-05-08T05:27:05.764614+0000","last_undegraded":"2024-05-08T05:27:05.764614+0000","last_fullsized":"2024-05-08T05:27:05.764614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:42.147084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765893+0000","last_change":"2024-05-08T05:25:25.677057+0000","last_active":"2024-05-08T05:27:05.765893+0000","last_peered":"2024-05-08T05:27:05.765893+0000","last_clean":"2024-05-08T05:27:05.765893+0000","last_became_active":"2024-05-08T05:25:25.676697+0000","last_became_peered":"2024-05-08T05:25:25.676697+0000","last_unstale":"2024-05-08T05:27:05.765893+0000","last_undegraded":"2024-05-08T05:27:05.765893+0000","last_fullsized":"2024-05-08T05:27:05.765893+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:40.786327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.560833+0000","last_change":"2024-05-08T05:25:37.053903+0000","last_active":"2024-05-08T05:27:05.560833+0000","last_peered":"2024-05-08T05:27:05.560833+0000","last_clean":"2024-05-08T05:27:05.560833+0000","last_became_active":"2024-05-08T05:25:37.053375+0000","last_became_peered":"2024-05-08T05:25:37.053375+0000","last_unstale":"2024-05-08T05:27:05.560833+0000","last_undegraded":"2024-05-08T05:27:05.560833+0000","last_fullsized":"2024-05-08T05:27:05.560833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:19.374355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.555428+0000","last_change":"2024-05-08T05:25:22.646931+0000","last_active":"2024-05-08T05:28:04.555428+0000","last_peered":"2024-05-08T05:28:04.555428+0000","last_clean":"2024-05-08T05:28:04.555428+0000","last_became_active":"2024-05-08T05:25:22.646599+0000","last_became_peered":"2024-05-08T05:25:22.646599+0000","last_unstale":"2024-05-08T05:28:04.555428+0000","last_undegraded":"2024-05-08T05:28:04.555428+0000","last_fullsized":"2024-05-08T05:28:04.555428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:55.873089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765503+0000","last_change":"2024-05-08T05:26:05.147977+0000","last_active":"2024-05-08T05:27:05.765503+0000","last_peered":"2024-05-08T05:27:05.765503+0000","last_clean":"2024-05-08T05:27:05.765503+0000","last_became_active":"2024-05-08T05:26:05.147637+0000","last_became_peered":"2024-05-08T05:26:05.147637+0000","last_unstale":"2024-05-08T05:27:05.765503+0000","last_undegraded":"2024-05-08T05:27:05.765503+0000","last_fullsized":"2024-05-08T05:27:05.765503+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:59.671306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.560851+0000","last_change":"2024-05-08T05:25:37.057913+0000","last_active":"2024-05-08T05:27:05.560851+0000","last_peered":"2024-05-08T05:27:05.560851+0000","last_clean":"2024-05-08T05:27:05.560851+0000","last_became_active":"2024-05-08T05:25:37.057474+0000","last_became_peered":"2024-05-08T05:25:37.057474+0000","last_unstale":"2024-05-08T05:27:05.560851+0000","last_undegraded":"2024-05-08T05:27:05.560851+0000","last_fullsized":"2024-05-08T05:27:05.560851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:19.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008256+0000","last_change":"2024-05-08T05:25:25.663594+0000","last_active":"2024-05-08T05:28:04.008256+0000","last_peered":"2024-05-08T05:28:04.008256+0000","last_clean":"2024-05-08T05:28:04.008256+0000","last_became_active":"2024-05-08T05:25:25.663275+0000","last_became_peered":"2024-05-08T05:25:25.663275+0000","last_unstale":"2024-05-08T05:28:04.008256+0000","last_undegraded":"2024-05-08T05:28:04.008256+0000","last_fullsized":"2024-05-08T05:28:04.008256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:20.581166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:05.557262+0000","last_change":"2024-05-08T05:25:22.748316+0000","last_active":"2024-05-08T05:28:05.557262+0000","last_peered":"2024-05-08T05:28:05.557262+0000","last_clean":"2024-05-08T05:28:05.557262+0000","last_became_active":"2024-05-08T05:25:22.747819+0000","last_became_peered":"2024-05-08T05:25:22.747819+0000","last_unstale":"2024-05-08T05:28:05.557262+0000","last_undegraded":"2024-05-08T05:28:05.557262+0000","last_fullsized":"2024-05-08T05:28:05.557262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:50:42.689853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561916+0000","last_change":"2024-05-08T05:25:37.058271+0000","last_active":"2024-05-08T05:27:05.561916+0000","last_peered":"2024-05-08T05:27:05.561916+0000","last_clean":"2024-05-08T05:27:05.561916+0000","last_became_active":"2024-05-08T05:25:37.057290+0000","last_became_peered":"2024-05-08T05:25:37.057290+0000","last_unstale":"2024-05-08T05:27:05.561916+0000","last_undegraded":"2024-05-08T05:27:05.561916+0000","last_fullsized":"2024-05-08T05:27:05.561916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:47.369909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018403+0000","last_change":"2024-05-08T05:25:25.666000+0000","last_active":"2024-05-08T05:28:04.018403+0000","last_peered":"2024-05-08T05:28:04.018403+0000","last_clean":"2024-05-08T05:28:04.018403+0000","last_became_active":"2024-05-08T05:25:25.665663+0000","last_became_peered":"2024-05-08T05:25:25.665663+0000","last_unstale":"2024-05-08T05:28:04.018403+0000","last_undegraded":"2024-05-08T05:28:04.018403+0000","last_fullsized":"2024-05-08T05:28:04.018403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:44.328879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765527+0000","last_change":"2024-05-08T05:25:22.648770+0000","last_active":"2024-05-08T05:27:05.765527+0000","last_peered":"2024-05-08T05:27:05.765527+0000","last_clean":"2024-05-08T05:27:05.765527+0000","last_became_active":"2024-05-08T05:25:22.648119+0000","last_became_peered":"2024-05-08T05:25:22.648119+0000","last_unstale":"2024-05-08T05:27:05.765527+0000","last_undegraded":"2024-05-08T05:27:05.765527+0000","last_fullsized":"2024-05-08T05:27:05.765527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:55.813080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561483+0000","last_change":"2024-05-08T05:23:55.820436+0000","last_active":"2024-05-08T05:27:05.561483+0000","last_peered":"2024-05-08T05:27:05.561483+0000","last_clean":"2024-05-08T05:27:05.561483+0000","last_became_active":"2024-05-08T05:23:55.814786+0000","last_became_peered":"2024-05-08T05:23:55.814786+0000","last_unstale":"2024-05-08T05:27:05.561483+0000","last_undegraded":"2024-05-08T05:27:05.561483+0000","last_fullsized":"2024-05-08T05:27:05.561483+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_clean_scrub_stamp":"2024-05-08T05:20:32.659283+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:11.169455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562544+0000","last_change":"2024-05-08T05:25:37.057287+0000","last_active":"2024-05-08T05:27:05.562544+0000","last_peered":"2024-05-08T05:27:05.562544+0000","last_clean":"2024-05-08T05:27:05.562544+0000","last_became_active":"2024-05-08T05:25:37.056871+0000","last_became_peered":"2024-05-08T05:25:37.056871+0000","last_unstale":"2024-05-08T05:27:05.562544+0000","last_undegraded":"2024-05-08T05:27:05.562544+0000","last_fullsized":"2024-05-08T05:27:05.562544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:19.004199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008120+0000","last_change":"2024-05-08T05:25:25.664795+0000","last_active":"2024-05-08T05:28:04.008120+0000","last_peered":"2024-05-08T05:28:04.008120+0000","last_clean":"2024-05-08T05:28:04.008120+0000","last_became_active":"2024-05-08T05:25:25.664444+0000","last_became_peered":"2024-05-08T05:25:25.664444+0000","last_unstale":"2024-05-08T05:28:04.008120+0000","last_undegraded":"2024-05-08T05:28:04.008120+0000","last_fullsized":"2024-05-08T05:28:04.008120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:27.172530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561853+0000","last_change":"2024-05-08T05:25:22.645205+0000","last_active":"2024-05-08T05:27:05.561853+0000","last_peered":"2024-05-08T05:27:05.561853+0000","last_clean":"2024-05-08T05:27:05.561853+0000","last_became_active":"2024-05-08T05:25:22.644720+0000","last_became_peered":"2024-05-08T05:25:22.644720+0000","last_unstale":"2024-05-08T05:27:05.561853+0000","last_undegraded":"2024-05-08T05:27:05.561853+0000","last_fullsized":"2024-05-08T05:27:05.561853+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:05.630188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564454+0000","last_change":"2024-05-08T05:25:37.055151+0000","last_active":"2024-05-08T05:27:05.564454+0000","last_peered":"2024-05-08T05:27:05.564454+0000","last_clean":"2024-05-08T05:27:05.564454+0000","last_became_active":"2024-05-08T05:25:37.054737+0000","last_became_peered":"2024-05-08T05:25:37.054737+0000","last_unstale":"2024-05-08T05:27:05.564454+0000","last_undegraded":"2024-05-08T05:27:05.564454+0000","last_fullsized":"2024-05-08T05:27:05.564454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:49.135453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564959+0000","last_change":"2024-05-08T05:26:05.148665+0000","last_active":"2024-05-08T05:27:05.564959+0000","last_peered":"2024-05-08T05:27:05.564959+0000","last_clean":"2024-05-08T05:27:05.564959+0000","last_became_active":"2024-05-08T05:26:05.148283+0000","last_became_peered":"2024-05-08T05:26:05.148283+0000","last_unstale":"2024-05-08T05:27:05.564959+0000","last_undegraded":"2024-05-08T05:27:05.564959+0000","last_fullsized":"2024-05-08T05:27:05.564959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:26.197521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:03.999698+0000","last_change":"2024-05-08T05:25:25.669130+0000","last_active":"2024-05-08T05:28:03.999698+0000","last_peered":"2024-05-08T05:28:03.999698+0000","last_clean":"2024-05-08T05:28:03.999698+0000","last_became_active":"2024-05-08T05:25:25.668812+0000","last_became_peered":"2024-05-08T05:25:25.668812+0000","last_unstale":"2024-05-08T05:28:03.999698+0000","last_undegraded":"2024-05-08T05:28:03.999698+0000","last_fullsized":"2024-05-08T05:28:03.999698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:34.514248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562972+0000","last_change":"2024-05-08T05:25:37.057146+0000","last_active":"2024-05-08T05:27:05.562972+0000","last_peered":"2024-05-08T05:27:05.562972+0000","last_clean":"2024-05-08T05:27:05.562972+0000","last_became_active":"2024-05-08T05:25:37.056706+0000","last_became_peered":"2024-05-08T05:25:37.056706+0000","last_unstale":"2024-05-08T05:27:05.562972+0000","last_undegraded":"2024-05-08T05:27:05.562972+0000","last_fullsized":"2024-05-08T05:27:05.562972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:09.724088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"88'15","reported_seq":59,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764593+0000","last_change":"2024-05-08T05:25:22.644573+0000","last_active":"2024-05-08T05:27:05.764593+0000","last_peered":"2024-05-08T05:27:05.764593+0000","last_clean":"2024-05-08T05:27:05.764593+0000","last_became_active":"2024-05-08T05:25:22.644231+0000","last_became_peered":"2024-05-08T05:25:22.644231+0000","last_unstale":"2024-05-08T05:27:05.764593+0000","last_undegraded":"2024-05-08T05:27:05.764593+0000","last_fullsized":"2024-05-08T05:27:05.764593+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:02.931823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":27,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.021549+0000","last_change":"2024-05-08T05:25:25.671036+0000","last_active":"2024-05-08T05:28:04.021549+0000","last_peered":"2024-05-08T05:28:04.021549+0000","last_clean":"2024-05-08T05:28:04.021549+0000","last_became_active":"2024-05-08T05:25:25.669170+0000","last_became_peered":"2024-05-08T05:25:25.669170+0000","last_unstale":"2024-05-08T05:28:04.021549+0000","last_undegraded":"2024-05-08T05:28:04.021549+0000","last_fullsized":"2024-05-08T05:28:04.021549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:18.963232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563674+0000","last_change":"2024-05-08T05:25:37.050656+0000","last_active":"2024-05-08T05:27:05.563674+0000","last_peered":"2024-05-08T05:27:05.563674+0000","last_clean":"2024-05-08T05:27:05.563674+0000","last_became_active":"2024-05-08T05:25:37.050051+0000","last_became_peered":"2024-05-08T05:25:37.050051+0000","last_unstale":"2024-05-08T05:27:05.563674+0000","last_undegraded":"2024-05-08T05:27:05.563674+0000","last_fullsized":"2024-05-08T05:27:05.563674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:56:07.316216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765551+0000","last_change":"2024-05-08T05:25:23.614874+0000","last_active":"2024-05-08T05:27:05.765551+0000","last_peered":"2024-05-08T05:27:05.765551+0000","last_clean":"2024-05-08T05:27:05.765551+0000","last_became_active":"2024-05-08T05:25:23.614550+0000","last_became_peered":"2024-05-08T05:25:23.614550+0000","last_unstale":"2024-05-08T05:27:05.765551+0000","last_undegraded":"2024-05-08T05:27:05.765551+0000","last_fullsized":"2024-05-08T05:27:05.765551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:10.183477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020276+0000","last_change":"2024-05-08T05:25:25.658710+0000","last_active":"2024-05-08T05:28:04.020276+0000","last_peered":"2024-05-08T05:28:04.020276+0000","last_clean":"2024-05-08T05:28:04.020276+0000","last_became_active":"2024-05-08T05:25:25.658393+0000","last_became_peered":"2024-05-08T05:25:25.658393+0000","last_unstale":"2024-05-08T05:28:04.020276+0000","last_undegraded":"2024-05-08T05:28:04.020276+0000","last_fullsized":"2024-05-08T05:28:04.020276+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:02.629256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765630+0000","last_change":"2024-05-08T05:25:37.059976+0000","last_active":"2024-05-08T05:27:05.765630+0000","last_peered":"2024-05-08T05:27:05.765630+0000","last_clean":"2024-05-08T05:27:05.765630+0000","last_became_active":"2024-05-08T05:25:37.059474+0000","last_became_peered":"2024-05-08T05:25:37.059474+0000","last_unstale":"2024-05-08T05:27:05.765630+0000","last_undegraded":"2024-05-08T05:27:05.765630+0000","last_fullsized":"2024-05-08T05:27:05.765630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:51.946242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764629+0000","last_change":"2024-05-08T05:25:22.748271+0000","last_active":"2024-05-08T05:27:05.764629+0000","last_peered":"2024-05-08T05:27:05.764629+0000","last_clean":"2024-05-08T05:27:05.764629+0000","last_became_active":"2024-05-08T05:25:22.747915+0000","last_became_peered":"2024-05-08T05:25:22.747915+0000","last_unstale":"2024-05-08T05:27:05.764629+0000","last_undegraded":"2024-05-08T05:27:05.764629+0000","last_fullsized":"2024-05-08T05:27:05.764629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:35.262298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:03.972752+0000","last_change":"2024-05-08T05:25:25.668722+0000","last_active":"2024-05-08T05:28:03.972752+0000","last_peered":"2024-05-08T05:28:03.972752+0000","last_clean":"2024-05-08T05:28:03.972752+0000","last_became_active":"2024-05-08T05:25:25.668223+0000","last_became_peered":"2024-05-08T05:25:25.668223+0000","last_unstale":"2024-05-08T05:28:03.972752+0000","last_undegraded":"2024-05-08T05:28:03.972752+0000","last_fullsized":"2024-05-08T05:28:03.972752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:43.511567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764581+0000","last_change":"2024-05-08T05:25:37.056131+0000","last_active":"2024-05-08T05:27:05.764581+0000","last_peered":"2024-05-08T05:27:05.764581+0000","last_clean":"2024-05-08T05:27:05.764581+0000","last_became_active":"2024-05-08T05:25:37.055424+0000","last_became_peered":"2024-05-08T05:25:37.055424+0000","last_unstale":"2024-05-08T05:27:05.764581+0000","last_undegraded":"2024-05-08T05:27:05.764581+0000","last_fullsized":"2024-05-08T05:27:05.764581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:43.279408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563203+0000","last_change":"2024-05-08T05:25:22.643908+0000","last_active":"2024-05-08T05:27:05.563203+0000","last_peered":"2024-05-08T05:27:05.563203+0000","last_clean":"2024-05-08T05:27:05.563203+0000","last_became_active":"2024-05-08T05:25:22.643542+0000","last_became_peered":"2024-05-08T05:25:22.643542+0000","last_unstale":"2024-05-08T05:27:05.563203+0000","last_undegraded":"2024-05-08T05:27:05.563203+0000","last_fullsized":"2024-05-08T05:27:05.563203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:01.671285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.014996+0000","last_change":"2024-05-08T05:25:25.667843+0000","last_active":"2024-05-08T05:28:04.014996+0000","last_peered":"2024-05-08T05:28:04.014996+0000","last_clean":"2024-05-08T05:28:04.014996+0000","last_became_active":"2024-05-08T05:25:25.667462+0000","last_became_peered":"2024-05-08T05:25:25.667462+0000","last_unstale":"2024-05-08T05:28:04.014996+0000","last_undegraded":"2024-05-08T05:28:04.014996+0000","last_fullsized":"2024-05-08T05:28:04.014996+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:51.325104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764958+0000","last_change":"2024-05-08T05:25:37.053017+0000","last_active":"2024-05-08T05:27:05.764958+0000","last_peered":"2024-05-08T05:27:05.764958+0000","last_clean":"2024-05-08T05:27:05.764958+0000","last_became_active":"2024-05-08T05:25:37.052601+0000","last_became_peered":"2024-05-08T05:25:37.052601+0000","last_unstale":"2024-05-08T05:27:05.764958+0000","last_undegraded":"2024-05-08T05:27:05.764958+0000","last_fullsized":"2024-05-08T05:27:05.764958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:35.393835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563935+0000","last_change":"2024-05-08T05:25:22.644564+0000","last_active":"2024-05-08T05:27:05.563935+0000","last_peered":"2024-05-08T05:27:05.563935+0000","last_clean":"2024-05-08T05:27:05.563935+0000","last_became_active":"2024-05-08T05:25:22.644170+0000","last_became_peered":"2024-05-08T05:25:22.644170+0000","last_unstale":"2024-05-08T05:27:05.563935+0000","last_undegraded":"2024-05-08T05:27:05.563935+0000","last_fullsized":"2024-05-08T05:27:05.563935+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:24.757574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.021250+0000","last_change":"2024-05-08T05:25:25.676983+0000","last_active":"2024-05-08T05:28:04.021250+0000","last_peered":"2024-05-08T05:28:04.021250+0000","last_clean":"2024-05-08T05:28:04.021250+0000","last_became_active":"2024-05-08T05:25:25.676610+0000","last_became_peered":"2024-05-08T05:25:25.676610+0000","last_unstale":"2024-05-08T05:28:04.021250+0000","last_undegraded":"2024-05-08T05:28:04.021250+0000","last_fullsized":"2024-05-08T05:28:04.021250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:39.137935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765483+0000","last_change":"2024-05-08T05:25:37.059381+0000","last_active":"2024-05-08T05:27:05.765483+0000","last_peered":"2024-05-08T05:27:05.765483+0000","last_clean":"2024-05-08T05:27:05.765483+0000","last_became_active":"2024-05-08T05:25:37.056851+0000","last_became_peered":"2024-05-08T05:25:37.056851+0000","last_unstale":"2024-05-08T05:27:05.765483+0000","last_undegraded":"2024-05-08T05:27:05.765483+0000","last_fullsized":"2024-05-08T05:27:05.765483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:19.339331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.566848+0000","last_change":"2024-05-08T05:25:22.648099+0000","last_active":"2024-05-08T05:27:05.566848+0000","last_peered":"2024-05-08T05:27:05.566848+0000","last_clean":"2024-05-08T05:27:05.566848+0000","last_became_active":"2024-05-08T05:25:22.647736+0000","last_became_peered":"2024-05-08T05:25:22.647736+0000","last_unstale":"2024-05-08T05:27:05.566848+0000","last_undegraded":"2024-05-08T05:27:05.566848+0000","last_fullsized":"2024-05-08T05:27:05.566848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:12.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.005434+0000","last_change":"2024-05-08T05:25:25.667565+0000","last_active":"2024-05-08T05:28:04.005434+0000","last_peered":"2024-05-08T05:28:04.005434+0000","last_clean":"2024-05-08T05:28:04.005434+0000","last_became_active":"2024-05-08T05:25:25.667217+0000","last_became_peered":"2024-05-08T05:25:25.667217+0000","last_unstale":"2024-05-08T05:28:04.005434+0000","last_undegraded":"2024-05-08T05:28:04.005434+0000","last_fullsized":"2024-05-08T05:28:04.005434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:05.228227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764997+0000","last_change":"2024-05-08T05:25:37.056923+0000","last_active":"2024-05-08T05:27:05.764997+0000","last_peered":"2024-05-08T05:27:05.764997+0000","last_clean":"2024-05-08T05:27:05.764997+0000","last_became_active":"2024-05-08T05:25:37.056496+0000","last_became_peered":"2024-05-08T05:25:37.056496+0000","last_unstale":"2024-05-08T05:27:05.764997+0000","last_undegraded":"2024-05-08T05:27:05.764997+0000","last_fullsized":"2024-05-08T05:27:05.764997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:21.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.555739+0000","last_change":"2024-05-08T05:25:23.614900+0000","last_active":"2024-05-08T05:28:04.555739+0000","last_peered":"2024-05-08T05:28:04.555739+0000","last_clean":"2024-05-08T05:28:04.555739+0000","last_became_active":"2024-05-08T05:25:23.614510+0000","last_became_peered":"2024-05-08T05:25:23.614510+0000","last_unstale":"2024-05-08T05:28:04.555739+0000","last_undegraded":"2024-05-08T05:28:04.555739+0000","last_fullsized":"2024-05-08T05:28:04.555739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:19.539656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.007662+0000","last_change":"2024-05-08T05:25:25.669016+0000","last_active":"2024-05-08T05:28:04.007662+0000","last_peered":"2024-05-08T05:28:04.007662+0000","last_clean":"2024-05-08T05:28:04.007662+0000","last_became_active":"2024-05-08T05:25:25.667809+0000","last_became_peered":"2024-05-08T05:25:25.667809+0000","last_unstale":"2024-05-08T05:28:04.007662+0000","last_undegraded":"2024-05-08T05:28:04.007662+0000","last_fullsized":"2024-05-08T05:28:04.007662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:56.475622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.723728+0000","last_change":"2024-05-08T05:25:37.053630+0000","last_active":"2024-05-08T05:27:05.723728+0000","last_peered":"2024-05-08T05:27:05.723728+0000","last_clean":"2024-05-08T05:27:05.723728+0000","last_became_active":"2024-05-08T05:25:37.053069+0000","last_became_peered":"2024-05-08T05:25:37.053069+0000","last_unstale":"2024-05-08T05:27:05.723728+0000","last_undegraded":"2024-05-08T05:27:05.723728+0000","last_fullsized":"2024-05-08T05:27:05.723728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:49:56.693447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562317+0000","last_change":"2024-05-08T05:25:22.645284+0000","last_active":"2024-05-08T05:27:05.562317+0000","last_peered":"2024-05-08T05:27:05.562317+0000","last_clean":"2024-05-08T05:27:05.562317+0000","last_became_active":"2024-05-08T05:25:22.644897+0000","last_became_peered":"2024-05-08T05:25:22.644897+0000","last_unstale":"2024-05-08T05:27:05.562317+0000","last_undegraded":"2024-05-08T05:27:05.562317+0000","last_fullsized":"2024-05-08T05:27:05.562317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:53:31.423412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.000490+0000","last_change":"2024-05-08T05:25:25.664930+0000","last_active":"2024-05-08T05:28:04.000490+0000","last_peered":"2024-05-08T05:28:04.000490+0000","last_clean":"2024-05-08T05:28:04.000490+0000","last_became_active":"2024-05-08T05:25:25.664395+0000","last_became_peered":"2024-05-08T05:25:25.664395+0000","last_unstale":"2024-05-08T05:28:04.000490+0000","last_undegraded":"2024-05-08T05:28:04.000490+0000","last_fullsized":"2024-05-08T05:28:04.000490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:37.971474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561719+0000","last_change":"2024-05-08T05:25:37.053258+0000","last_active":"2024-05-08T05:27:05.561719+0000","last_peered":"2024-05-08T05:27:05.561719+0000","last_clean":"2024-05-08T05:27:05.561719+0000","last_became_active":"2024-05-08T05:25:37.052786+0000","last_became_peered":"2024-05-08T05:25:37.052786+0000","last_unstale":"2024-05-08T05:27:05.561719+0000","last_undegraded":"2024-05-08T05:27:05.561719+0000","last_fullsized":"2024-05-08T05:27:05.561719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:07.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.770795+0000","last_change":"2024-05-08T05:27:05.770358+0000","last_active":"2024-05-08T05:27:05.770795+0000","last_peered":"2024-05-08T05:27:05.770795+0000","last_clean":"2024-05-08T05:27:05.770795+0000","last_became_active":"2024-05-08T05:27:05.766778+0000","last_became_peered":"2024-05-08T05:27:05.766778+0000","last_unstale":"2024-05-08T05:27:05.770795+0000","last_undegraded":"2024-05-08T05:27:05.770795+0000","last_fullsized":"2024-05-08T05:27:05.770795+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:48:52.941549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.004619+0000","last_change":"2024-05-08T05:25:25.663771+0000","last_active":"2024-05-08T05:28:04.004619+0000","last_peered":"2024-05-08T05:28:04.004619+0000","last_clean":"2024-05-08T05:28:04.004619+0000","last_became_active":"2024-05-08T05:25:25.663281+0000","last_became_peered":"2024-05-08T05:25:25.663281+0000","last_unstale":"2024-05-08T05:28:04.004619+0000","last_undegraded":"2024-05-08T05:28:04.004619+0000","last_fullsized":"2024-05-08T05:28:04.004619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:43.892781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.005736+0000","last_change":"2024-05-08T05:25:25.665446+0000","last_active":"2024-05-08T05:28:04.005736+0000","last_peered":"2024-05-08T05:28:04.005736+0000","last_clean":"2024-05-08T05:28:04.005736+0000","last_became_active":"2024-05-08T05:25:25.664389+0000","last_became_peered":"2024-05-08T05:25:25.664389+0000","last_unstale":"2024-05-08T05:28:04.005736+0000","last_undegraded":"2024-05-08T05:28:04.005736+0000","last_fullsized":"2024-05-08T05:28:04.005736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.010957+0000","last_change":"2024-05-08T05:26:05.149870+0000","last_active":"2024-05-08T05:28:04.010957+0000","last_peered":"2024-05-08T05:28:04.010957+0000","last_clean":"2024-05-08T05:28:04.010957+0000","last_became_active":"2024-05-08T05:26:05.149476+0000","last_became_peered":"2024-05-08T05:26:05.149476+0000","last_unstale":"2024-05-08T05:28:04.010957+0000","last_undegraded":"2024-05-08T05:28:04.010957+0000","last_fullsized":"2024-05-08T05:28:04.010957+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:40:43.401832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.013483+0000","last_change":"2024-05-08T05:25:25.667836+0000","last_active":"2024-05-08T05:28:04.013483+0000","last_peered":"2024-05-08T05:28:04.013483+0000","last_clean":"2024-05-08T05:28:04.013483+0000","last_became_active":"2024-05-08T05:25:25.667496+0000","last_became_peered":"2024-05-08T05:25:25.667496+0000","last_unstale":"2024-05-08T05:28:04.013483+0000","last_undegraded":"2024-05-08T05:28:04.013483+0000","last_fullsized":"2024-05-08T05:28:04.013483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:01.482837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.011360+0000","last_change":"2024-05-08T05:25:25.667279+0000","last_active":"2024-05-08T05:28:04.011360+0000","last_peered":"2024-05-08T05:28:04.011360+0000","last_clean":"2024-05-08T05:28:04.011360+0000","last_became_active":"2024-05-08T05:25:25.666960+0000","last_became_peered":"2024-05-08T05:25:25.666960+0000","last_unstale":"2024-05-08T05:28:04.011360+0000","last_undegraded":"2024-05-08T05:28:04.011360+0000","last_fullsized":"2024-05-08T05:28:04.011360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:06.335850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.021057+0000","last_change":"2024-05-08T05:25:25.667644+0000","last_active":"2024-05-08T05:28:04.021057+0000","last_peered":"2024-05-08T05:28:04.021057+0000","last_clean":"2024-05-08T05:28:04.021057+0000","last_became_active":"2024-05-08T05:25:25.667133+0000","last_became_peered":"2024-05-08T05:25:25.667133+0000","last_unstale":"2024-05-08T05:28:04.021057+0000","last_undegraded":"2024-05-08T05:28:04.021057+0000","last_fullsized":"2024-05-08T05:28:04.021057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:03.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.004619+0000","last_change":"2024-05-08T05:25:25.670761+0000","last_active":"2024-05-08T05:28:04.004619+0000","last_peered":"2024-05-08T05:28:04.004619+0000","last_clean":"2024-05-08T05:28:04.004619+0000","last_became_active":"2024-05-08T05:25:25.670342+0000","last_became_peered":"2024-05-08T05:25:25.670342+0000","last_unstale":"2024-05-08T05:28:04.004619+0000","last_undegraded":"2024-05-08T05:28:04.004619+0000","last_fullsized":"2024-05-08T05:28:04.004619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:08.069332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.006746+0000","last_change":"2024-05-08T05:25:25.664512+0000","last_active":"2024-05-08T05:28:04.006746+0000","last_peered":"2024-05-08T05:28:04.006746+0000","last_clean":"2024-05-08T05:28:04.006746+0000","last_became_active":"2024-05-08T05:25:25.664191+0000","last_became_peered":"2024-05-08T05:25:25.664191+0000","last_unstale":"2024-05-08T05:28:04.006746+0000","last_undegraded":"2024-05-08T05:28:04.006746+0000","last_fullsized":"2024-05-08T05:28:04.006746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:07.586637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020017+0000","last_change":"2024-05-08T05:25:25.675711+0000","last_active":"2024-05-08T05:28:04.020017+0000","last_peered":"2024-05-08T05:28:04.020017+0000","last_clean":"2024-05-08T05:28:04.020017+0000","last_became_active":"2024-05-08T05:25:25.671890+0000","last_became_peered":"2024-05-08T05:25:25.671890+0000","last_unstale":"2024-05-08T05:28:04.020017+0000","last_undegraded":"2024-05-08T05:28:04.020017+0000","last_fullsized":"2024-05-08T05:28:04.020017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:57:49.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008300+0000","last_change":"2024-05-08T05:25:25.662603+0000","last_active":"2024-05-08T05:28:04.008300+0000","last_peered":"2024-05-08T05:28:04.008300+0000","last_clean":"2024-05-08T05:28:04.008300+0000","last_became_active":"2024-05-08T05:25:25.662287+0000","last_became_peered":"2024-05-08T05:25:25.662287+0000","last_unstale":"2024-05-08T05:28:04.008300+0000","last_undegraded":"2024-05-08T05:28:04.008300+0000","last_fullsized":"2024-05-08T05:28:04.008300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:02.152557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020684+0000","last_change":"2024-05-08T05:25:25.667309+0000","last_active":"2024-05-08T05:28:04.020684+0000","last_peered":"2024-05-08T05:28:04.020684+0000","last_clean":"2024-05-08T05:28:04.020684+0000","last_became_active":"2024-05-08T05:25:25.666968+0000","last_became_peered":"2024-05-08T05:25:25.666968+0000","last_unstale":"2024-05-08T05:28:04.020684+0000","last_undegraded":"2024-05-08T05:28:04.020684+0000","last_fullsized":"2024-05-08T05:28:04.020684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:28.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020720+0000","last_change":"2024-05-08T05:25:25.673579+0000","last_active":"2024-05-08T05:28:04.020720+0000","last_peered":"2024-05-08T05:28:04.020720+0000","last_clean":"2024-05-08T05:28:04.020720+0000","last_became_active":"2024-05-08T05:25:25.673130+0000","last_became_peered":"2024-05-08T05:25:25.673130+0000","last_unstale":"2024-05-08T05:28:04.020720+0000","last_undegraded":"2024-05-08T05:28:04.020720+0000","last_fullsized":"2024-05-08T05:28:04.020720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:01.291319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020138+0000","last_change":"2024-05-08T05:25:25.667359+0000","last_active":"2024-05-08T05:28:04.020138+0000","last_peered":"2024-05-08T05:28:04.020138+0000","last_clean":"2024-05-08T05:28:04.020138+0000","last_became_active":"2024-05-08T05:25:25.667044+0000","last_became_peered":"2024-05-08T05:25:25.667044+0000","last_unstale":"2024-05-08T05:28:04.020138+0000","last_undegraded":"2024-05-08T05:28:04.020138+0000","last_fullsized":"2024-05-08T05:28:04.020138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:54.973053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008877+0000","last_change":"2024-05-08T05:26:05.150407+0000","last_active":"2024-05-08T05:28:04.008877+0000","last_peered":"2024-05-08T05:28:04.008877+0000","last_clean":"2024-05-08T05:28:04.008877+0000","last_became_active":"2024-05-08T05:26:05.149916+0000","last_became_peered":"2024-05-08T05:26:05.149916+0000","last_unstale":"2024-05-08T05:28:04.008877+0000","last_undegraded":"2024-05-08T05:28:04.008877+0000","last_fullsized":"2024-05-08T05:28:04.008877+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:05:57.075928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":446,"ondisk_log_size":446,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247352,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73195,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1011,"num_read_kb":584,"num_write":256,"num_write_kb":151,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":219942,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678064,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":521664,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874293,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28148,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717004,"statfs":{"total":95995035648,"available":95966212096,"internally_reserved":0,"allocated":1150976,"data_stored":1172872,"data_compressed":68262,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037819,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28028,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717124,"statfs":{"total":95995035648,"available":95966334976,"internally_reserved":0,"allocated":1028096,"data_stored":525046,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234049,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":521216,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462980,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1119030,"data_compressed":68262,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626507,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":516986,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":582910,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":518442,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":583550,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182489,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32172,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712980,"statfs":{"total":95995035648,"available":95962091520,"internally_reserved":0,"allocated":1011712,"data_stored":516162,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986261,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":521172,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822735,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1113548,"data_compressed":68262,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":66772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:28:08.971 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:28:09.226 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:09.684 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:09 smithi059 ceph-mon[30304]: pgmap v382: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-08T05:28:09.684 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:09 smithi059 ceph-mon[30304]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:28:09.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:09 smithi143 ceph-mon[34337]: pgmap v382: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-08T05:28:09.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:09 smithi143 ceph-mon[34337]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:28:09.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:09 smithi096 ceph-mon[34253]: pgmap v382: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-05-08T05:28:09.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:09 smithi096 ceph-mon[34253]: from='client.14979 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:28:11.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3985507851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8103780 msgr2=0x7fe3d8103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:11.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3985507851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8103780 0x7fe3d8103b60 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fe3c80099c0 tx=0x7fe3c8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3985507851 shutdown_connections 2024-05-08T05:28:11.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3985507851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d8108870 0x7fe3d81103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3985507851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8103780 0x7fe3d8103b60 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3985507851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3d8104130 0x7fe3d8108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3985507851 >> 172.21.15.59:0/3985507851 conn(0x7fe3d807a610 msgr2=0x7fe3d807aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:11.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3985507851 shutdown_connections 2024-05-08T05:28:11.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3985507851 wait complete. 2024-05-08T05:28:11.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.006+0000 7fe3dd9ba700 1 Processor -- start 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3dd9ba700 1 -- start start 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3dd9ba700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d8103780 0x7fe3d819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3dd9ba700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3d8104130 0x7fe3d819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3dd9ba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8108870 0x7fe3d81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3dd9ba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d81136c0 con 0x7fe3d8108870 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3dd9ba700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8113540 con 0x7fe3d8104130 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3dd9ba700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe3d8113840 con 0x7fe3d8103780 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8108870 0x7fe3d81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8108870 0x7fe3d81999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39278/0 (socket says 172.21.15.59:39278) 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d77fe700 1 -- 172.21.15.59:0/3385405388 learned_addr learned my addr 172.21.15.59:0/3385405388 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:11.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d6ffd700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d8103780 0x7fe3d819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d67fc700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3d8104130 0x7fe3d819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d77fe700 1 -- 172.21.15.59:0/3385405388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d8103780 msgr2=0x7fe3d819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d77fe700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d8103780 0x7fe3d819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d77fe700 1 -- 172.21.15.59:0/3385405388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3d8104130 msgr2=0x7fe3d819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d77fe700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3d8104130 0x7fe3d819fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.007+0000 7fe3d77fe700 1 -- 172.21.15.59:0/3385405388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3d819a100 con 0x7fe3d8108870 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.008+0000 7fe3d77fe700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8108870 0x7fe3d81999a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000b970 tx=0x7fe3d000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.008+0000 7fe3dc9b8700 1 -- 172.21.15.59:0/3385405388 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d000d860 con 0x7fe3d8108870 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.008+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3d819a3f0 con 0x7fe3d8108870 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.008+0000 7fe3dc9b8700 1 -- 172.21.15.59:0/3385405388 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe3d000e430 con 0x7fe3d8108870 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.008+0000 7fe3dc9b8700 1 -- 172.21.15.59:0/3385405388 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe3d0017660 con 0x7fe3d8108870 2024-05-08T05:28:11.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.008+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3d81a6660 con 0x7fe3d8108870 2024-05-08T05:28:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.009+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3d810bab0 con 0x7fe3d8108870 2024-05-08T05:28:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.010+0000 7fe3dc9b8700 1 -- 172.21.15.59:0/3385405388 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe3d000da00 con 0x7fe3d8108870 2024-05-08T05:28:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.011+0000 7fe3dc9b8700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3c00b2880 0x7fe3c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.011+0000 7fe3d6ffd700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3c00b2880 0x7fe3c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:11.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.011+0000 7fe3dc9b8700 1 -- 172.21.15.59:0/3385405388 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe3d00e2220 con 0x7fe3d8108870 2024-05-08T05:28:11.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.011+0000 7fe3d6ffd700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3c00b2880 0x7fe3c00b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8108320 tx=0x7fe3c800b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:11.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.015+0000 7fe3dc9b8700 1 -- 172.21.15.59:0/3385405388 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fe3d0013030 con 0x7fe3d8108870 2024-05-08T05:28:11.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.223+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe3d80638e0 con 0x7fe3c00b2880 2024-05-08T05:28:11.232 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.232+0000 7fe3dc9b8700 1 -- 172.21.15.59:0/3385405388 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340931 (secure 0 0 0) 0x7fe3d80638e0 con 0x7fe3c00b2880 2024-05-08T05:28:11.237 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3c00b2880 msgr2=0x7fe3c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3c00b2880 0x7fe3c00b4d40 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe3d8108320 tx=0x7fe3c800b870 comp rx=0 tx=0).stop 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8108870 msgr2=0x7fe3d81999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8108870 0x7fe3d81999a0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fe3d000b970 tx=0x7fe3d000bc80 comp rx=0 tx=0).stop 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 shutdown_connections 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe3d8108870 0x7fe3d81999a0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe3d8103780 0x7fe3d819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe3c00b2880 0x7fe3c00b4d40 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 --2- 172.21.15.59:0/3385405388 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe3d8104130 0x7fe3d819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 >> 172.21.15.59:0/3385405388 conn(0x7fe3d807a610 msgr2=0x7fe3d8101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 shutdown_connections 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:11.236+0000 7fe3dd9ba700 1 -- 172.21.15.59:0/3385405388 wait complete. 2024-05-08T05:28:11.240 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T05:28:11.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:11 smithi059 ceph-mon[30304]: pgmap v383: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:11.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:11 smithi143 ceph-mon[34337]: pgmap v383: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:11.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:11 smithi096 ceph-mon[34253]: pgmap v383: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:11.918 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":383,"stamp":"2024-05-08T05:28:10.023902+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":664046,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":1118,"num_read_kb":674,"num_write":389,"num_write_kb":2309,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951840,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":822,"ondisk_log_size":822,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361412,"kb_used_data":12260,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580412,"statfs":{"total":1151940427776,"available":1151570341888,"internally_reserved":0,"allocated":12554240,"data_stored":8212988,"data_compressed":496944,"data_compressed_allocated":675840,"data_compressed_original":2850816,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":813,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":130,"num_read_kb":75,"num_write":13,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003279"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018215+0000","last_change":"2024-05-08T05:25:25.667895+0000","last_active":"2024-05-08T05:28:04.018215+0000","last_peered":"2024-05-08T05:28:04.018215+0000","last_clean":"2024-05-08T05:28:04.018215+0000","last_became_active":"2024-05-08T05:25:25.667563+0000","last_became_peered":"2024-05-08T05:25:25.667563+0000","last_unstale":"2024-05-08T05:28:04.018215+0000","last_undegraded":"2024-05-08T05:28:04.018215+0000","last_fullsized":"2024-05-08T05:28:04.018215+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:47.677082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018936+0000","last_change":"2024-05-08T05:25:25.673346+0000","last_active":"2024-05-08T05:28:04.018936+0000","last_peered":"2024-05-08T05:28:04.018936+0000","last_clean":"2024-05-08T05:28:04.018936+0000","last_became_active":"2024-05-08T05:25:25.673011+0000","last_became_peered":"2024-05-08T05:25:25.673011+0000","last_unstale":"2024-05-08T05:28:04.018936+0000","last_undegraded":"2024-05-08T05:28:04.018936+0000","last_fullsized":"2024-05-08T05:28:04.018936+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:21:05.091177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.013105+0000","last_change":"2024-05-08T05:25:25.670469+0000","last_active":"2024-05-08T05:28:04.013105+0000","last_peered":"2024-05-08T05:28:04.013105+0000","last_clean":"2024-05-08T05:28:04.013105+0000","last_became_active":"2024-05-08T05:25:25.669700+0000","last_became_peered":"2024-05-08T05:25:25.669700+0000","last_unstale":"2024-05-08T05:28:04.013105+0000","last_undegraded":"2024-05-08T05:28:04.013105+0000","last_fullsized":"2024-05-08T05:28:04.013105+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:06.046633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.009536+0000","last_change":"2024-05-08T05:25:25.659937+0000","last_active":"2024-05-08T05:28:04.009536+0000","last_peered":"2024-05-08T05:28:04.009536+0000","last_clean":"2024-05-08T05:28:04.009536+0000","last_became_active":"2024-05-08T05:25:25.658892+0000","last_became_peered":"2024-05-08T05:25:25.658892+0000","last_unstale":"2024-05-08T05:28:04.009536+0000","last_undegraded":"2024-05-08T05:28:04.009536+0000","last_fullsized":"2024-05-08T05:28:04.009536+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:10.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.006988+0000","last_change":"2024-05-08T05:25:25.669132+0000","last_active":"2024-05-08T05:28:04.006988+0000","last_peered":"2024-05-08T05:28:04.006988+0000","last_clean":"2024-05-08T05:28:04.006988+0000","last_became_active":"2024-05-08T05:25:25.668556+0000","last_became_peered":"2024-05-08T05:25:25.668556+0000","last_unstale":"2024-05-08T05:28:04.006988+0000","last_undegraded":"2024-05-08T05:28:04.006988+0000","last_fullsized":"2024-05-08T05:28:04.006988+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:28.443763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.009901+0000","last_change":"2024-05-08T05:25:25.668978+0000","last_active":"2024-05-08T05:28:04.009901+0000","last_peered":"2024-05-08T05:28:04.009901+0000","last_clean":"2024-05-08T05:28:04.009901+0000","last_became_active":"2024-05-08T05:25:25.668647+0000","last_became_peered":"2024-05-08T05:25:25.668647+0000","last_unstale":"2024-05-08T05:28:04.009901+0000","last_undegraded":"2024-05-08T05:28:04.009901+0000","last_fullsized":"2024-05-08T05:28:04.009901+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:07:00.962620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.014161+0000","last_change":"2024-05-08T05:25:25.664784+0000","last_active":"2024-05-08T05:28:04.014161+0000","last_peered":"2024-05-08T05:28:04.014161+0000","last_clean":"2024-05-08T05:28:04.014161+0000","last_became_active":"2024-05-08T05:25:25.664390+0000","last_became_peered":"2024-05-08T05:25:25.664390+0000","last_unstale":"2024-05-08T05:28:04.014161+0000","last_undegraded":"2024-05-08T05:28:04.014161+0000","last_fullsized":"2024-05-08T05:28:04.014161+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:08:35.358796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.021523+0000","last_change":"2024-05-08T05:25:25.677139+0000","last_active":"2024-05-08T05:28:04.021523+0000","last_peered":"2024-05-08T05:28:04.021523+0000","last_clean":"2024-05-08T05:28:04.021523+0000","last_became_active":"2024-05-08T05:25:25.676444+0000","last_became_peered":"2024-05-08T05:25:25.676444+0000","last_unstale":"2024-05-08T05:28:04.021523+0000","last_undegraded":"2024-05-08T05:28:04.021523+0000","last_fullsized":"2024-05-08T05:28:04.021523+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:56.704152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.016183+0000","last_change":"2024-05-08T05:25:25.675593+0000","last_active":"2024-05-08T05:28:04.016183+0000","last_peered":"2024-05-08T05:28:04.016183+0000","last_clean":"2024-05-08T05:28:04.016183+0000","last_became_active":"2024-05-08T05:25:25.675136+0000","last_became_peered":"2024-05-08T05:25:25.675136+0000","last_unstale":"2024-05-08T05:28:04.016183+0000","last_undegraded":"2024-05-08T05:28:04.016183+0000","last_fullsized":"2024-05-08T05:28:04.016183+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:49.042956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.010310+0000","last_change":"2024-05-08T05:25:25.669384+0000","last_active":"2024-05-08T05:28:04.010310+0000","last_peered":"2024-05-08T05:28:04.010310+0000","last_clean":"2024-05-08T05:28:04.010310+0000","last_became_active":"2024-05-08T05:25:25.668861+0000","last_became_peered":"2024-05-08T05:25:25.668861+0000","last_unstale":"2024-05-08T05:28:04.010310+0000","last_undegraded":"2024-05-08T05:28:04.010310+0000","last_fullsized":"2024-05-08T05:28:04.010310+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:57.491242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.014903+0000","last_change":"2024-05-08T05:25:25.669463+0000","last_active":"2024-05-08T05:28:04.014903+0000","last_peered":"2024-05-08T05:28:04.014903+0000","last_clean":"2024-05-08T05:28:04.014903+0000","last_became_active":"2024-05-08T05:25:25.668672+0000","last_became_peered":"2024-05-08T05:25:25.668672+0000","last_unstale":"2024-05-08T05:28:04.014903+0000","last_undegraded":"2024-05-08T05:28:04.014903+0000","last_fullsized":"2024-05-08T05:28:04.014903+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:30.779473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.005727+0000","last_change":"2024-05-08T05:25:25.670378+0000","last_active":"2024-05-08T05:28:04.005727+0000","last_peered":"2024-05-08T05:28:04.005727+0000","last_clean":"2024-05-08T05:28:04.005727+0000","last_became_active":"2024-05-08T05:25:25.669801+0000","last_became_peered":"2024-05-08T05:25:25.669801+0000","last_unstale":"2024-05-08T05:28:04.005727+0000","last_undegraded":"2024-05-08T05:28:04.005727+0000","last_fullsized":"2024-05-08T05:28:04.005727+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:49.614163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.012174+0000","last_change":"2024-05-08T05:25:25.670874+0000","last_active":"2024-05-08T05:28:04.012174+0000","last_peered":"2024-05-08T05:28:04.012174+0000","last_clean":"2024-05-08T05:28:04.012174+0000","last_became_active":"2024-05-08T05:25:25.670557+0000","last_became_peered":"2024-05-08T05:25:25.670557+0000","last_unstale":"2024-05-08T05:28:04.012174+0000","last_undegraded":"2024-05-08T05:28:04.012174+0000","last_fullsized":"2024-05-08T05:28:04.012174+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:58.594223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.010134+0000","last_change":"2024-05-08T05:25:25.670015+0000","last_active":"2024-05-08T05:28:04.010134+0000","last_peered":"2024-05-08T05:28:04.010134+0000","last_clean":"2024-05-08T05:28:04.010134+0000","last_became_active":"2024-05-08T05:25:25.669493+0000","last_became_peered":"2024-05-08T05:25:25.669493+0000","last_unstale":"2024-05-08T05:28:04.010134+0000","last_undegraded":"2024-05-08T05:28:04.010134+0000","last_fullsized":"2024-05-08T05:28:04.010134+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:49.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.013746+0000","last_change":"2024-05-08T05:25:25.659651+0000","last_active":"2024-05-08T05:28:04.013746+0000","last_peered":"2024-05-08T05:28:04.013746+0000","last_clean":"2024-05-08T05:28:04.013746+0000","last_became_active":"2024-05-08T05:25:25.659308+0000","last_became_peered":"2024-05-08T05:25:25.659308+0000","last_unstale":"2024-05-08T05:28:04.013746+0000","last_undegraded":"2024-05-08T05:28:04.013746+0000","last_fullsized":"2024-05-08T05:28:04.013746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:49.945474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.017418+0000","last_change":"2024-05-08T05:25:25.663696+0000","last_active":"2024-05-08T05:28:04.017418+0000","last_peered":"2024-05-08T05:28:04.017418+0000","last_clean":"2024-05-08T05:28:04.017418+0000","last_became_active":"2024-05-08T05:25:25.662590+0000","last_became_peered":"2024-05-08T05:25:25.662590+0000","last_unstale":"2024-05-08T05:28:04.017418+0000","last_undegraded":"2024-05-08T05:28:04.017418+0000","last_fullsized":"2024-05-08T05:28:04.017418+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:29:21.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018348+0000","last_change":"2024-05-08T05:25:25.668552+0000","last_active":"2024-05-08T05:28:04.018348+0000","last_peered":"2024-05-08T05:28:04.018348+0000","last_clean":"2024-05-08T05:28:04.018348+0000","last_became_active":"2024-05-08T05:25:25.668185+0000","last_became_peered":"2024-05-08T05:25:25.668185+0000","last_unstale":"2024-05-08T05:28:04.018348+0000","last_undegraded":"2024-05-08T05:28:04.018348+0000","last_fullsized":"2024-05-08T05:28:04.018348+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:25.762909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008065+0000","last_change":"2024-05-08T05:25:25.668259+0000","last_active":"2024-05-08T05:28:04.008065+0000","last_peered":"2024-05-08T05:28:04.008065+0000","last_clean":"2024-05-08T05:28:04.008065+0000","last_became_active":"2024-05-08T05:25:25.667767+0000","last_became_peered":"2024-05-08T05:25:25.667767+0000","last_unstale":"2024-05-08T05:28:04.008065+0000","last_undegraded":"2024-05-08T05:28:04.008065+0000","last_fullsized":"2024-05-08T05:28:04.008065+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:36:34.984139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.017148+0000","last_change":"2024-05-08T05:25:25.663381+0000","last_active":"2024-05-08T05:28:04.017148+0000","last_peered":"2024-05-08T05:28:04.017148+0000","last_clean":"2024-05-08T05:28:04.017148+0000","last_became_active":"2024-05-08T05:25:25.663019+0000","last_became_peered":"2024-05-08T05:25:25.663019+0000","last_unstale":"2024-05-08T05:28:04.017148+0000","last_undegraded":"2024-05-08T05:28:04.017148+0000","last_fullsized":"2024-05-08T05:28:04.017148+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:58.440446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564094+0000","last_change":"2024-05-08T05:25:37.052656+0000","last_active":"2024-05-08T05:27:05.564094+0000","last_peered":"2024-05-08T05:27:05.564094+0000","last_clean":"2024-05-08T05:27:05.564094+0000","last_became_active":"2024-05-08T05:25:37.052237+0000","last_became_peered":"2024-05-08T05:25:37.052237+0000","last_unstale":"2024-05-08T05:27:05.564094+0000","last_undegraded":"2024-05-08T05:27:05.564094+0000","last_fullsized":"2024-05-08T05:27:05.564094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:54.354488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":160,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:07.559186+0000","last_change":"2024-05-08T05:25:22.649492+0000","last_active":"2024-05-08T05:28:07.559186+0000","last_peered":"2024-05-08T05:28:07.559186+0000","last_clean":"2024-05-08T05:28:07.559186+0000","last_became_active":"2024-05-08T05:25:22.649132+0000","last_became_peered":"2024-05-08T05:25:22.649132+0000","last_unstale":"2024-05-08T05:28:07.559186+0000","last_undegraded":"2024-05-08T05:28:07.559186+0000","last_fullsized":"2024-05-08T05:28:07.559186+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:24.474848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":102,"num_read_kb":102,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.013390+0000","last_change":"2024-05-08T05:25:25.667811+0000","last_active":"2024-05-08T05:28:04.013390+0000","last_peered":"2024-05-08T05:28:04.013390+0000","last_clean":"2024-05-08T05:28:04.013390+0000","last_became_active":"2024-05-08T05:25:25.667449+0000","last_became_peered":"2024-05-08T05:25:25.667449+0000","last_unstale":"2024-05-08T05:28:04.013390+0000","last_undegraded":"2024-05-08T05:28:04.013390+0000","last_fullsized":"2024-05-08T05:28:04.013390+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:29.293702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764379+0000","last_change":"2024-05-08T05:26:05.308251+0000","last_active":"2024-05-08T05:27:05.764379+0000","last_peered":"2024-05-08T05:27:05.764379+0000","last_clean":"2024-05-08T05:27:05.764379+0000","last_became_active":"2024-05-08T05:26:05.307622+0000","last_became_peered":"2024-05-08T05:26:05.307622+0000","last_unstale":"2024-05-08T05:27:05.764379+0000","last_undegraded":"2024-05-08T05:27:05.764379+0000","last_fullsized":"2024-05-08T05:27:05.764379+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:51.116094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563675+0000","last_change":"2024-05-08T05:25:22.748130+0000","last_active":"2024-05-08T05:27:05.563675+0000","last_peered":"2024-05-08T05:27:05.563675+0000","last_clean":"2024-05-08T05:27:05.563675+0000","last_became_active":"2024-05-08T05:25:22.747769+0000","last_became_peered":"2024-05-08T05:25:22.747769+0000","last_unstale":"2024-05-08T05:27:05.563675+0000","last_undegraded":"2024-05-08T05:27:05.563675+0000","last_fullsized":"2024-05-08T05:27:05.563675+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:04.086851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.016074+0000","last_change":"2024-05-08T05:25:25.672114+0000","last_active":"2024-05-08T05:28:04.016074+0000","last_peered":"2024-05-08T05:28:04.016074+0000","last_clean":"2024-05-08T05:28:04.016074+0000","last_became_active":"2024-05-08T05:25:25.671761+0000","last_became_peered":"2024-05-08T05:25:25.671761+0000","last_unstale":"2024-05-08T05:28:04.016074+0000","last_undegraded":"2024-05-08T05:28:04.016074+0000","last_fullsized":"2024-05-08T05:28:04.016074+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:40.684565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765356+0000","last_change":"2024-05-08T05:25:37.058375+0000","last_active":"2024-05-08T05:27:05.765356+0000","last_peered":"2024-05-08T05:27:05.765356+0000","last_clean":"2024-05-08T05:27:05.765356+0000","last_became_active":"2024-05-08T05:25:37.057844+0000","last_became_peered":"2024-05-08T05:25:37.057844+0000","last_unstale":"2024-05-08T05:27:05.765356+0000","last_undegraded":"2024-05-08T05:27:05.765356+0000","last_fullsized":"2024-05-08T05:27:05.765356+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:51:42.881830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.576781+0000","last_change":"2024-05-08T05:27:05.576781+0000","last_active":"2024-05-08T05:27:05.576781+0000","last_peered":"2024-05-08T05:27:05.576781+0000","last_clean":"2024-05-08T05:27:05.576781+0000","last_became_active":"2024-05-08T05:27:05.566420+0000","last_became_peered":"2024-05-08T05:27:05.566420+0000","last_unstale":"2024-05-08T05:27:05.576781+0000","last_undegraded":"2024-05-08T05:27:05.576781+0000","last_fullsized":"2024-05-08T05:27:05.576781+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:09.385135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.019679+0000","last_change":"2024-05-08T05:25:25.673300+0000","last_active":"2024-05-08T05:28:04.019679+0000","last_peered":"2024-05-08T05:28:04.019679+0000","last_clean":"2024-05-08T05:28:04.019679+0000","last_became_active":"2024-05-08T05:25:25.672881+0000","last_became_peered":"2024-05-08T05:25:25.672881+0000","last_unstale":"2024-05-08T05:28:04.019679+0000","last_undegraded":"2024-05-08T05:28:04.019679+0000","last_fullsized":"2024-05-08T05:28:04.019679+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:16.111775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764689+0000","last_change":"2024-05-08T05:25:37.058903+0000","last_active":"2024-05-08T05:27:05.764689+0000","last_peered":"2024-05-08T05:27:05.764689+0000","last_clean":"2024-05-08T05:27:05.764689+0000","last_became_active":"2024-05-08T05:25:37.058508+0000","last_became_peered":"2024-05-08T05:25:37.058508+0000","last_unstale":"2024-05-08T05:27:05.764689+0000","last_undegraded":"2024-05-08T05:27:05.764689+0000","last_fullsized":"2024-05-08T05:27:05.764689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:15:16.605748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.567090+0000","last_change":"2024-05-08T05:25:22.646191+0000","last_active":"2024-05-08T05:27:05.567090+0000","last_peered":"2024-05-08T05:27:05.567090+0000","last_clean":"2024-05-08T05:27:05.567090+0000","last_became_active":"2024-05-08T05:25:22.645187+0000","last_became_peered":"2024-05-08T05:25:22.645187+0000","last_unstale":"2024-05-08T05:27:05.567090+0000","last_undegraded":"2024-05-08T05:27:05.567090+0000","last_fullsized":"2024-05-08T05:27:05.567090+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:13.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008387+0000","last_change":"2024-05-08T05:26:05.308149+0000","last_active":"2024-05-08T05:28:04.008387+0000","last_peered":"2024-05-08T05:28:04.008387+0000","last_clean":"2024-05-08T05:28:04.008387+0000","last_became_active":"2024-05-08T05:26:05.307602+0000","last_became_peered":"2024-05-08T05:26:05.307602+0000","last_unstale":"2024-05-08T05:28:04.008387+0000","last_undegraded":"2024-05-08T05:28:04.008387+0000","last_fullsized":"2024-05-08T05:28:04.008387+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:02.791894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564446+0000","last_change":"2024-05-08T05:25:37.059052+0000","last_active":"2024-05-08T05:27:05.564446+0000","last_peered":"2024-05-08T05:27:05.564446+0000","last_clean":"2024-05-08T05:27:05.564446+0000","last_became_active":"2024-05-08T05:25:37.058545+0000","last_became_peered":"2024-05-08T05:25:37.058545+0000","last_unstale":"2024-05-08T05:27:05.564446+0000","last_undegraded":"2024-05-08T05:27:05.564446+0000","last_fullsized":"2024-05-08T05:27:05.564446+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:31.521284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.555379+0000","last_change":"2024-05-08T05:25:22.648874+0000","last_active":"2024-05-08T05:28:04.555379+0000","last_peered":"2024-05-08T05:28:04.555379+0000","last_clean":"2024-05-08T05:28:04.555379+0000","last_became_active":"2024-05-08T05:25:22.648477+0000","last_became_peered":"2024-05-08T05:25:22.648477+0000","last_unstale":"2024-05-08T05:28:04.555379+0000","last_undegraded":"2024-05-08T05:28:04.555379+0000","last_fullsized":"2024-05-08T05:28:04.555379+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:08.075904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.003459+0000","last_change":"2024-05-08T05:25:25.662449+0000","last_active":"2024-05-08T05:28:04.003459+0000","last_peered":"2024-05-08T05:28:04.003459+0000","last_clean":"2024-05-08T05:28:04.003459+0000","last_became_active":"2024-05-08T05:25:25.662089+0000","last_became_peered":"2024-05-08T05:25:25.662089+0000","last_unstale":"2024-05-08T05:28:04.003459+0000","last_undegraded":"2024-05-08T05:28:04.003459+0000","last_fullsized":"2024-05-08T05:28:04.003459+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:31.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":26,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563983+0000","last_change":"2024-05-08T05:25:37.055548+0000","last_active":"2024-05-08T05:27:05.563983+0000","last_peered":"2024-05-08T05:27:05.563983+0000","last_clean":"2024-05-08T05:27:05.563983+0000","last_became_active":"2024-05-08T05:25:37.055138+0000","last_became_peered":"2024-05-08T05:25:37.055138+0000","last_unstale":"2024-05-08T05:27:05.563983+0000","last_undegraded":"2024-05-08T05:27:05.563983+0000","last_fullsized":"2024-05-08T05:27:05.563983+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:45.697831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765546+0000","last_change":"2024-05-08T05:25:22.648072+0000","last_active":"2024-05-08T05:27:05.765546+0000","last_peered":"2024-05-08T05:27:05.765546+0000","last_clean":"2024-05-08T05:27:05.765546+0000","last_became_active":"2024-05-08T05:25:22.647457+0000","last_became_peered":"2024-05-08T05:25:22.647457+0000","last_unstale":"2024-05-08T05:27:05.765546+0000","last_undegraded":"2024-05-08T05:27:05.765546+0000","last_fullsized":"2024-05-08T05:27:05.765546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:17.476479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.010562+0000","last_change":"2024-05-08T05:25:25.662698+0000","last_active":"2024-05-08T05:28:04.010562+0000","last_peered":"2024-05-08T05:28:04.010562+0000","last_clean":"2024-05-08T05:28:04.010562+0000","last_became_active":"2024-05-08T05:25:25.662368+0000","last_became_peered":"2024-05-08T05:25:25.662368+0000","last_unstale":"2024-05-08T05:28:04.010562+0000","last_undegraded":"2024-05-08T05:28:04.010562+0000","last_fullsized":"2024-05-08T05:28:04.010562+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:38.511037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.565170+0000","last_change":"2024-05-08T05:25:37.054407+0000","last_active":"2024-05-08T05:27:05.565170+0000","last_peered":"2024-05-08T05:27:05.565170+0000","last_clean":"2024-05-08T05:27:05.565170+0000","last_became_active":"2024-05-08T05:25:37.053999+0000","last_became_peered":"2024-05-08T05:25:37.053999+0000","last_unstale":"2024-05-08T05:27:05.565170+0000","last_undegraded":"2024-05-08T05:27:05.565170+0000","last_fullsized":"2024-05-08T05:27:05.565170+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:16.942601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.560574+0000","last_change":"2024-05-08T05:25:22.643654+0000","last_active":"2024-05-08T05:27:05.560574+0000","last_peered":"2024-05-08T05:27:05.560574+0000","last_clean":"2024-05-08T05:27:05.560574+0000","last_became_active":"2024-05-08T05:25:22.643315+0000","last_became_peered":"2024-05-08T05:25:22.643315+0000","last_unstale":"2024-05-08T05:27:05.560574+0000","last_undegraded":"2024-05-08T05:27:05.560574+0000","last_fullsized":"2024-05-08T05:27:05.560574+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:53.514629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020446+0000","last_change":"2024-05-08T05:25:25.663148+0000","last_active":"2024-05-08T05:28:04.020446+0000","last_peered":"2024-05-08T05:28:04.020446+0000","last_clean":"2024-05-08T05:28:04.020446+0000","last_became_active":"2024-05-08T05:25:25.662770+0000","last_became_peered":"2024-05-08T05:25:25.662770+0000","last_unstale":"2024-05-08T05:28:04.020446+0000","last_undegraded":"2024-05-08T05:28:04.020446+0000","last_fullsized":"2024-05-08T05:28:04.020446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:46.458233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.723878+0000","last_change":"2024-05-08T05:25:37.054265+0000","last_active":"2024-05-08T05:27:05.723878+0000","last_peered":"2024-05-08T05:27:05.723878+0000","last_clean":"2024-05-08T05:27:05.723878+0000","last_became_active":"2024-05-08T05:25:37.053579+0000","last_became_peered":"2024-05-08T05:25:37.053579+0000","last_unstale":"2024-05-08T05:27:05.723878+0000","last_undegraded":"2024-05-08T05:27:05.723878+0000","last_fullsized":"2024-05-08T05:27:05.723878+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:21.748715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561931+0000","last_change":"2024-05-08T05:25:22.648083+0000","last_active":"2024-05-08T05:27:05.561931+0000","last_peered":"2024-05-08T05:27:05.561931+0000","last_clean":"2024-05-08T05:27:05.561931+0000","last_became_active":"2024-05-08T05:25:22.647650+0000","last_became_peered":"2024-05-08T05:25:22.647650+0000","last_unstale":"2024-05-08T05:27:05.561931+0000","last_undegraded":"2024-05-08T05:27:05.561931+0000","last_fullsized":"2024-05-08T05:27:05.561931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:56.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.002042+0000","last_change":"2024-05-08T05:25:25.668290+0000","last_active":"2024-05-08T05:28:04.002042+0000","last_peered":"2024-05-08T05:28:04.002042+0000","last_clean":"2024-05-08T05:28:04.002042+0000","last_became_active":"2024-05-08T05:25:25.667888+0000","last_became_peered":"2024-05-08T05:25:25.667888+0000","last_unstale":"2024-05-08T05:28:04.002042+0000","last_undegraded":"2024-05-08T05:28:04.002042+0000","last_fullsized":"2024-05-08T05:28:04.002042+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:19.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563156+0000","last_change":"2024-05-08T05:25:37.057639+0000","last_active":"2024-05-08T05:27:05.563156+0000","last_peered":"2024-05-08T05:27:05.563156+0000","last_clean":"2024-05-08T05:27:05.563156+0000","last_became_active":"2024-05-08T05:25:37.056848+0000","last_became_peered":"2024-05-08T05:25:37.056848+0000","last_unstale":"2024-05-08T05:27:05.563156+0000","last_undegraded":"2024-05-08T05:27:05.563156+0000","last_fullsized":"2024-05-08T05:27:05.563156+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:37.137224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"90'6","reported_seq":250,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:07.559399+0000","last_change":"2024-05-08T05:25:22.649658+0000","last_active":"2024-05-08T05:28:07.559399+0000","last_peered":"2024-05-08T05:28:07.559399+0000","last_clean":"2024-05-08T05:28:07.559399+0000","last_became_active":"2024-05-08T05:25:22.648771+0000","last_became_peered":"2024-05-08T05:25:22.648771+0000","last_unstale":"2024-05-08T05:28:07.559399+0000","last_undegraded":"2024-05-08T05:28:07.559399+0000","last_fullsized":"2024-05-08T05:28:07.559399+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:54.412710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":200,"num_read_kb":200,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018564+0000","last_change":"2024-05-08T05:25:25.673931+0000","last_active":"2024-05-08T05:28:04.018564+0000","last_peered":"2024-05-08T05:28:04.018564+0000","last_clean":"2024-05-08T05:28:04.018564+0000","last_became_active":"2024-05-08T05:25:25.673585+0000","last_became_peered":"2024-05-08T05:25:25.673585+0000","last_unstale":"2024-05-08T05:28:04.018564+0000","last_undegraded":"2024-05-08T05:28:04.018564+0000","last_fullsized":"2024-05-08T05:28:04.018564+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:27.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764218+0000","last_change":"2024-05-08T05:25:37.047651+0000","last_active":"2024-05-08T05:27:05.764218+0000","last_peered":"2024-05-08T05:27:05.764218+0000","last_clean":"2024-05-08T05:27:05.764218+0000","last_became_active":"2024-05-08T05:25:37.047246+0000","last_became_peered":"2024-05-08T05:25:37.047246+0000","last_unstale":"2024-05-08T05:27:05.764218+0000","last_undegraded":"2024-05-08T05:27:05.764218+0000","last_fullsized":"2024-05-08T05:27:05.764218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:22.300076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561924+0000","last_change":"2024-05-08T05:25:22.646973+0000","last_active":"2024-05-08T05:27:05.561924+0000","last_peered":"2024-05-08T05:27:05.561924+0000","last_clean":"2024-05-08T05:27:05.561924+0000","last_became_active":"2024-05-08T05:25:22.646232+0000","last_became_peered":"2024-05-08T05:25:22.646232+0000","last_unstale":"2024-05-08T05:27:05.561924+0000","last_undegraded":"2024-05-08T05:27:05.561924+0000","last_fullsized":"2024-05-08T05:27:05.561924+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:38.147470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.016976+0000","last_change":"2024-05-08T05:25:25.661373+0000","last_active":"2024-05-08T05:28:04.016976+0000","last_peered":"2024-05-08T05:28:04.016976+0000","last_clean":"2024-05-08T05:28:04.016976+0000","last_became_active":"2024-05-08T05:25:25.661080+0000","last_became_peered":"2024-05-08T05:25:25.661080+0000","last_unstale":"2024-05-08T05:28:04.016976+0000","last_undegraded":"2024-05-08T05:28:04.016976+0000","last_fullsized":"2024-05-08T05:28:04.016976+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:18.718610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765639+0000","last_change":"2024-05-08T05:25:37.051797+0000","last_active":"2024-05-08T05:27:05.765639+0000","last_peered":"2024-05-08T05:27:05.765639+0000","last_clean":"2024-05-08T05:27:05.765639+0000","last_became_active":"2024-05-08T05:25:37.051381+0000","last_became_peered":"2024-05-08T05:25:37.051381+0000","last_unstale":"2024-05-08T05:27:05.765639+0000","last_undegraded":"2024-05-08T05:27:05.765639+0000","last_fullsized":"2024-05-08T05:27:05.765639+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:34.337512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562222+0000","last_change":"2024-05-08T05:25:22.748510+0000","last_active":"2024-05-08T05:27:05.562222+0000","last_peered":"2024-05-08T05:27:05.562222+0000","last_clean":"2024-05-08T05:27:05.562222+0000","last_became_active":"2024-05-08T05:25:22.748140+0000","last_became_peered":"2024-05-08T05:25:22.748140+0000","last_unstale":"2024-05-08T05:27:05.562222+0000","last_undegraded":"2024-05-08T05:27:05.562222+0000","last_fullsized":"2024-05-08T05:27:05.562222+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:32.325653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.017787+0000","last_change":"2024-05-08T05:25:25.671261+0000","last_active":"2024-05-08T05:28:04.017787+0000","last_peered":"2024-05-08T05:28:04.017787+0000","last_clean":"2024-05-08T05:28:04.017787+0000","last_became_active":"2024-05-08T05:25:25.670937+0000","last_became_peered":"2024-05-08T05:25:25.670937+0000","last_unstale":"2024-05-08T05:28:04.017787+0000","last_undegraded":"2024-05-08T05:28:04.017787+0000","last_fullsized":"2024-05-08T05:28:04.017787+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:41.375908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765231+0000","last_change":"2024-05-08T05:26:05.150754+0000","last_active":"2024-05-08T05:27:05.765231+0000","last_peered":"2024-05-08T05:27:05.765231+0000","last_clean":"2024-05-08T05:27:05.765231+0000","last_became_active":"2024-05-08T05:26:05.150331+0000","last_became_peered":"2024-05-08T05:26:05.150331+0000","last_unstale":"2024-05-08T05:27:05.765231+0000","last_undegraded":"2024-05-08T05:27:05.765231+0000","last_fullsized":"2024-05-08T05:27:05.765231+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:23.309103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"90'38","reported_seq":292,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:07.575717+0000","last_change":"2024-05-08T05:25:22.645627+0000","last_active":"2024-05-08T05:28:07.575717+0000","last_peered":"2024-05-08T05:28:07.575717+0000","last_clean":"2024-05-08T05:28:07.575717+0000","last_became_active":"2024-05-08T05:25:22.645243+0000","last_became_peered":"2024-05-08T05:25:22.645243+0000","last_unstale":"2024-05-08T05:28:07.575717+0000","last_undegraded":"2024-05-08T05:28:07.575717+0000","last_fullsized":"2024-05-08T05:28:07.575717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:23.256903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":64894,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":210,"num_read_kb":169,"num_write":40,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.002021+0000","last_change":"2024-05-08T05:25:25.661258+0000","last_active":"2024-05-08T05:28:04.002021+0000","last_peered":"2024-05-08T05:28:04.002021+0000","last_clean":"2024-05-08T05:28:04.002021+0000","last_became_active":"2024-05-08T05:25:25.660925+0000","last_became_peered":"2024-05-08T05:25:25.660925+0000","last_unstale":"2024-05-08T05:28:04.002021+0000","last_undegraded":"2024-05-08T05:28:04.002021+0000","last_fullsized":"2024-05-08T05:28:04.002021+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:58.764540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562882+0000","last_change":"2024-05-08T05:25:37.053539+0000","last_active":"2024-05-08T05:27:05.562882+0000","last_peered":"2024-05-08T05:27:05.562882+0000","last_clean":"2024-05-08T05:27:05.562882+0000","last_became_active":"2024-05-08T05:25:37.053102+0000","last_became_peered":"2024-05-08T05:25:37.053102+0000","last_unstale":"2024-05-08T05:27:05.562882+0000","last_undegraded":"2024-05-08T05:27:05.562882+0000","last_fullsized":"2024-05-08T05:27:05.562882+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:48.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":55,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564138+0000","last_change":"2024-05-08T05:25:22.646651+0000","last_active":"2024-05-08T05:27:05.564138+0000","last_peered":"2024-05-08T05:27:05.564138+0000","last_clean":"2024-05-08T05:27:05.564138+0000","last_became_active":"2024-05-08T05:25:22.646285+0000","last_became_peered":"2024-05-08T05:25:22.646285+0000","last_unstale":"2024-05-08T05:27:05.564138+0000","last_undegraded":"2024-05-08T05:27:05.564138+0000","last_fullsized":"2024-05-08T05:27:05.564138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.843858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020203+0000","last_change":"2024-05-08T05:25:25.666018+0000","last_active":"2024-05-08T05:28:04.020203+0000","last_peered":"2024-05-08T05:28:04.020203+0000","last_clean":"2024-05-08T05:28:04.020203+0000","last_became_active":"2024-05-08T05:25:25.663609+0000","last_became_peered":"2024-05-08T05:25:25.663609+0000","last_unstale":"2024-05-08T05:28:04.020203+0000","last_undegraded":"2024-05-08T05:28:04.020203+0000","last_fullsized":"2024-05-08T05:28:04.020203+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:36.223121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564512+0000","last_change":"2024-05-08T05:25:37.052907+0000","last_active":"2024-05-08T05:27:05.564512+0000","last_peered":"2024-05-08T05:27:05.564512+0000","last_clean":"2024-05-08T05:27:05.564512+0000","last_became_active":"2024-05-08T05:25:37.052506+0000","last_became_peered":"2024-05-08T05:25:37.052506+0000","last_unstale":"2024-05-08T05:27:05.564512+0000","last_undegraded":"2024-05-08T05:27:05.564512+0000","last_fullsized":"2024-05-08T05:27:05.564512+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:41.356996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562180+0000","last_change":"2024-05-08T05:25:22.646671+0000","last_active":"2024-05-08T05:27:05.562180+0000","last_peered":"2024-05-08T05:27:05.562180+0000","last_clean":"2024-05-08T05:27:05.562180+0000","last_became_active":"2024-05-08T05:25:22.646202+0000","last_became_peered":"2024-05-08T05:25:22.646202+0000","last_unstale":"2024-05-08T05:27:05.562180+0000","last_undegraded":"2024-05-08T05:27:05.562180+0000","last_fullsized":"2024-05-08T05:27:05.562180+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:59.224960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":54,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.016014+0000","last_change":"2024-05-08T05:25:25.666969+0000","last_active":"2024-05-08T05:28:04.016014+0000","last_peered":"2024-05-08T05:28:04.016014+0000","last_clean":"2024-05-08T05:28:04.016014+0000","last_became_active":"2024-05-08T05:25:25.666482+0000","last_became_peered":"2024-05-08T05:25:25.666482+0000","last_unstale":"2024-05-08T05:28:04.016014+0000","last_undegraded":"2024-05-08T05:28:04.016014+0000","last_fullsized":"2024-05-08T05:28:04.016014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:07.819116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764513+0000","last_change":"2024-05-08T05:25:37.055807+0000","last_active":"2024-05-08T05:27:05.764513+0000","last_peered":"2024-05-08T05:27:05.764513+0000","last_clean":"2024-05-08T05:27:05.764513+0000","last_became_active":"2024-05-08T05:25:37.054778+0000","last_became_peered":"2024-05-08T05:25:37.054778+0000","last_unstale":"2024-05-08T05:27:05.764513+0000","last_undegraded":"2024-05-08T05:27:05.764513+0000","last_fullsized":"2024-05-08T05:27:05.764513+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:33.767885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.556573+0000","last_change":"2024-05-08T05:25:22.647093+0000","last_active":"2024-05-08T05:28:04.556573+0000","last_peered":"2024-05-08T05:28:04.556573+0000","last_clean":"2024-05-08T05:28:04.556573+0000","last_became_active":"2024-05-08T05:25:22.646733+0000","last_became_peered":"2024-05-08T05:25:22.646733+0000","last_unstale":"2024-05-08T05:28:04.556573+0000","last_undegraded":"2024-05-08T05:28:04.556573+0000","last_fullsized":"2024-05-08T05:28:04.556573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:04.346568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":103,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.001297+0000","last_change":"2024-05-08T05:25:25.666565+0000","last_active":"2024-05-08T05:28:04.001297+0000","last_peered":"2024-05-08T05:28:04.001297+0000","last_clean":"2024-05-08T05:28:04.001297+0000","last_became_active":"2024-05-08T05:25:25.666138+0000","last_became_peered":"2024-05-08T05:25:25.666138+0000","last_unstale":"2024-05-08T05:28:04.001297+0000","last_undegraded":"2024-05-08T05:28:04.001297+0000","last_fullsized":"2024-05-08T05:28:04.001297+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:05.396530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561191+0000","last_change":"2024-05-08T05:25:37.060933+0000","last_active":"2024-05-08T05:27:05.561191+0000","last_peered":"2024-05-08T05:27:05.561191+0000","last_clean":"2024-05-08T05:27:05.561191+0000","last_became_active":"2024-05-08T05:25:37.060388+0000","last_became_peered":"2024-05-08T05:25:37.060388+0000","last_unstale":"2024-05-08T05:27:05.561191+0000","last_undegraded":"2024-05-08T05:27:05.561191+0000","last_fullsized":"2024-05-08T05:27:05.561191+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:09.776051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":79,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.566531+0000","last_change":"2024-05-08T05:26:05.156863+0000","last_active":"2024-05-08T05:27:05.566531+0000","last_peered":"2024-05-08T05:27:05.566531+0000","last_clean":"2024-05-08T05:27:05.566531+0000","last_became_active":"2024-05-08T05:26:05.149008+0000","last_became_peered":"2024-05-08T05:26:05.149008+0000","last_unstale":"2024-05-08T05:27:05.566531+0000","last_undegraded":"2024-05-08T05:27:05.566531+0000","last_fullsized":"2024-05-08T05:27:05.566531+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:56.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008664+0000","last_change":"2024-05-08T05:25:25.670415+0000","last_active":"2024-05-08T05:28:04.008664+0000","last_peered":"2024-05-08T05:28:04.008664+0000","last_clean":"2024-05-08T05:28:04.008664+0000","last_became_active":"2024-05-08T05:25:25.670095+0000","last_became_peered":"2024-05-08T05:25:25.670095+0000","last_unstale":"2024-05-08T05:28:04.008664+0000","last_undegraded":"2024-05-08T05:28:04.008664+0000","last_fullsized":"2024-05-08T05:28:04.008664+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.489203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":16,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.566513+0000","last_change":"2024-05-08T05:26:05.307721+0000","last_active":"2024-05-08T05:27:05.566513+0000","last_peered":"2024-05-08T05:27:05.566513+0000","last_clean":"2024-05-08T05:27:05.566513+0000","last_became_active":"2024-05-08T05:26:05.307279+0000","last_became_peered":"2024-05-08T05:26:05.307279+0000","last_unstale":"2024-05-08T05:27:05.566513+0000","last_undegraded":"2024-05-08T05:27:05.566513+0000","last_fullsized":"2024-05-08T05:27:05.566513+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:08.857770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"88'2","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.567101+0000","last_change":"2024-05-08T05:25:22.644279+0000","last_active":"2024-05-08T05:27:05.567101+0000","last_peered":"2024-05-08T05:27:05.567101+0000","last_clean":"2024-05-08T05:27:05.567101+0000","last_became_active":"2024-05-08T05:25:22.643900+0000","last_became_peered":"2024-05-08T05:25:22.643900+0000","last_unstale":"2024-05-08T05:27:05.567101+0000","last_undegraded":"2024-05-08T05:27:05.567101+0000","last_fullsized":"2024-05-08T05:27:05.567101+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:32.435843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.006132+0000","last_change":"2024-05-08T05:25:25.670130+0000","last_active":"2024-05-08T05:28:04.006132+0000","last_peered":"2024-05-08T05:28:04.006132+0000","last_clean":"2024-05-08T05:28:04.006132+0000","last_became_active":"2024-05-08T05:25:25.669784+0000","last_became_peered":"2024-05-08T05:25:25.669784+0000","last_unstale":"2024-05-08T05:28:04.006132+0000","last_undegraded":"2024-05-08T05:28:04.006132+0000","last_fullsized":"2024-05-08T05:28:04.006132+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:07.188883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":36,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.566659+0000","last_change":"2024-05-08T05:26:05.308804+0000","last_active":"2024-05-08T05:27:05.566659+0000","last_peered":"2024-05-08T05:27:05.566659+0000","last_clean":"2024-05-08T05:27:05.566659+0000","last_became_active":"2024-05-08T05:26:05.308226+0000","last_became_peered":"2024-05-08T05:26:05.308226+0000","last_unstale":"2024-05-08T05:27:05.566659+0000","last_undegraded":"2024-05-08T05:27:05.566659+0000","last_fullsized":"2024-05-08T05:27:05.566659+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:33.442023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764614+0000","last_change":"2024-05-08T05:25:22.748409+0000","last_active":"2024-05-08T05:27:05.764614+0000","last_peered":"2024-05-08T05:27:05.764614+0000","last_clean":"2024-05-08T05:27:05.764614+0000","last_became_active":"2024-05-08T05:25:22.748062+0000","last_became_peered":"2024-05-08T05:25:22.748062+0000","last_unstale":"2024-05-08T05:27:05.764614+0000","last_undegraded":"2024-05-08T05:27:05.764614+0000","last_fullsized":"2024-05-08T05:27:05.764614+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:42.147084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":42,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.019034+0000","last_change":"2024-05-08T05:25:25.677057+0000","last_active":"2024-05-08T05:28:04.019034+0000","last_peered":"2024-05-08T05:28:04.019034+0000","last_clean":"2024-05-08T05:28:04.019034+0000","last_became_active":"2024-05-08T05:25:25.676697+0000","last_became_peered":"2024-05-08T05:25:25.676697+0000","last_unstale":"2024-05-08T05:28:04.019034+0000","last_undegraded":"2024-05-08T05:28:04.019034+0000","last_fullsized":"2024-05-08T05:28:04.019034+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:40.786327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.560833+0000","last_change":"2024-05-08T05:25:37.053903+0000","last_active":"2024-05-08T05:27:05.560833+0000","last_peered":"2024-05-08T05:27:05.560833+0000","last_clean":"2024-05-08T05:27:05.560833+0000","last_became_active":"2024-05-08T05:25:37.053375+0000","last_became_peered":"2024-05-08T05:25:37.053375+0000","last_unstale":"2024-05-08T05:27:05.560833+0000","last_undegraded":"2024-05-08T05:27:05.560833+0000","last_fullsized":"2024-05-08T05:27:05.560833+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:19.374355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.555428+0000","last_change":"2024-05-08T05:25:22.646931+0000","last_active":"2024-05-08T05:28:04.555428+0000","last_peered":"2024-05-08T05:28:04.555428+0000","last_clean":"2024-05-08T05:28:04.555428+0000","last_became_active":"2024-05-08T05:25:22.646599+0000","last_became_peered":"2024-05-08T05:25:22.646599+0000","last_unstale":"2024-05-08T05:28:04.555428+0000","last_undegraded":"2024-05-08T05:28:04.555428+0000","last_fullsized":"2024-05-08T05:28:04.555428+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:55.873089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":101,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.010893+0000","last_change":"2024-05-08T05:26:05.147977+0000","last_active":"2024-05-08T05:28:04.010893+0000","last_peered":"2024-05-08T05:28:04.010893+0000","last_clean":"2024-05-08T05:28:04.010893+0000","last_became_active":"2024-05-08T05:26:05.147637+0000","last_became_peered":"2024-05-08T05:26:05.147637+0000","last_unstale":"2024-05-08T05:28:04.010893+0000","last_undegraded":"2024-05-08T05:28:04.010893+0000","last_fullsized":"2024-05-08T05:28:04.010893+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:59.671306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.560851+0000","last_change":"2024-05-08T05:25:37.057913+0000","last_active":"2024-05-08T05:27:05.560851+0000","last_peered":"2024-05-08T05:27:05.560851+0000","last_clean":"2024-05-08T05:27:05.560851+0000","last_became_active":"2024-05-08T05:25:37.057474+0000","last_became_peered":"2024-05-08T05:25:37.057474+0000","last_unstale":"2024-05-08T05:27:05.560851+0000","last_undegraded":"2024-05-08T05:27:05.560851+0000","last_fullsized":"2024-05-08T05:27:05.560851+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:19.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008256+0000","last_change":"2024-05-08T05:25:25.663594+0000","last_active":"2024-05-08T05:28:04.008256+0000","last_peered":"2024-05-08T05:28:04.008256+0000","last_clean":"2024-05-08T05:28:04.008256+0000","last_became_active":"2024-05-08T05:25:25.663275+0000","last_became_peered":"2024-05-08T05:25:25.663275+0000","last_unstale":"2024-05-08T05:28:04.008256+0000","last_undegraded":"2024-05-08T05:28:04.008256+0000","last_fullsized":"2024-05-08T05:28:04.008256+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:20.581166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'6","reported_seq":151,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:05.557262+0000","last_change":"2024-05-08T05:25:22.748316+0000","last_active":"2024-05-08T05:28:05.557262+0000","last_peered":"2024-05-08T05:28:05.557262+0000","last_clean":"2024-05-08T05:28:05.557262+0000","last_became_active":"2024-05-08T05:25:22.747819+0000","last_became_peered":"2024-05-08T05:25:22.747819+0000","last_unstale":"2024-05-08T05:28:05.557262+0000","last_undegraded":"2024-05-08T05:28:05.557262+0000","last_fullsized":"2024-05-08T05:28:05.557262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:50:42.689853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":100,"num_read_kb":100,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":30,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561916+0000","last_change":"2024-05-08T05:25:37.058271+0000","last_active":"2024-05-08T05:27:05.561916+0000","last_peered":"2024-05-08T05:27:05.561916+0000","last_clean":"2024-05-08T05:27:05.561916+0000","last_became_active":"2024-05-08T05:25:37.057290+0000","last_became_peered":"2024-05-08T05:25:37.057290+0000","last_unstale":"2024-05-08T05:27:05.561916+0000","last_undegraded":"2024-05-08T05:27:05.561916+0000","last_fullsized":"2024-05-08T05:27:05.561916+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:47.369909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.018403+0000","last_change":"2024-05-08T05:25:25.666000+0000","last_active":"2024-05-08T05:28:04.018403+0000","last_peered":"2024-05-08T05:28:04.018403+0000","last_clean":"2024-05-08T05:28:04.018403+0000","last_became_active":"2024-05-08T05:25:25.665663+0000","last_became_peered":"2024-05-08T05:25:25.665663+0000","last_unstale":"2024-05-08T05:28:04.018403+0000","last_undegraded":"2024-05-08T05:28:04.018403+0000","last_fullsized":"2024-05-08T05:28:04.018403+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:44.328879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765527+0000","last_change":"2024-05-08T05:25:22.648770+0000","last_active":"2024-05-08T05:27:05.765527+0000","last_peered":"2024-05-08T05:27:05.765527+0000","last_clean":"2024-05-08T05:27:05.765527+0000","last_became_active":"2024-05-08T05:25:22.648119+0000","last_became_peered":"2024-05-08T05:25:22.648119+0000","last_unstale":"2024-05-08T05:27:05.765527+0000","last_undegraded":"2024-05-08T05:27:05.765527+0000","last_fullsized":"2024-05-08T05:27:05.765527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:55.813080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":119,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561483+0000","last_change":"2024-05-08T05:23:55.820436+0000","last_active":"2024-05-08T05:27:05.561483+0000","last_peered":"2024-05-08T05:27:05.561483+0000","last_clean":"2024-05-08T05:27:05.561483+0000","last_became_active":"2024-05-08T05:23:55.814786+0000","last_became_peered":"2024-05-08T05:23:55.814786+0000","last_unstale":"2024-05-08T05:27:05.561483+0000","last_undegraded":"2024-05-08T05:27:05.561483+0000","last_fullsized":"2024-05-08T05:27:05.561483+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_clean_scrub_stamp":"2024-05-08T05:20:32.659283+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:11.169455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562544+0000","last_change":"2024-05-08T05:25:37.057287+0000","last_active":"2024-05-08T05:27:05.562544+0000","last_peered":"2024-05-08T05:27:05.562544+0000","last_clean":"2024-05-08T05:27:05.562544+0000","last_became_active":"2024-05-08T05:25:37.056871+0000","last_became_peered":"2024-05-08T05:25:37.056871+0000","last_unstale":"2024-05-08T05:27:05.562544+0000","last_undegraded":"2024-05-08T05:27:05.562544+0000","last_fullsized":"2024-05-08T05:27:05.562544+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:19.004199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008120+0000","last_change":"2024-05-08T05:25:25.664795+0000","last_active":"2024-05-08T05:28:04.008120+0000","last_peered":"2024-05-08T05:28:04.008120+0000","last_clean":"2024-05-08T05:28:04.008120+0000","last_became_active":"2024-05-08T05:25:25.664444+0000","last_became_peered":"2024-05-08T05:25:25.664444+0000","last_unstale":"2024-05-08T05:28:04.008120+0000","last_undegraded":"2024-05-08T05:28:04.008120+0000","last_fullsized":"2024-05-08T05:28:04.008120+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:27.172530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561853+0000","last_change":"2024-05-08T05:25:22.645205+0000","last_active":"2024-05-08T05:27:05.561853+0000","last_peered":"2024-05-08T05:27:05.561853+0000","last_clean":"2024-05-08T05:27:05.561853+0000","last_became_active":"2024-05-08T05:25:22.644720+0000","last_became_peered":"2024-05-08T05:25:22.644720+0000","last_unstale":"2024-05-08T05:27:05.561853+0000","last_undegraded":"2024-05-08T05:27:05.561853+0000","last_fullsized":"2024-05-08T05:27:05.561853+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:05.630188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564454+0000","last_change":"2024-05-08T05:25:37.055151+0000","last_active":"2024-05-08T05:27:05.564454+0000","last_peered":"2024-05-08T05:27:05.564454+0000","last_clean":"2024-05-08T05:27:05.564454+0000","last_became_active":"2024-05-08T05:25:37.054737+0000","last_became_peered":"2024-05-08T05:25:37.054737+0000","last_unstale":"2024-05-08T05:27:05.564454+0000","last_undegraded":"2024-05-08T05:27:05.564454+0000","last_fullsized":"2024-05-08T05:27:05.564454+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:49.135453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":20,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.564959+0000","last_change":"2024-05-08T05:26:05.148665+0000","last_active":"2024-05-08T05:27:05.564959+0000","last_peered":"2024-05-08T05:27:05.564959+0000","last_clean":"2024-05-08T05:27:05.564959+0000","last_became_active":"2024-05-08T05:26:05.148283+0000","last_became_peered":"2024-05-08T05:26:05.148283+0000","last_unstale":"2024-05-08T05:27:05.564959+0000","last_undegraded":"2024-05-08T05:27:05.564959+0000","last_fullsized":"2024-05-08T05:27:05.564959+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:26.197521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:03.999698+0000","last_change":"2024-05-08T05:25:25.669130+0000","last_active":"2024-05-08T05:28:03.999698+0000","last_peered":"2024-05-08T05:28:03.999698+0000","last_clean":"2024-05-08T05:28:03.999698+0000","last_became_active":"2024-05-08T05:25:25.668812+0000","last_became_peered":"2024-05-08T05:25:25.668812+0000","last_unstale":"2024-05-08T05:28:03.999698+0000","last_undegraded":"2024-05-08T05:28:03.999698+0000","last_fullsized":"2024-05-08T05:28:03.999698+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:34.514248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562972+0000","last_change":"2024-05-08T05:25:37.057146+0000","last_active":"2024-05-08T05:27:05.562972+0000","last_peered":"2024-05-08T05:27:05.562972+0000","last_clean":"2024-05-08T05:27:05.562972+0000","last_became_active":"2024-05-08T05:25:37.056706+0000","last_became_peered":"2024-05-08T05:25:37.056706+0000","last_unstale":"2024-05-08T05:27:05.562972+0000","last_undegraded":"2024-05-08T05:27:05.562972+0000","last_fullsized":"2024-05-08T05:27:05.562972+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:09.724088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"90'16","reported_seq":60,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:31.022147+0000","last_change":"2024-05-08T05:25:22.644573+0000","last_active":"2024-05-08T05:27:31.022147+0000","last_peered":"2024-05-08T05:27:31.022147+0000","last_clean":"2024-05-08T05:27:31.022147+0000","last_became_active":"2024-05-08T05:25:22.644231+0000","last_became_peered":"2024-05-08T05:25:22.644231+0000","last_unstale":"2024-05-08T05:27:31.022147+0000","last_undegraded":"2024-05-08T05:27:31.022147+0000","last_fullsized":"2024-05-08T05:27:31.022147+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:02.931823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":29,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.021549+0000","last_change":"2024-05-08T05:25:25.671036+0000","last_active":"2024-05-08T05:28:04.021549+0000","last_peered":"2024-05-08T05:28:04.021549+0000","last_clean":"2024-05-08T05:28:04.021549+0000","last_became_active":"2024-05-08T05:25:25.669170+0000","last_became_peered":"2024-05-08T05:25:25.669170+0000","last_unstale":"2024-05-08T05:28:04.021549+0000","last_undegraded":"2024-05-08T05:28:04.021549+0000","last_fullsized":"2024-05-08T05:28:04.021549+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:18.963232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563674+0000","last_change":"2024-05-08T05:25:37.050656+0000","last_active":"2024-05-08T05:27:05.563674+0000","last_peered":"2024-05-08T05:27:05.563674+0000","last_clean":"2024-05-08T05:27:05.563674+0000","last_became_active":"2024-05-08T05:25:37.050051+0000","last_became_peered":"2024-05-08T05:25:37.050051+0000","last_unstale":"2024-05-08T05:27:05.563674+0000","last_undegraded":"2024-05-08T05:27:05.563674+0000","last_fullsized":"2024-05-08T05:27:05.563674+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:56:07.316216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":52,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765551+0000","last_change":"2024-05-08T05:25:23.614874+0000","last_active":"2024-05-08T05:27:05.765551+0000","last_peered":"2024-05-08T05:27:05.765551+0000","last_clean":"2024-05-08T05:27:05.765551+0000","last_became_active":"2024-05-08T05:25:23.614550+0000","last_became_peered":"2024-05-08T05:25:23.614550+0000","last_unstale":"2024-05-08T05:27:05.765551+0000","last_undegraded":"2024-05-08T05:27:05.765551+0000","last_fullsized":"2024-05-08T05:27:05.765551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:10.183477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020276+0000","last_change":"2024-05-08T05:25:25.658710+0000","last_active":"2024-05-08T05:28:04.020276+0000","last_peered":"2024-05-08T05:28:04.020276+0000","last_clean":"2024-05-08T05:28:04.020276+0000","last_became_active":"2024-05-08T05:25:25.658393+0000","last_became_peered":"2024-05-08T05:25:25.658393+0000","last_unstale":"2024-05-08T05:28:04.020276+0000","last_undegraded":"2024-05-08T05:28:04.020276+0000","last_fullsized":"2024-05-08T05:28:04.020276+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:02.629256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765630+0000","last_change":"2024-05-08T05:25:37.059976+0000","last_active":"2024-05-08T05:27:05.765630+0000","last_peered":"2024-05-08T05:27:05.765630+0000","last_clean":"2024-05-08T05:27:05.765630+0000","last_became_active":"2024-05-08T05:25:37.059474+0000","last_became_peered":"2024-05-08T05:25:37.059474+0000","last_unstale":"2024-05-08T05:27:05.765630+0000","last_undegraded":"2024-05-08T05:27:05.765630+0000","last_fullsized":"2024-05-08T05:27:05.765630+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:51.946242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764629+0000","last_change":"2024-05-08T05:25:22.748271+0000","last_active":"2024-05-08T05:27:05.764629+0000","last_peered":"2024-05-08T05:27:05.764629+0000","last_clean":"2024-05-08T05:27:05.764629+0000","last_became_active":"2024-05-08T05:25:22.747915+0000","last_became_peered":"2024-05-08T05:25:22.747915+0000","last_unstale":"2024-05-08T05:27:05.764629+0000","last_undegraded":"2024-05-08T05:27:05.764629+0000","last_fullsized":"2024-05-08T05:27:05.764629+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:35.262298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":41,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:03.972752+0000","last_change":"2024-05-08T05:25:25.668722+0000","last_active":"2024-05-08T05:28:03.972752+0000","last_peered":"2024-05-08T05:28:03.972752+0000","last_clean":"2024-05-08T05:28:03.972752+0000","last_became_active":"2024-05-08T05:25:25.668223+0000","last_became_peered":"2024-05-08T05:25:25.668223+0000","last_unstale":"2024-05-08T05:28:03.972752+0000","last_undegraded":"2024-05-08T05:28:03.972752+0000","last_fullsized":"2024-05-08T05:28:03.972752+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:43.511567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764581+0000","last_change":"2024-05-08T05:25:37.056131+0000","last_active":"2024-05-08T05:27:05.764581+0000","last_peered":"2024-05-08T05:27:05.764581+0000","last_clean":"2024-05-08T05:27:05.764581+0000","last_became_active":"2024-05-08T05:25:37.055424+0000","last_became_peered":"2024-05-08T05:25:37.055424+0000","last_unstale":"2024-05-08T05:27:05.764581+0000","last_undegraded":"2024-05-08T05:27:05.764581+0000","last_fullsized":"2024-05-08T05:27:05.764581+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:43.279408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563203+0000","last_change":"2024-05-08T05:25:22.643908+0000","last_active":"2024-05-08T05:27:05.563203+0000","last_peered":"2024-05-08T05:27:05.563203+0000","last_clean":"2024-05-08T05:27:05.563203+0000","last_became_active":"2024-05-08T05:25:22.643542+0000","last_became_peered":"2024-05-08T05:25:22.643542+0000","last_unstale":"2024-05-08T05:27:05.563203+0000","last_undegraded":"2024-05-08T05:27:05.563203+0000","last_fullsized":"2024-05-08T05:27:05.563203+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:01.671285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.014996+0000","last_change":"2024-05-08T05:25:25.667843+0000","last_active":"2024-05-08T05:28:04.014996+0000","last_peered":"2024-05-08T05:28:04.014996+0000","last_clean":"2024-05-08T05:28:04.014996+0000","last_became_active":"2024-05-08T05:25:25.667462+0000","last_became_peered":"2024-05-08T05:25:25.667462+0000","last_unstale":"2024-05-08T05:28:04.014996+0000","last_undegraded":"2024-05-08T05:28:04.014996+0000","last_fullsized":"2024-05-08T05:28:04.014996+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:51.325104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":28,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764958+0000","last_change":"2024-05-08T05:25:37.053017+0000","last_active":"2024-05-08T05:27:05.764958+0000","last_peered":"2024-05-08T05:27:05.764958+0000","last_clean":"2024-05-08T05:27:05.764958+0000","last_became_active":"2024-05-08T05:25:37.052601+0000","last_became_peered":"2024-05-08T05:25:37.052601+0000","last_unstale":"2024-05-08T05:27:05.764958+0000","last_undegraded":"2024-05-08T05:27:05.764958+0000","last_fullsized":"2024-05-08T05:27:05.764958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:35.393835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"88'3","reported_seq":47,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.563935+0000","last_change":"2024-05-08T05:25:22.644564+0000","last_active":"2024-05-08T05:27:05.563935+0000","last_peered":"2024-05-08T05:27:05.563935+0000","last_clean":"2024-05-08T05:27:05.563935+0000","last_became_active":"2024-05-08T05:25:22.644170+0000","last_became_peered":"2024-05-08T05:25:22.644170+0000","last_unstale":"2024-05-08T05:27:05.563935+0000","last_undegraded":"2024-05-08T05:27:05.563935+0000","last_fullsized":"2024-05-08T05:27:05.563935+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:24.757574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.021250+0000","last_change":"2024-05-08T05:25:25.676983+0000","last_active":"2024-05-08T05:28:04.021250+0000","last_peered":"2024-05-08T05:28:04.021250+0000","last_clean":"2024-05-08T05:28:04.021250+0000","last_became_active":"2024-05-08T05:25:25.676610+0000","last_became_peered":"2024-05-08T05:25:25.676610+0000","last_unstale":"2024-05-08T05:28:04.021250+0000","last_undegraded":"2024-05-08T05:28:04.021250+0000","last_fullsized":"2024-05-08T05:28:04.021250+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:39.137935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.765483+0000","last_change":"2024-05-08T05:25:37.059381+0000","last_active":"2024-05-08T05:27:05.765483+0000","last_peered":"2024-05-08T05:27:05.765483+0000","last_clean":"2024-05-08T05:27:05.765483+0000","last_became_active":"2024-05-08T05:25:37.056851+0000","last_became_peered":"2024-05-08T05:25:37.056851+0000","last_unstale":"2024-05-08T05:27:05.765483+0000","last_undegraded":"2024-05-08T05:27:05.765483+0000","last_fullsized":"2024-05-08T05:27:05.765483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:19.339331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.566848+0000","last_change":"2024-05-08T05:25:22.648099+0000","last_active":"2024-05-08T05:27:05.566848+0000","last_peered":"2024-05-08T05:27:05.566848+0000","last_clean":"2024-05-08T05:27:05.566848+0000","last_became_active":"2024-05-08T05:25:22.647736+0000","last_became_peered":"2024-05-08T05:25:22.647736+0000","last_unstale":"2024-05-08T05:27:05.566848+0000","last_undegraded":"2024-05-08T05:27:05.566848+0000","last_fullsized":"2024-05-08T05:27:05.566848+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:12.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":44,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.005434+0000","last_change":"2024-05-08T05:25:25.667565+0000","last_active":"2024-05-08T05:28:04.005434+0000","last_peered":"2024-05-08T05:28:04.005434+0000","last_clean":"2024-05-08T05:28:04.005434+0000","last_became_active":"2024-05-08T05:25:25.667217+0000","last_became_peered":"2024-05-08T05:25:25.667217+0000","last_unstale":"2024-05-08T05:28:04.005434+0000","last_undegraded":"2024-05-08T05:28:04.005434+0000","last_fullsized":"2024-05-08T05:28:04.005434+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:05.228227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.764997+0000","last_change":"2024-05-08T05:25:37.056923+0000","last_active":"2024-05-08T05:27:05.764997+0000","last_peered":"2024-05-08T05:27:05.764997+0000","last_clean":"2024-05-08T05:27:05.764997+0000","last_became_active":"2024-05-08T05:25:37.056496+0000","last_became_peered":"2024-05-08T05:25:37.056496+0000","last_unstale":"2024-05-08T05:27:05.764997+0000","last_undegraded":"2024-05-08T05:27:05.764997+0000","last_fullsized":"2024-05-08T05:27:05.764997+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:21.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"90'7","reported_seq":153,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.555739+0000","last_change":"2024-05-08T05:25:23.614900+0000","last_active":"2024-05-08T05:28:04.555739+0000","last_peered":"2024-05-08T05:28:04.555739+0000","last_clean":"2024-05-08T05:28:04.555739+0000","last_became_active":"2024-05-08T05:25:23.614510+0000","last_became_peered":"2024-05-08T05:25:23.614510+0000","last_unstale":"2024-05-08T05:28:04.555739+0000","last_undegraded":"2024-05-08T05:28:04.555739+0000","last_fullsized":"2024-05-08T05:28:04.555739+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:19.539656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":102,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.007662+0000","last_change":"2024-05-08T05:25:25.669016+0000","last_active":"2024-05-08T05:28:04.007662+0000","last_peered":"2024-05-08T05:28:04.007662+0000","last_clean":"2024-05-08T05:28:04.007662+0000","last_became_active":"2024-05-08T05:25:25.667809+0000","last_became_peered":"2024-05-08T05:25:25.667809+0000","last_unstale":"2024-05-08T05:28:04.007662+0000","last_undegraded":"2024-05-08T05:28:04.007662+0000","last_fullsized":"2024-05-08T05:28:04.007662+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:56.475622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":27,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.723728+0000","last_change":"2024-05-08T05:25:37.053630+0000","last_active":"2024-05-08T05:27:05.723728+0000","last_peered":"2024-05-08T05:27:05.723728+0000","last_clean":"2024-05-08T05:27:05.723728+0000","last_became_active":"2024-05-08T05:25:37.053069+0000","last_became_peered":"2024-05-08T05:25:37.053069+0000","last_unstale":"2024-05-08T05:27:05.723728+0000","last_undegraded":"2024-05-08T05:27:05.723728+0000","last_fullsized":"2024-05-08T05:27:05.723728+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:49:56.693447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"88'3","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.562317+0000","last_change":"2024-05-08T05:25:22.645284+0000","last_active":"2024-05-08T05:27:05.562317+0000","last_peered":"2024-05-08T05:27:05.562317+0000","last_clean":"2024-05-08T05:27:05.562317+0000","last_became_active":"2024-05-08T05:25:22.644897+0000","last_became_peered":"2024-05-08T05:25:22.644897+0000","last_unstale":"2024-05-08T05:27:05.562317+0000","last_undegraded":"2024-05-08T05:27:05.562317+0000","last_fullsized":"2024-05-08T05:27:05.562317+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:53:31.423412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.000490+0000","last_change":"2024-05-08T05:25:25.664930+0000","last_active":"2024-05-08T05:28:04.000490+0000","last_peered":"2024-05-08T05:28:04.000490+0000","last_clean":"2024-05-08T05:28:04.000490+0000","last_became_active":"2024-05-08T05:25:25.664395+0000","last_became_peered":"2024-05-08T05:25:25.664395+0000","last_unstale":"2024-05-08T05:28:04.000490+0000","last_undegraded":"2024-05-08T05:28:04.000490+0000","last_fullsized":"2024-05-08T05:28:04.000490+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:37.971474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":29,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.561719+0000","last_change":"2024-05-08T05:25:37.053258+0000","last_active":"2024-05-08T05:27:05.561719+0000","last_peered":"2024-05-08T05:27:05.561719+0000","last_clean":"2024-05-08T05:27:05.561719+0000","last_became_active":"2024-05-08T05:25:37.052786+0000","last_became_peered":"2024-05-08T05:25:37.052786+0000","last_unstale":"2024-05-08T05:27:05.561719+0000","last_undegraded":"2024-05-08T05:27:05.561719+0000","last_fullsized":"2024-05-08T05:27:05.561719+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:07.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":62,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:27:05.770795+0000","last_change":"2024-05-08T05:27:05.770358+0000","last_active":"2024-05-08T05:27:05.770795+0000","last_peered":"2024-05-08T05:27:05.770795+0000","last_clean":"2024-05-08T05:27:05.770795+0000","last_became_active":"2024-05-08T05:27:05.766778+0000","last_became_peered":"2024-05-08T05:27:05.766778+0000","last_unstale":"2024-05-08T05:27:05.770795+0000","last_undegraded":"2024-05-08T05:27:05.770795+0000","last_fullsized":"2024-05-08T05:27:05.770795+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:48:52.941549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":48,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.004619+0000","last_change":"2024-05-08T05:25:25.663771+0000","last_active":"2024-05-08T05:28:04.004619+0000","last_peered":"2024-05-08T05:28:04.004619+0000","last_clean":"2024-05-08T05:28:04.004619+0000","last_became_active":"2024-05-08T05:25:25.663281+0000","last_became_peered":"2024-05-08T05:25:25.663281+0000","last_unstale":"2024-05-08T05:28:04.004619+0000","last_undegraded":"2024-05-08T05:28:04.004619+0000","last_fullsized":"2024-05-08T05:28:04.004619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:43.892781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.005736+0000","last_change":"2024-05-08T05:25:25.665446+0000","last_active":"2024-05-08T05:28:04.005736+0000","last_peered":"2024-05-08T05:28:04.005736+0000","last_clean":"2024-05-08T05:28:04.005736+0000","last_became_active":"2024-05-08T05:25:25.664389+0000","last_became_peered":"2024-05-08T05:25:25.664389+0000","last_unstale":"2024-05-08T05:28:04.005736+0000","last_undegraded":"2024-05-08T05:28:04.005736+0000","last_fullsized":"2024-05-08T05:28:04.005736+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":56,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.010957+0000","last_change":"2024-05-08T05:26:05.149870+0000","last_active":"2024-05-08T05:28:04.010957+0000","last_peered":"2024-05-08T05:28:04.010957+0000","last_clean":"2024-05-08T05:28:04.010957+0000","last_became_active":"2024-05-08T05:26:05.149476+0000","last_became_peered":"2024-05-08T05:26:05.149476+0000","last_unstale":"2024-05-08T05:28:04.010957+0000","last_undegraded":"2024-05-08T05:28:04.010957+0000","last_fullsized":"2024-05-08T05:28:04.010957+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:40:43.401832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.013483+0000","last_change":"2024-05-08T05:25:25.667836+0000","last_active":"2024-05-08T05:28:04.013483+0000","last_peered":"2024-05-08T05:28:04.013483+0000","last_clean":"2024-05-08T05:28:04.013483+0000","last_became_active":"2024-05-08T05:25:25.667496+0000","last_became_peered":"2024-05-08T05:25:25.667496+0000","last_unstale":"2024-05-08T05:28:04.013483+0000","last_undegraded":"2024-05-08T05:28:04.013483+0000","last_fullsized":"2024-05-08T05:28:04.013483+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:01.482837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.011360+0000","last_change":"2024-05-08T05:25:25.667279+0000","last_active":"2024-05-08T05:28:04.011360+0000","last_peered":"2024-05-08T05:28:04.011360+0000","last_clean":"2024-05-08T05:28:04.011360+0000","last_became_active":"2024-05-08T05:25:25.666960+0000","last_became_peered":"2024-05-08T05:25:25.666960+0000","last_unstale":"2024-05-08T05:28:04.011360+0000","last_undegraded":"2024-05-08T05:28:04.011360+0000","last_fullsized":"2024-05-08T05:28:04.011360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:06.335850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.021057+0000","last_change":"2024-05-08T05:25:25.667644+0000","last_active":"2024-05-08T05:28:04.021057+0000","last_peered":"2024-05-08T05:28:04.021057+0000","last_clean":"2024-05-08T05:28:04.021057+0000","last_became_active":"2024-05-08T05:25:25.667133+0000","last_became_peered":"2024-05-08T05:25:25.667133+0000","last_unstale":"2024-05-08T05:28:04.021057+0000","last_undegraded":"2024-05-08T05:28:04.021057+0000","last_fullsized":"2024-05-08T05:28:04.021057+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:03.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":43,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.004619+0000","last_change":"2024-05-08T05:25:25.670761+0000","last_active":"2024-05-08T05:28:04.004619+0000","last_peered":"2024-05-08T05:28:04.004619+0000","last_clean":"2024-05-08T05:28:04.004619+0000","last_became_active":"2024-05-08T05:25:25.670342+0000","last_became_peered":"2024-05-08T05:25:25.670342+0000","last_unstale":"2024-05-08T05:28:04.004619+0000","last_undegraded":"2024-05-08T05:28:04.004619+0000","last_fullsized":"2024-05-08T05:28:04.004619+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:08.069332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":53,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.006746+0000","last_change":"2024-05-08T05:25:25.664512+0000","last_active":"2024-05-08T05:28:04.006746+0000","last_peered":"2024-05-08T05:28:04.006746+0000","last_clean":"2024-05-08T05:28:04.006746+0000","last_became_active":"2024-05-08T05:25:25.664191+0000","last_became_peered":"2024-05-08T05:25:25.664191+0000","last_unstale":"2024-05-08T05:28:04.006746+0000","last_undegraded":"2024-05-08T05:28:04.006746+0000","last_fullsized":"2024-05-08T05:28:04.006746+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:07.586637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":40,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020017+0000","last_change":"2024-05-08T05:25:25.675711+0000","last_active":"2024-05-08T05:28:04.020017+0000","last_peered":"2024-05-08T05:28:04.020017+0000","last_clean":"2024-05-08T05:28:04.020017+0000","last_became_active":"2024-05-08T05:25:25.671890+0000","last_became_peered":"2024-05-08T05:25:25.671890+0000","last_unstale":"2024-05-08T05:28:04.020017+0000","last_undegraded":"2024-05-08T05:28:04.020017+0000","last_fullsized":"2024-05-08T05:28:04.020017+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:57:49.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":45,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008300+0000","last_change":"2024-05-08T05:25:25.662603+0000","last_active":"2024-05-08T05:28:04.008300+0000","last_peered":"2024-05-08T05:28:04.008300+0000","last_clean":"2024-05-08T05:28:04.008300+0000","last_became_active":"2024-05-08T05:25:25.662287+0000","last_became_peered":"2024-05-08T05:25:25.662287+0000","last_unstale":"2024-05-08T05:28:04.008300+0000","last_undegraded":"2024-05-08T05:28:04.008300+0000","last_fullsized":"2024-05-08T05:28:04.008300+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:02.152557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":46,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020684+0000","last_change":"2024-05-08T05:25:25.667309+0000","last_active":"2024-05-08T05:28:04.020684+0000","last_peered":"2024-05-08T05:28:04.020684+0000","last_clean":"2024-05-08T05:28:04.020684+0000","last_became_active":"2024-05-08T05:25:25.666968+0000","last_became_peered":"2024-05-08T05:25:25.666968+0000","last_unstale":"2024-05-08T05:28:04.020684+0000","last_undegraded":"2024-05-08T05:28:04.020684+0000","last_fullsized":"2024-05-08T05:28:04.020684+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:28.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":49,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020720+0000","last_change":"2024-05-08T05:25:25.673579+0000","last_active":"2024-05-08T05:28:04.020720+0000","last_peered":"2024-05-08T05:28:04.020720+0000","last_clean":"2024-05-08T05:28:04.020720+0000","last_became_active":"2024-05-08T05:25:25.673130+0000","last_became_peered":"2024-05-08T05:25:25.673130+0000","last_unstale":"2024-05-08T05:28:04.020720+0000","last_undegraded":"2024-05-08T05:28:04.020720+0000","last_fullsized":"2024-05-08T05:28:04.020720+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:01.291319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":50,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.020138+0000","last_change":"2024-05-08T05:25:25.667359+0000","last_active":"2024-05-08T05:28:04.020138+0000","last_peered":"2024-05-08T05:28:04.020138+0000","last_clean":"2024-05-08T05:28:04.020138+0000","last_became_active":"2024-05-08T05:25:25.667044+0000","last_became_peered":"2024-05-08T05:25:25.667044+0000","last_unstale":"2024-05-08T05:28:04.020138+0000","last_undegraded":"2024-05-08T05:28:04.020138+0000","last_fullsized":"2024-05-08T05:28:04.020138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:54.973053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":51,"reported_epoch":90,"state":"active+clean","last_fresh":"2024-05-08T05:28:04.008877+0000","last_change":"2024-05-08T05:26:05.150407+0000","last_active":"2024-05-08T05:28:04.008877+0000","last_peered":"2024-05-08T05:28:04.008877+0000","last_clean":"2024-05-08T05:28:04.008877+0000","last_became_active":"2024-05-08T05:26:05.149916+0000","last_became_peered":"2024-05-08T05:26:05.149916+0000","last_unstale":"2024-05-08T05:28:04.008877+0000","last_undegraded":"2024-05-08T05:28:04.008877+0000","last_fullsized":"2024-05-08T05:28:04.008877+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:05:57.075928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":376832,"data_stored":4132576,"data_compressed":247352,"data_compressed_allocated":315392,"data_compressed_original":4096000,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":73444,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":1037,"num_read_kb":605,"num_write":264,"num_write_kb":157,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":220332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":228,"ondisk_log_size":228,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678064,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28000,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717152,"statfs":{"total":95995035648,"available":95966363648,"internally_reserved":0,"allocated":1007616,"data_stored":521664,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874294,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28140,"kb_used_data":1124,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717012,"statfs":{"total":95995035648,"available":95966220288,"internally_reserved":0,"allocated":1150976,"data_stored":1173262,"data_compressed":68262,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037820,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28020,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717132,"statfs":{"total":95995035648,"available":95966343168,"internally_reserved":0,"allocated":1028096,"data_stored":525046,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234049,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":521216,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462980,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28064,"kb_used_data":1048,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717088,"statfs":{"total":95995035648,"available":95966298112,"internally_reserved":0,"allocated":1073152,"data_stored":1119030,"data_compressed":68262,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626507,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":516986,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574948,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1044,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1069056,"data_stored":582910,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738472,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":518442,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411423,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36392,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708760,"statfs":{"total":95995035648,"available":95957770240,"internally_reserved":0,"allocated":1081344,"data_stored":583550,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182490,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32164,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712988,"statfs":{"total":95995035648,"available":95962099712,"internally_reserved":0,"allocated":1011712,"data_stored":516162,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986261,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32168,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712984,"statfs":{"total":95995035648,"available":95962095616,"internally_reserved":0,"allocated":1015808,"data_stored":521172,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822735,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32232,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712920,"statfs":{"total":95995035648,"available":95962030080,"internally_reserved":0,"allocated":1081344,"data_stored":1113548,"data_compressed":68262,"data_compressed_allocated":90112,"data_compressed_original":679936,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":590368,"data_compressed":34176,"data_compressed_allocated":45056,"data_compressed_original":573440,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":86016,"data_stored":67178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":67818,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1344,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":4738,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1388,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":67162,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1836,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:28:11.923 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T05:28:11.923 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-05-08T05:28:11.923 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-05-08T05:28:11.924 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph health --format=json 2024-05-08T05:28:12.179 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:12.576 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:12 smithi059 ceph-mon[30304]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:28:12.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:12 smithi143 ceph-mon[34337]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:28:12.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:12 smithi096 ceph-mon[34253]: from='client.14985 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:28:13.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:13 smithi143 ceph-mon[34337]: pgmap v384: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:13.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:13 smithi096 ceph-mon[34253]: pgmap v384: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:13.780 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:13 smithi059 ceph-mon[30304]: pgmap v384: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.014+0000 7ff0da34a700 1 -- 172.21.15.59:0/2812056450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d4109510 msgr2=0x7ff0d41098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.014+0000 7ff0da34a700 1 --2- 172.21.15.59:0/2812056450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d4109510 0x7ff0d41098f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4009990 tx=0x7ff0c4009ca0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.014+0000 7ff0da34a700 1 -- 172.21.15.59:0/2812056450 shutdown_connections 2024-05-08T05:28:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.014+0000 7ff0da34a700 1 --2- 172.21.15.59:0/2812056450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d406bb40 0x7ff0d4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.014+0000 7ff0da34a700 1 --2- 172.21.15.59:0/2812056450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d4109510 0x7ff0d41098f0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.014+0000 7ff0da34a700 1 --2- 172.21.15.59:0/2812056450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d406b180 0x7ff0d406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.014+0000 7ff0da34a700 1 -- 172.21.15.59:0/2812056450 >> 172.21.15.59:0/2812056450 conn(0x7ff0d4079350 msgr2=0x7ff0d4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.014+0000 7ff0da34a700 1 -- 172.21.15.59:0/2812056450 shutdown_connections 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.014+0000 7ff0da34a700 1 -- 172.21.15.59:0/2812056450 wait complete. 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.015+0000 7ff0da34a700 1 Processor -- start 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.015+0000 7ff0da34a700 1 -- start start 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.015+0000 7ff0da34a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d406b180 0x7ff0d419f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.015+0000 7ff0da34a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d406bb40 0x7ff0d419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.015+0000 7ff0da34a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d4109510 0x7ff0d4199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.015+0000 7ff0da34a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d4113640 con 0x7ff0d406b180 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.015+0000 7ff0da34a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41134c0 con 0x7ff0d4109510 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.015+0000 7ff0da34a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41137c0 con 0x7ff0d406bb40 2024-05-08T05:28:14.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d406b180 0x7ff0d419f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d406b180 0x7ff0d419f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39298/0 (socket says 172.21.15.59:39298) 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d3fff700 1 -- 172.21.15.59:0/3577826687 learned_addr learned my addr 172.21.15.59:0/3577826687 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d88e7700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d4109510 0x7ff0d4199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d37fe700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d406bb40 0x7ff0d419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d3fff700 1 -- 172.21.15.59:0/3577826687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d406bb40 msgr2=0x7ff0d419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d3fff700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d406bb40 0x7ff0d419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d3fff700 1 -- 172.21.15.59:0/3577826687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d4109510 msgr2=0x7ff0d4199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d3fff700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d4109510 0x7ff0d4199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d3fff700 1 -- 172.21.15.59:0/3577826687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d419a0e0 con 0x7ff0d406b180 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d3fff700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d406b180 0x7ff0d419f7c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000c00 tx=0x7ff0c40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d17fa700 1 -- 172.21.15.59:0/3577826687 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c4027070 con 0x7ff0d406b180 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d17fa700 1 -- 172.21.15.59:0/3577826687 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c40041d0 con 0x7ff0d406b180 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d419a370 con 0x7ff0d406b180 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0d17fa700 1 -- 172.21.15.59:0/3577826687 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400f460 con 0x7ff0d406b180 2024-05-08T05:28:14.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.016+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d41a6690 con 0x7ff0d406b180 2024-05-08T05:28:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.017+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0d410b9f0 con 0x7ff0d406b180 2024-05-08T05:28:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.019+0000 7ff0d17fa700 1 -- 172.21.15.59:0/3577826687 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7ff0c402a070 con 0x7ff0d406b180 2024-05-08T05:28:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.019+0000 7ff0d17fa700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff0c00b2880 0x7ff0c00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.020+0000 7ff0d37fe700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff0c00b2880 0x7ff0c00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.020+0000 7ff0d17fa700 1 -- 172.21.15.59:0/3577826687 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7ff0c40e39e0 con 0x7ff0d406b180 2024-05-08T05:28:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.020+0000 7ff0d37fe700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff0c00b2880 0x7ff0c00b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4079a30 tx=0x7ff0bc005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:14.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.023+0000 7ff0d17fa700 1 -- 172.21.15.59:0/3577826687 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff0c4031070 con 0x7ff0d406b180 2024-05-08T05:28:14.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.284+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ff0d406a190 con 0x7ff0d406b180 2024-05-08T05:28:14.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.284+0000 7ff0d17fa700 1 -- 172.21.15.59:0/3577826687 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ff0c4013070 con 0x7ff0d406b180 2024-05-08T05:28:14.285 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:14.285 INFO:teuthology.orchestra.run.smithi059.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-05-08T05:28:14.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff0c00b2880 msgr2=0x7ff0c00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:14.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff0c00b2880 0x7ff0c00b4d40 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7ff0d4079a30 tx=0x7ff0bc005eb0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d406b180 msgr2=0x7ff0d419f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:14.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d406b180 0x7ff0d419f7c0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7ff0c4000c00 tx=0x7ff0c40047b0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 shutdown_connections 2024-05-08T05:28:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7ff0c00b2880 0x7ff0c00b4d40 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff0d4109510 0x7ff0d4199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff0d406b180 0x7ff0d419f7c0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 --2- 172.21.15.59:0/3577826687 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff0d406bb40 0x7ff0d419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 >> 172.21.15.59:0/3577826687 conn(0x7ff0d4079350 msgr2=0x7ff0d41010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 shutdown_connections 2024-05-08T05:28:14.288 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:14.287+0000 7ff0da34a700 1 -- 172.21.15.59:0/3577826687 wait complete. 2024-05-08T05:28:14.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:14 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3577826687' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T05:28:14.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:14 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3577826687' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T05:28:14.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:14 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3577826687' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-05-08T05:28:15.016 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-05-08T05:28:15.016 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:28:15.274 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:15.652 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:15 smithi059 ceph-mon[30304]: pgmap v385: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:15.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:15 smithi143 ceph-mon[34337]: pgmap v385: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:15.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:15 smithi096 ceph-mon[34253]: pgmap v385: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.086+0000 7fadc400c700 1 -- 172.21.15.59:0/2808929049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 msgr2=0x7fadbc102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.086+0000 7fadc400c700 1 --2- 172.21.15.59:0/2808929049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 0x7fadbc102820 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fadac009990 tx=0x7fadac009ca0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.086+0000 7fadc400c700 1 -- 172.21.15.59:0/2808929049 shutdown_connections 2024-05-08T05:28:17.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.086+0000 7fadc400c700 1 --2- 172.21.15.59:0/2808929049 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc102d60 0x7fadbc10f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.086+0000 7fadc400c700 1 --2- 172.21.15.59:0/2808929049 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadbc108410 0x7fadbc1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.086+0000 7fadc400c700 1 --2- 172.21.15.59:0/2808929049 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 0x7fadbc102820 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.086+0000 7fadc400c700 1 -- 172.21.15.59:0/2808929049 >> 172.21.15.59:0/2808929049 conn(0x7fadbc079350 msgr2=0x7fadbc079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.086+0000 7fadc400c700 1 -- 172.21.15.59:0/2808929049 shutdown_connections 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.086+0000 7fadc400c700 1 -- 172.21.15.59:0/2808929049 wait complete. 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc400c700 1 Processor -- start 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc400c700 1 -- start start 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc400c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 0x7fadbc19e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc400c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadbc102d60 0x7fadbc19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc400c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc108410 0x7fadbc198970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc400c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc1125b0 con 0x7fadbc1023a0 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc400c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc112430 con 0x7fadbc108410 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc400c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fadbc112730 con 0x7fadbc102d60 2024-05-08T05:28:17.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc1da8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 0x7fadbc19e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc1da8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 0x7fadbc19e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39316/0 (socket says 172.21.15.59:39316) 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc1da8700 1 -- 172.21.15.59:0/3872763472 learned_addr learned my addr 172.21.15.59:0/3872763472 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc15a7700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadbc102d60 0x7fadbc19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc25a9700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc108410 0x7fadbc198970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc1da8700 1 -- 172.21.15.59:0/3872763472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadbc102d60 msgr2=0x7fadbc19eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc1da8700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadbc102d60 0x7fadbc19eca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc1da8700 1 -- 172.21.15.59:0/3872763472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc108410 msgr2=0x7fadbc198970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc1da8700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc108410 0x7fadbc198970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.087+0000 7fadc1da8700 1 -- 172.21.15.59:0/3872763472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadbc1990d0 con 0x7fadbc1023a0 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.088+0000 7fadc25a9700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc108410 0x7fadbc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.088+0000 7fadc1da8700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 0x7fadbc19e760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fadb800ba50 tx=0x7fadb800bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.088+0000 7fadb2ffd700 1 -- 172.21.15.59:0/3872763472 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb8019070 con 0x7fadbc1023a0 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.088+0000 7fadb2ffd700 1 -- 172.21.15.59:0/3872763472 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fadb8014440 con 0x7fadbc1023a0 2024-05-08T05:28:17.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.088+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadbc1993c0 con 0x7fadbc1023a0 2024-05-08T05:28:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.088+0000 7fadb2ffd700 1 -- 172.21.15.59:0/3872763472 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fadb800ce60 con 0x7fadbc1023a0 2024-05-08T05:28:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.088+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadbc1a5630 con 0x7fadbc1023a0 2024-05-08T05:28:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.089+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fadbc103ac0 con 0x7fadbc1023a0 2024-05-08T05:28:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.090+0000 7fadb2ffd700 1 -- 172.21.15.59:0/3872763472 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fadb801c070 con 0x7fadbc1023a0 2024-05-08T05:28:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.091+0000 7fadb2ffd700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fada80b2900 0x7fada80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.091+0000 7fadc15a7700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fada80b2900 0x7fada80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.091+0000 7fadb2ffd700 1 -- 172.21.15.59:0/3872763472 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fadb80e1db0 con 0x7fadbc1023a0 2024-05-08T05:28:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.091+0000 7fadc15a7700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fada80b2900 0x7fada80b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fadac00b5c0 tx=0x7fadac005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:17.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.094+0000 7fadb2ffd700 1 -- 172.21.15.59:0/3872763472 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fadb80a9740 con 0x7fadbc1023a0 2024-05-08T05:28:17.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.358+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fadbc199e50 con 0x7fadbc1023a0 2024-05-08T05:28:17.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.358+0000 7fadb2ffd700 1 -- 172.21.15.59:0/3872763472 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7fadb80a9040 con 0x7fadbc1023a0 2024-05-08T05:28:17.359 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:17.359 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:26:31.316568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:28:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fada80b2900 msgr2=0x7fada80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fada80b2900 0x7fada80b4dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fadac00b5c0 tx=0x7fadac005960 comp rx=0 tx=0).stop 2024-05-08T05:28:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 msgr2=0x7fadbc19e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 0x7fadbc19e760 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fadb800ba50 tx=0x7fadb800bd60 comp rx=0 tx=0).stop 2024-05-08T05:28:17.361 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 shutdown_connections 2024-05-08T05:28:17.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fada80b2900 0x7fada80b4dc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fadbc108410 0x7fadbc198970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fadbc1023a0 0x7fadbc19e760 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 --2- 172.21.15.59:0/3872763472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fadbc102d60 0x7fadbc19eca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:17.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 >> 172.21.15.59:0/3872763472 conn(0x7fadbc079350 msgr2=0x7fadbc100f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:17.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 shutdown_connections 2024-05-08T05:28:17.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:17.361+0000 7fadc400c700 1 -- 172.21.15.59:0/3872763472 wait complete. 2024-05-08T05:28:17.362 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 29 2024-05-08T05:28:17.732 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:17 smithi059 ceph-mon[30304]: pgmap v386: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:17.732 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:17 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3872763472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:28:17.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:17 smithi143 ceph-mon[34337]: pgmap v386: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:17.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:17 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3872763472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:28:17.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:17 smithi096 ceph-mon[34253]: pgmap v386: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 938 B/s wr, 11 op/s 2024-05-08T05:28:17.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:17 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3872763472' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:28:18.166 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:28:18.422 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:19.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:19 smithi143 ceph-mon[34337]: pgmap v387: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-08T05:28:19.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:19 smithi096 ceph-mon[34253]: pgmap v387: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-08T05:28:19.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:19 smithi059 ceph-mon[30304]: pgmap v387: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-08T05:28:20.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.219+0000 7fba5a465700 1 -- 172.21.15.59:0/1318559429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54108850 msgr2=0x7fba541103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.219+0000 7fba5a465700 1 --2- 172.21.15.59:0/1318559429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54108850 0x7fba541103d0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fba4c00b0d0 tx=0x7fba4c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.219+0000 7fba5a465700 1 -- 172.21.15.59:0/1318559429 shutdown_connections 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.219+0000 7fba5a465700 1 --2- 172.21.15.59:0/1318559429 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54108850 0x7fba541103d0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.219+0000 7fba5a465700 1 --2- 172.21.15.59:0/1318559429 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba54103750 0x7fba54103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.219+0000 7fba5a465700 1 --2- 172.21.15.59:0/1318559429 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba54104100 0x7fba54108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.219+0000 7fba5a465700 1 -- 172.21.15.59:0/1318559429 >> 172.21.15.59:0/1318559429 conn(0x7fba5407a600 msgr2=0x7fba5407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.219+0000 7fba5a465700 1 -- 172.21.15.59:0/1318559429 shutdown_connections 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.219+0000 7fba5a465700 1 -- 172.21.15.59:0/1318559429 wait complete. 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba5a465700 1 Processor -- start 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba5a465700 1 -- start start 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba5a465700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba54103750 0x7fba5419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:20.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba5a465700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54104100 0x7fba5419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba5a465700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba54108850 0x7fba54199900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba5a465700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fba54113640 con 0x7fba54104100 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba5a465700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fba541134c0 con 0x7fba54103750 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba5a465700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fba541137c0 con 0x7fba54108850 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba537fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54104100 0x7fba5419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba537fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54104100 0x7fba5419fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41570/0 (socket says 172.21.15.59:41570) 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba537fe700 1 -- 172.21.15.59:0/3016822461 learned_addr learned my addr 172.21.15.59:0/3016822461 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba53fff700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba54103750 0x7fba5419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.220+0000 7fba58a02700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba54108850 0x7fba54199900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba537fe700 1 -- 172.21.15.59:0/3016822461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba54108850 msgr2=0x7fba54199900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba537fe700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba54108850 0x7fba54199900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba537fe700 1 -- 172.21.15.59:0/3016822461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba54103750 msgr2=0x7fba5419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba537fe700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba54103750 0x7fba5419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba537fe700 1 -- 172.21.15.59:0/3016822461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba5419a060 con 0x7fba54104100 2024-05-08T05:28:20.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba53fff700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba54103750 0x7fba5419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:20.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba537fe700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54104100 0x7fba5419fc80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fba4800d980 tx=0x7fba4800dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:20.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba517fa700 1 -- 172.21.15.59:0/3016822461 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48017070 con 0x7fba54104100 2024-05-08T05:28:20.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba517fa700 1 -- 172.21.15.59:0/3016822461 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba48004830 con 0x7fba54104100 2024-05-08T05:28:20.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba5419a350 con 0x7fba54104100 2024-05-08T05:28:20.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba517fa700 1 -- 172.21.15.59:0/3016822461 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba48005440 con 0x7fba54104100 2024-05-08T05:28:20.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.221+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba541a6610 con 0x7fba54104100 2024-05-08T05:28:20.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.222+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba5406a190 con 0x7fba54104100 2024-05-08T05:28:20.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.223+0000 7fba517fa700 1 -- 172.21.15.59:0/3016822461 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fba480055e0 con 0x7fba54104100 2024-05-08T05:28:20.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.224+0000 7fba517fa700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fba3c0b2880 0x7fba3c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:20.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.224+0000 7fba53fff700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fba3c0b2880 0x7fba3c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:20.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.224+0000 7fba517fa700 1 -- 172.21.15.59:0/3016822461 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fba480e24f0 con 0x7fba54104100 2024-05-08T05:28:20.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.225+0000 7fba53fff700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fba3c0b2880 0x7fba3c0b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba5419aa40 tx=0x7fba44009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:20.229 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.229+0000 7fba517fa700 1 -- 172.21.15.59:0/3016822461 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fba48020020 con 0x7fba54104100 2024-05-08T05:28:20.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.499+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fba5402e510 con 0x7fba54104100 2024-05-08T05:28:20.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.500+0000 7fba517fa700 1 -- 172.21.15.59:0/3016822461 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7fba480a9e20 con 0x7fba54104100 2024-05-08T05:28:20.501 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:20.501 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:26:31.316568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:28:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.503+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fba3c0b2880 msgr2=0x7fba3c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.503+0000 7fba5a465700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fba3c0b2880 0x7fba3c0b4d40 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fba5419aa40 tx=0x7fba44009260 comp rx=0 tx=0).stop 2024-05-08T05:28:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.503+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54104100 msgr2=0x7fba5419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.503+0000 7fba5a465700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54104100 0x7fba5419fc80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fba4800d980 tx=0x7fba4800dd40 comp rx=0 tx=0).stop 2024-05-08T05:28:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.504+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 shutdown_connections 2024-05-08T05:28:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.504+0000 7fba5a465700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fba54108850 0x7fba54199900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.504+0000 7fba5a465700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fba3c0b2880 0x7fba3c0b4d40 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.504+0000 7fba5a465700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fba54103750 0x7fba5419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.504+0000 7fba5a465700 1 --2- 172.21.15.59:0/3016822461 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fba54104100 0x7fba5419fc80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.504+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 >> 172.21.15.59:0/3016822461 conn(0x7fba5407a600 msgr2=0x7fba54101540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.504+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 shutdown_connections 2024-05-08T05:28:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:20.504+0000 7fba5a465700 1 -- 172.21.15.59:0/3016822461 wait complete. 2024-05-08T05:28:20.505 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 29 2024-05-08T05:28:21.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:28:21.474 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:21.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:21 smithi059 ceph-mon[30304]: pgmap v388: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-08T05:28:21.739 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:21 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3016822461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:28:21.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:21 smithi143 ceph-mon[34337]: pgmap v388: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-08T05:28:21.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:21 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3016822461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:28:21.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:21 smithi096 ceph-mon[34253]: pgmap v388: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 511 B/s wr, 11 op/s 2024-05-08T05:28:21.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:21 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3016822461' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:28:23.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.346+0000 7fa134726700 1 -- 172.21.15.59:0/3382759242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c109510 msgr2=0x7fa12c1098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:23.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.346+0000 7fa134726700 1 --2- 172.21.15.59:0/3382759242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c109510 0x7fa12c1098f0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fa128009990 tx=0x7fa128009ca0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.346+0000 7fa134726700 1 -- 172.21.15.59:0/3382759242 shutdown_connections 2024-05-08T05:28:23.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.346+0000 7fa134726700 1 --2- 172.21.15.59:0/3382759242 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa12c06bb40 0x7fa12c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.346+0000 7fa134726700 1 --2- 172.21.15.59:0/3382759242 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c109510 0x7fa12c1098f0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.346+0000 7fa134726700 1 --2- 172.21.15.59:0/3382759242 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12c06b180 0x7fa12c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.346+0000 7fa134726700 1 -- 172.21.15.59:0/3382759242 >> 172.21.15.59:0/3382759242 conn(0x7fa12c079350 msgr2=0x7fa12c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:23.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.346+0000 7fa134726700 1 -- 172.21.15.59:0/3382759242 shutdown_connections 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.346+0000 7fa134726700 1 -- 172.21.15.59:0/3382759242 wait complete. 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa134726700 1 Processor -- start 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa134726700 1 -- start start 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa134726700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12c06b180 0x7fa12c19f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa134726700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c06bb40 0x7fa12c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa134726700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa12c109510 0x7fa12c1999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa134726700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c113640 con 0x7fa12c06bb40 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa134726700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1134c0 con 0x7fa12c109510 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa134726700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa12c1137c0 con 0x7fa12c06b180 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa131cc1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c06bb40 0x7fa12c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:23.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa131cc1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c06bb40 0x7fa12c19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41582/0 (socket says 172.21.15.59:41582) 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.347+0000 7fa131cc1700 1 -- 172.21.15.59:0/1964504732 learned_addr learned my addr 172.21.15.59:0/1964504732 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa1324c2700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12c06b180 0x7fa12c19f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa132cc3700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa12c109510 0x7fa12c1999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa131cc1700 1 -- 172.21.15.59:0/1964504732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12c06b180 msgr2=0x7fa12c19f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa131cc1700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12c06b180 0x7fa12c19f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa131cc1700 1 -- 172.21.15.59:0/1964504732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa12c109510 msgr2=0x7fa12c1999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa131cc1700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa12c109510 0x7fa12c1999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa131cc1700 1 -- 172.21.15.59:0/1964504732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa12c19a100 con 0x7fa12c06bb40 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa132cc3700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa12c109510 0x7fa12c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa131cc1700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c06bb40 0x7fa12c19fd20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00d980 tx=0x7fa11c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa1237fe700 1 -- 172.21.15.59:0/1964504732 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c017070 con 0x7fa12c06bb40 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa1237fe700 1 -- 172.21.15.59:0/1964504732 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa11c005020 con 0x7fa12c06bb40 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa12c19a3f0 con 0x7fa12c06bb40 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa1237fe700 1 -- 172.21.15.59:0/1964504732 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa11c009b50 con 0x7fa12c06bb40 2024-05-08T05:28:23.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.348+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa12c1a66b0 con 0x7fa12c06bb40 2024-05-08T05:28:23.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.349+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa12c10b9f0 con 0x7fa12c06bb40 2024-05-08T05:28:23.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.351+0000 7fa1237fe700 1 -- 172.21.15.59:0/1964504732 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa11c0051d0 con 0x7fa12c06bb40 2024-05-08T05:28:23.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.352+0000 7fa1237fe700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa1180b2880 0x7fa1180b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:23.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.352+0000 7fa1324c2700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa1180b2880 0x7fa1180b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:23.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.352+0000 7fa1237fe700 1 -- 172.21.15.59:0/1964504732 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa11c0e2930 con 0x7fa12c06bb40 2024-05-08T05:28:23.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.353+0000 7fa1324c2700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa1180b2880 0x7fa1180b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa12800b5c0 tx=0x7fa128005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:23.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.356+0000 7fa1237fe700 1 -- 172.21.15.59:0/1964504732 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa11c020020 con 0x7fa12c06bb40 2024-05-08T05:28:23.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.573+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa12c10b3d0 con 0x7fa12c06bb40 2024-05-08T05:28:23.574 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:23 smithi059 ceph-mon[30304]: pgmap v389: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:23.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.575+0000 7fa1237fe700 1 -- 172.21.15.59:0/1964504732 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24695 (secure 0 0 0) 0x7fa11c0aa2c0 con 0x7fa12c06bb40 2024-05-08T05:28:23.576 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:23.576 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:27:05.550089+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:28:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa1180b2880 msgr2=0x7fa1180b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa1180b2880 0x7fa1180b4d40 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa12800b5c0 tx=0x7fa128005960 comp rx=0 tx=0).stop 2024-05-08T05:28:23.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c06bb40 msgr2=0x7fa12c19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c06bb40 0x7fa12c19fd20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa11c00d980 tx=0x7fa11c00dc90 comp rx=0 tx=0).stop 2024-05-08T05:28:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 shutdown_connections 2024-05-08T05:28:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa1180b2880 0x7fa1180b4d40 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa12c109510 0x7fa12c1999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa12c06b180 0x7fa12c19f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 --2- 172.21.15.59:0/1964504732 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa12c06bb40 0x7fa12c19fd20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 >> 172.21.15.59:0/1964504732 conn(0x7fa12c079350 msgr2=0x7fa12c1010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 shutdown_connections 2024-05-08T05:28:23.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:23.577+0000 7fa134726700 1 -- 172.21.15.59:0/1964504732 wait complete. 2024-05-08T05:28:23.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:23 smithi143 ceph-mon[34337]: pgmap v389: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:23.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:23 smithi096 ceph-mon[34253]: pgmap v389: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:24.377 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:28:24.633 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:24.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:24 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1964504732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:28:24.757 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:24 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1964504732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:28:24.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:24 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1964504732' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:28:25.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:25 smithi059 ceph-mon[30304]: pgmap v390: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:25 smithi143 ceph-mon[34337]: pgmap v390: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:25.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:25 smithi096 ceph-mon[34253]: pgmap v390: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:26.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.565+0000 7f675a45b700 1 -- 172.21.15.59:0/2634421725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 msgr2=0x7f67541103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.565+0000 7f675a45b700 1 --2- 172.21.15.59:0/2634421725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 0x7f67541103d0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f674c00b0d0 tx=0x7f674c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.565+0000 7f675a45b700 1 -- 172.21.15.59:0/2634421725 shutdown_connections 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.565+0000 7f675a45b700 1 --2- 172.21.15.59:0/2634421725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 0x7f67541103d0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.565+0000 7f675a45b700 1 --2- 172.21.15.59:0/2634421725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6754103750 0x7f6754103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.565+0000 7f675a45b700 1 --2- 172.21.15.59:0/2634421725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6754104100 0x7f6754108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.565+0000 7f675a45b700 1 -- 172.21.15.59:0/2634421725 >> 172.21.15.59:0/2634421725 conn(0x7f675407a600 msgr2=0x7f675407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.566+0000 7f675a45b700 1 -- 172.21.15.59:0/2634421725 shutdown_connections 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.566+0000 7f675a45b700 1 -- 172.21.15.59:0/2634421725 wait complete. 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.566+0000 7f675a45b700 1 Processor -- start 2024-05-08T05:28:26.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.566+0000 7f675a45b700 1 -- start start 2024-05-08T05:28:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.566+0000 7f675a45b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6754103750 0x7f675419f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f675a45b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6754104100 0x7f675419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f675a45b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 0x7f6754199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f675a45b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f67541136a0 con 0x7f6754108850 2024-05-08T05:28:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f675a45b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6754113520 con 0x7f6754103750 2024-05-08T05:28:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f675a45b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6754113820 con 0x7f6754104100 2024-05-08T05:28:26.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f67589f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 0x7f6754199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f67589f8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 0x7f6754199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41602/0 (socket says 172.21.15.59:41602) 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f67589f8700 1 -- 172.21.15.59:0/57945483 learned_addr learned my addr 172.21.15.59:0/57945483 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f674bfff700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6754104100 0x7f675419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f6753fff700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6754103750 0x7f675419f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f67589f8700 1 -- 172.21.15.59:0/57945483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6754104100 msgr2=0x7f675419fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f67589f8700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6754104100 0x7f675419fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f67589f8700 1 -- 172.21.15.59:0/57945483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6754103750 msgr2=0x7f675419f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f67589f8700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6754103750 0x7f675419f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f67589f8700 1 -- 172.21.15.59:0/57945483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675419a0c0 con 0x7f6754108850 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.567+0000 7f67589f8700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 0x7f6754199960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f674c0095c0 tx=0x7f674c007a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.568+0000 7f6751ffb700 1 -- 172.21.15.59:0/57945483 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c01f070 con 0x7f6754108850 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.568+0000 7f6751ffb700 1 -- 172.21.15.59:0/57945483 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f674c007e20 con 0x7f6754108850 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.568+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f675419a350 con 0x7f6754108850 2024-05-08T05:28:26.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.568+0000 7f6751ffb700 1 -- 172.21.15.59:0/57945483 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674c01b4a0 con 0x7f6754108850 2024-05-08T05:28:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.568+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67541a6620 con 0x7f6754108850 2024-05-08T05:28:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.569+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f675406a190 con 0x7f6754108850 2024-05-08T05:28:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.570+0000 7f6751ffb700 1 -- 172.21.15.59:0/57945483 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f674c022070 con 0x7f6754108850 2024-05-08T05:28:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.571+0000 7f6751ffb700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f67400b2880 0x7f67400b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.571+0000 7f6753fff700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f67400b2880 0x7f67400b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:26.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.571+0000 7f6751ffb700 1 -- 172.21.15.59:0/57945483 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f674c00f020 con 0x7f6754108850 2024-05-08T05:28:26.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.571+0000 7f6753fff700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f67400b2880 0x7f67400b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6754108350 tx=0x7f6744009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:26.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.576+0000 7f6751ffb700 1 -- 172.21.15.59:0/57945483 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f674c029020 con 0x7f6754108850 2024-05-08T05:28:26.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.847+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f675410ba90 con 0x7f6754108850 2024-05-08T05:28:26.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.847+0000 7f6751ffb700 1 -- 172.21.15.59:0/57945483 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7f674c0ab1a0 con 0x7f6754108850 2024-05-08T05:28:26.848 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:26.848 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:26:31.316568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:28:26.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f67400b2880 msgr2=0x7f67400b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f67400b2880 0x7f67400b4d40 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f6754108350 tx=0x7f6744009190 comp rx=0 tx=0).stop 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 msgr2=0x7f6754199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 0x7f6754199960 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f674c0095c0 tx=0x7f674c007a20 comp rx=0 tx=0).stop 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 shutdown_connections 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6754108850 0x7f6754199960 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f67400b2880 0x7f67400b4d40 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6754103750 0x7f675419f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 --2- 172.21.15.59:0/57945483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6754104100 0x7f675419fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 >> 172.21.15.59:0/57945483 conn(0x7f675407a600 msgr2=0x7f6754101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 shutdown_connections 2024-05-08T05:28:26.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:26.850+0000 7f675a45b700 1 -- 172.21.15.59:0/57945483 wait complete. 2024-05-08T05:28:26.852 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 29 2024-05-08T05:28:27.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T05:25:27.601131+0000', 'modified': '2024-05-08T05:26:31.316568+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24473, 'mds_1': 24443, 'mds_2': 24433, 'mds_3': 24421, 'mds_4': 24419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14565': {'gid': 14565, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6839/2756426260', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 2756426260}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 2756426260}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14571': {'gid': 14571, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6841/2496793439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6840', 'nonce': 2496793439}, {'type': 'v1', 'addr': '172.21.15.59:6841', 'nonce': 2496793439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6833/1879950931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6832', 'nonce': 1879950931}, {'type': 'v1', 'addr': '172.21.15.143:6833', 'nonce': 1879950931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6833/2593895617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 2593895617}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 2593895617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6835/2504523054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 2504523054}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 2504523054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.96:6835/2294126295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 2294126295}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 2294126295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.143:6837/555225813', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 555225813}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 555225813}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.143:6839/2166229809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 2166229809}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 2166229809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.96:6837/780978870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 780978870}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 780978870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6839/126053836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 126053836}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 126053836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24473, 'qdb_cluster': [24419, 24433, 24421, 24443, 24473]} 2024-05-08T05:28:27.647 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T05:28:27.647 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.e status 2024-05-08T05:28:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:27 smithi143 ceph-mon[34337]: pgmap v391: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:27 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/57945483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:28:27.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:27 smithi096 ceph-mon[34253]: pgmap v391: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:27.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:27 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/57945483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:28:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:27 smithi059 ceph-mon[30304]: pgmap v391: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:27.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:27 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/57945483' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:28:27.907 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.568+0000 7fe8de04e700 1 -- 172.21.15.59:0/1743393090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d81095e0 msgr2=0x7fe8d81099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.568+0000 7fe8de04e700 1 --2- 172.21.15.59:0/1743393090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d81095e0 0x7fe8d81099c0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800b0d0 tx=0x7fe8c800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.568+0000 7fe8de04e700 1 -- 172.21.15.59:0/1743393090 shutdown_connections 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.568+0000 7fe8de04e700 1 --2- 172.21.15.59:0/1743393090 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8d8103f00 0x7fe8d81103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.568+0000 7fe8de04e700 1 --2- 172.21.15.59:0/1743393090 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d81095e0 0x7fe8d81099c0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.568+0000 7fe8de04e700 1 --2- 172.21.15.59:0/1743393090 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d8103540 0x7fe8d81039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.568+0000 7fe8de04e700 1 -- 172.21.15.59:0/1743393090 >> 172.21.15.59:0/1743393090 conn(0x7fe8d807a650 msgr2=0x7fe8d807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.568+0000 7fe8de04e700 1 -- 172.21.15.59:0/1743393090 shutdown_connections 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.568+0000 7fe8de04e700 1 -- 172.21.15.59:0/1743393090 wait complete. 2024-05-08T05:28:29.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.569+0000 7fe8de04e700 1 Processor -- start 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.569+0000 7fe8de04e700 1 -- start start 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.569+0000 7fe8de04e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d8103540 0x7fe8d819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.569+0000 7fe8de04e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8d8103f00 0x7fe8d819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.569+0000 7fe8de04e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d81095e0 0x7fe8d81999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.569+0000 7fe8de04e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d81136e0 con 0x7fe8d8103540 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.569+0000 7fe8de04e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d8113560 con 0x7fe8d81095e0 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.569+0000 7fe8de04e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8d8113860 con 0x7fe8d8103f00 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.569+0000 7fe8d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d8103540 0x7fe8d819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d8103540 0x7fe8d819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45064/0 (socket says 172.21.15.59:45064) 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d77fe700 1 -- 172.21.15.59:0/2084849974 learned_addr learned my addr 172.21.15.59:0/2084849974 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d81095e0 0x7fe8d81999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8ceffd700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8d8103f00 0x7fe8d819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d77fe700 1 -- 172.21.15.59:0/2084849974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8d8103f00 msgr2=0x7fe8d819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8d8103f00 0x7fe8d819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d77fe700 1 -- 172.21.15.59:0/2084849974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d81095e0 msgr2=0x7fe8d81999a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d81095e0 0x7fe8d81999a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d77fe700 1 -- 172.21.15.59:0/2084849974 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8d819a100 con 0x7fe8d8103540 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d81095e0 0x7fe8d81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d8103540 0x7fe8d819f7e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d010 tx=0x7fe8c8007ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d57fa700 1 -- 172.21.15.59:0/2084849974 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c801f070 con 0x7fe8d8103540 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8de04e700 1 -- 172.21.15.59:0/2084849974 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8d819a390 con 0x7fe8d8103540 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d57fa700 1 -- 172.21.15.59:0/2084849974 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c8003c00 con 0x7fe8d8103540 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8d57fa700 1 -- 172.21.15.59:0/2084849974 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c801b6a0 con 0x7fe8d8103540 2024-05-08T05:28:29.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.570+0000 7fe8de04e700 1 -- 172.21.15.59:0/2084849974 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8d81a66b0 con 0x7fe8d8103540 2024-05-08T05:28:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.573+0000 7fe8d57fa700 1 -- 172.21.15.59:0/2084849974 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fe8c8022070 con 0x7fe8d8103540 2024-05-08T05:28:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.573+0000 7fe8d57fa700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe8c40b2880 0x7fe8c40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8d57fa700 1 -- 172.21.15.59:0/2084849974 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe8c800f020 con 0x7fe8d8103540 2024-05-08T05:28:29.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8ceffd700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe8c40b2880 0x7fe8c40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8e88e1b80 1 Processor -- start 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8ceffd700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe8c40b2880 0x7fe8c40b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0005fd0 tx=0x7fe8c000a560 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8e88e1b80 1 -- start start 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828281db0 0x564828282190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648282826d0 0x564828286110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 0x564828286a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5648281cf8f0 con 0x564828281db0 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564828134d50 con 0x5648282826d0 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5648281cb8b0 con 0x564828286650 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828281db0 0x564828282190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8d77fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828281db0 0x564828282190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45080/0 (socket says 172.21.15.59:45080) 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.574+0000 7fe8d77fe700 1 -- 172.21.15.59:0/3501735975 learned_addr learned my addr 172.21.15.59:0/3501735975 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:29.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8ceffd700 1 --2- 172.21.15.59:0/3501735975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648282826d0 0x564828286110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/3501735975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 0x564828286a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8d77fe700 1 -- 172.21.15.59:0/3501735975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 msgr2=0x564828286a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/3501735975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 0x564828286a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8d77fe700 1 -- 172.21.15.59:0/3501735975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648282826d0 msgr2=0x564828286110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/3501735975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648282826d0 0x564828286110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8d77fe700 1 -- 172.21.15.59:0/3501735975 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56482811c880 con 0x564828281db0 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/3501735975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828281db0 0x564828282190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80a6c50 tx=0x7fe8c8003db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b266928f8ea8902a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8cdffb700 1 -- 172.21.15.59:0/3501735975 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80ab2d0 con 0x564828281db0 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8cdffb700 1 -- 172.21.15.59:0/3501735975 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c80aabd0 con 0x564828281db0 2024-05-08T05:28:29.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.575+0000 7fe8cdffb700 1 -- 172.21.15.59:0/3501735975 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c80ab050 con 0x564828281db0 2024-05-08T05:28:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3501735975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828281db0 msgr2=0x564828282190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3501735975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828281db0 0x564828282190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fe8c80a6c50 tx=0x7fe8c8003db0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3501735975 shutdown_connections 2024-05-08T05:28:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3501735975 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 0x564828286a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3501735975 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828281db0 0x564828282190 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3501735975 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5648282826d0 0x564828286110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3501735975 >> 172.21.15.59:0/3501735975 conn(0x5648281a9b20 msgr2=0x564828278250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3501735975 shutdown_connections 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3501735975 wait complete. 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.576+0000 7fe8e88e1b80 1 -- start start 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8e88e1b80 1 Processor -- start 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x56482828db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 0x56482828af80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x56482828b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5648281cf8f0 con 0x564828286650 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564828188e80 con 0x564828281db0 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56482828e5b0 con 0x5648282826d0 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8e88e1b80 10 client.0 _ll_register_callbacks cb 0x564828163560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8d7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x56482828b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8d7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x56482828b4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45084/0 (socket says 172.21.15.59:45084) 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8d7fff700 1 -- 172.21.15.59:0/3407994798 learned_addr learned my addr 172.21.15.59:0/3407994798 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8ceffd700 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 0x56482828af80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x56482828db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8d7fff700 1 -- 172.21.15.59:0/3407994798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 msgr2=0x56482828af80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 0x56482828af80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8d7fff700 1 -- 172.21.15.59:0/3407994798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 msgr2=0x56482828db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x56482828db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.577+0000 7fe8d7fff700 1 -- 172.21.15.59:0/3407994798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564828186710 con 0x564828286650 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x56482828b4c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0010670 tx=0x7fe8d0010980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c71d948e3194f7c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/3407994798 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d0012860 con 0x564828286650 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/3407994798 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe8d0012a00 con 0x564828286650 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3407994798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8c80aabd0 con 0x564828286650 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/3407994798 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8d0024050 con 0x564828286650 2024-05-08T05:28:29.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8e88e1b80 10 client.15027 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3407994798 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564828298d90 con 0x564828286650 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/3407994798 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c80aabd0 con 0x564828286650 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/3407994798 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe8d000f040 con 0x564828286650 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.578+0000 7fe8e88e1b80 10 client.15027 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.579+0000 7fe8e88e1b80 10 client.15027 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.579+0000 7fe8e88e1b80 20 client.15027 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.579+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x56482829b400 0x56482829c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.579+0000 7fe8e88e1b80 4 client.15027 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.579+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3407994798 --> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648281dd2d0 con 0x56482829b400 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.579+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x56482829b400 0x56482829c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.579+0000 7fe8b6ffd700 10 client.15027 ms_handle_connect on v2:172.21.15.96:6834/2294126295 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.579+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x56482829b400 0x56482829c870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=31791c07465aa6b7 server_cookie=50b2744da51aa046 in_seq=0 out_seq=0 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/3407994798 <== mds.4 v2:172.21.15.96:6834/2294126295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5648281dd2d0 con 0x56482829b400 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8b6ffd700 10 client.15027 handle_command_reply: tid=0 2024-05-08T05:28:29.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8e88e1b80 1 client.15027 shutdown 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3407994798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 msgr2=0x56482828b4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x56482828b4c0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0010670 tx=0x7fe8d0010980 comp rx=0 tx=0).stop 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3407994798 shutdown_connections 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x56482829b400 0x56482829c870 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x56482828b4c0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x56482828db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3407994798 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 0x56482828af80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3407994798 >> 172.21.15.59:0/3407994798 conn(0x5648281a9b20 msgr2=0x564828287b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.580+0000 7fe8b6ffd700 0 client.15027 ms_handle_reset on v2:172.21.15.96:6834/2294126295 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.581+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3407994798 shutdown_connections 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.581+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3407994798 wait complete. 2024-05-08T05:28:29.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.581+0000 7fe8e88e1b80 20 client.15027 trim_cache size 0 max 16384 2024-05-08T05:28:29.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:29 smithi059 ceph-mon[30304]: pgmap v392: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:29.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8e88e1b80 1 Processor -- start 2024-05-08T05:28:29.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8e88e1b80 1 -- start start 2024-05-08T05:28:29.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x56482828af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648282826d0 0x56482828b480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 0x5648282a5cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56482829abb0 con 0x5648282826d0 2024-05-08T05:28:29.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56482829b280 con 0x564828281db0 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56482829b950 con 0x564828286650 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8ceffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648282826d0 0x56482828b480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8ceffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648282826d0 0x56482828b480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45088/0 (socket says 172.21.15.59:45088) 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8ceffd700 1 -- 172.21.15.59:0/3833395919 learned_addr learned my addr 172.21.15.59:0/3833395919 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x56482828af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 0x5648282a5cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8ceffd700 1 -- 172.21.15.59:0/3833395919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 msgr2=0x5648282a5cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8ceffd700 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 0x5648282a5cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8ceffd700 1 -- 172.21.15.59:0/3833395919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 msgr2=0x56482828af40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8ceffd700 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x56482828af40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8ceffd700 1 -- 172.21.15.59:0/3833395919 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56482828b9c0 con 0x5648282826d0 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 0x5648282a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.603+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x56482828af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.604+0000 7fe8ceffd700 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648282826d0 0x56482828b480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0005950 tx=0x7fe8c0011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c1c4bd91b28e7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.604+0000 7fe8cd7fa700 1 -- 172.21.15.59:0/3833395919 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c0004a30 con 0x5648282826d0 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.604+0000 7fe8cd7fa700 1 -- 172.21.15.59:0/3833395919 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8c000ee80 con 0x5648282826d0 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.604+0000 7fe8cd7fa700 1 -- 172.21.15.59:0/3833395919 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8c00109c0 con 0x5648282826d0 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.604+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3833395919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648282826d0 msgr2=0x56482828b480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.604+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648282826d0 0x56482828b480 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0005950 tx=0x7fe8c0011000 comp rx=0 tx=0).stop 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3833395919 shutdown_connections 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564828286650 0x5648282a5cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x56482828af40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/3833395919 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5648282826d0 0x56482828b480 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3833395919 >> 172.21.15.59:0/3833395919 conn(0x5648281a9b20 msgr2=0x564828289890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:29.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3833395919 shutdown_connections 2024-05-08T05:28:29.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/3833395919 wait complete. 2024-05-08T05:28:29.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 -- start start 2024-05-08T05:28:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 Processor -- start 2024-05-08T05:28:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x5648282a1f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 0x5648282a24b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x564828305870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56482829c6f0 con 0x564828286650 2024-05-08T05:28:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56482829c020 con 0x564828281db0 2024-05-08T05:28:29.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56482829cdc0 con 0x5648282826d0 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.605+0000 7fe8e88e1b80 10 client.0 _ll_register_callbacks cb 0x564828163560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x564828305870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x564828305870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45092/0 (socket says 172.21.15.59:45092) 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d7fff700 1 -- 172.21.15.59:0/2502810492 learned_addr learned my addr 172.21.15.59:0/2502810492 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x5648282a1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8ceffd700 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 0x5648282a24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d7fff700 1 -- 172.21.15.59:0/2502810492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 msgr2=0x5648282a24b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 0x5648282a24b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d7fff700 1 -- 172.21.15.59:0/2502810492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 msgr2=0x5648282a1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:29.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x5648282a1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d7fff700 1 -- 172.21.15.59:0/2502810492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564828305db0 con 0x564828286650 2024-05-08T05:28:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8d7fff700 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x564828305870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0006010 tx=0x7fe8d000bc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faac63092c865083 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8b6ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/2502810492 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8d001b480 con 0x564828286650 2024-05-08T05:28:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/2502810492 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fe8d001b620 con 0x564828286650 2024-05-08T05:28:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/2502810492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5648281d7370 con 0x564828286650 2024-05-08T05:28:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/2502810492 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe8d001b820 con 0x564828286650 2024-05-08T05:28:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8e88e1b80 10 client.15033 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:29.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.606+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/2502810492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564828307080 con 0x564828286650 2024-05-08T05:28:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.607+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/2502810492 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5648281d7370 con 0x564828286650 2024-05-08T05:28:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.607+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/2502810492 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe8d0005460 con 0x564828286650 2024-05-08T05:28:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.607+0000 7fe8e88e1b80 10 client.15033 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.607+0000 7fe8e88e1b80 10 client.15033 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T05:28:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.607+0000 7fe8e88e1b80 20 client.15033 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.607+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x56482830bc80 0x56482830a990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.607+0000 7fe8e88e1b80 4 client.15033 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:28:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.607+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/2502810492 --> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] -- command(tid 0: {"prefix": "status"}) -- 0x56482804f200 con 0x56482830bc80 2024-05-08T05:28:29.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.607+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x56482830bc80 0x56482830a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8b6ffd700 10 client.15033 ms_handle_connect on v2:172.21.15.96:6834/2294126295 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8d77fe700 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x56482830bc80 0x56482830a990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e98547fbb46042de server_cookie=bf3824b501060ca7 in_seq=0 out_seq=0 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8b6ffd700 1 -- 172.21.15.59:0/2502810492 <== mds.4 v2:172.21.15.96:6834/2294126295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x56482804f200 con 0x56482830bc80 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8b6ffd700 10 client.15033 handle_command_reply: tid=0 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 client.15033 shutdown 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/2502810492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 msgr2=0x564828305870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x564828305870 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fe8d0006010 tx=0x7fe8d000bc60 comp rx=0 tx=0).stop 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/2502810492 shutdown_connections 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x56482830bc80 0x56482830a990 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564828286650 0x564828305870 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564828281db0 0x5648282a1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8b6ffd700 0 client.15033 ms_handle_reset on v2:172.21.15.96:6834/2294126295 2024-05-08T05:28:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 --2- 172.21.15.59:0/2502810492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5648282826d0 0x5648282a24b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/2502810492 >> 172.21.15.59:0/2502810492 conn(0x5648281a9b20 msgr2=0x564828285de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/2502810492 shutdown_connections 2024-05-08T05:28:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 1 -- 172.21.15.59:0/2502810492 wait complete. 2024-05-08T05:28:29.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.608+0000 7fe8e88e1b80 20 client.15033 trim_cache size 0 max 16384 2024-05-08T05:28:29.617 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:28:29.618 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:28:29.618 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 4, 2024-05-08T05:28:29.618 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24419, 2024-05-08T05:28:29.618 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:28:29.618 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:28:29.618 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:28:29.618 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 122.199859305, 2024-05-08T05:28:29.618 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:28:29.619 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:28:29.619 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:28:29.619 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 219.30446308200001 2024-05-08T05:28:29.619 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:28:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 -- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe8c40b2880 msgr2=0x7fe8c40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe8c40b2880 0x7fe8c40b4d40 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fe8c0005fd0 tx=0x7fe8c000a560 comp rx=0 tx=0).stop 2024-05-08T05:28:29.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 -- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d8103540 msgr2=0x7fe8d819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d8103540 0x7fe8d819f7e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fe8c800d010 tx=0x7fe8c8007ad0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 -- 172.21.15.59:0/2084849974 shutdown_connections 2024-05-08T05:28:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fe8c40b2880 0x7fe8c40b4d40 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8d81095e0 0x7fe8d81999a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8d8103540 0x7fe8d819f7e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 --2- 172.21.15.59:0/2084849974 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8d8103f00 0x7fe8d819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 -- 172.21.15.59:0/2084849974 >> 172.21.15.59:0/2084849974 conn(0x7fe8d807a650 msgr2=0x7fe8d8102290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 -- 172.21.15.59:0/2084849974 shutdown_connections 2024-05-08T05:28:29.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:29.609+0000 7fe8de04e700 1 -- 172.21.15.59:0/2084849974 wait complete. 2024-05-08T05:28:29.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:29 smithi143 ceph-mon[34337]: pgmap v392: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:29.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:29 smithi096 ceph-mon[34253]: pgmap v392: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:30.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-08T05:28:30.534 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:31.740 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:31 smithi059 ceph-mon[30304]: pgmap v393: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:31.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:31 smithi143 ceph-mon[34337]: pgmap v393: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:31.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:31 smithi096 ceph-mon[34253]: pgmap v393: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.326+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3972863151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c10ff50 msgr2=0x7f5c6c112340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.326+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3972863151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c10ff50 0x7f5c6c112340 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5c6400b0a0 tx=0x7f5c6400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.326+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3972863151 shutdown_connections 2024-05-08T05:28:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.326+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3972863151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c10ff50 0x7f5c6c112340 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.326+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3972863151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c06b1b0 0x7f5c6c06b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.326+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3972863151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c6c06bad0 0x7f5c6c10fa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.326+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3972863151 >> 172.21.15.59:0/3972863151 conn(0x7f5c6c079350 msgr2=0x7f5c6c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.326+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3972863151 shutdown_connections 2024-05-08T05:28:32.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.326+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3972863151 wait complete. 2024-05-08T05:28:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c70a5c700 1 Processor -- start 2024-05-08T05:28:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c70a5c700 1 -- start start 2024-05-08T05:28:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c70a5c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c6c06b1b0 0x7f5c6c19be60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c70a5c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c06bad0 0x7f5c6c19c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c70a5c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c10ff50 0x7f5c6c1a0730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c70a5c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c1158d0 con 0x7f5c6c06bad0 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c70a5c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c115750 con 0x7f5c6c06b1b0 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c70a5c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5c6c115a50 con 0x7f5c6c10ff50 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c69d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c06bad0 0x7f5c6c19c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c69d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c06bad0 0x7f5c6c19c3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45110/0 (socket says 172.21.15.59:45110) 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c69d9b700 1 -- 172.21.15.59:0/3585557742 learned_addr learned my addr 172.21.15.59:0/3585557742 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.327+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c6c06b1b0 0x7f5c6c19be60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c6ad9d700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c10ff50 0x7f5c6c1a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c69d9b700 1 -- 172.21.15.59:0/3585557742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c10ff50 msgr2=0x7f5c6c1a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c10ff50 0x7f5c6c1a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c69d9b700 1 -- 172.21.15.59:0/3585557742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c6c06b1b0 msgr2=0x7f5c6c19be60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c6c06b1b0 0x7f5c6c19be60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c69d9b700 1 -- 172.21.15.59:0/3585557742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c6c1a0e70 con 0x7f5c6c06bad0 2024-05-08T05:28:32.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c06bad0 0x7f5c6c19c3a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000b530 tx=0x7f5c6000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c5b7fe700 1 -- 172.21.15.59:0/3585557742 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60015020 con 0x7f5c6c06bad0 2024-05-08T05:28:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3585557742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c6c1a1160 con 0x7f5c6c06bad0 2024-05-08T05:28:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c5b7fe700 1 -- 172.21.15.59:0/3585557742 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c60011ce0 con 0x7f5c6c06bad0 2024-05-08T05:28:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c5b7fe700 1 -- 172.21.15.59:0/3585557742 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c60010dd0 con 0x7f5c6c06bad0 2024-05-08T05:28:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.328+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3585557742 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c6c1a89e0 con 0x7f5c6c06bad0 2024-05-08T05:28:32.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.331+0000 7f5c5b7fe700 1 -- 172.21.15.59:0/3585557742 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5c60018020 con 0x7f5c6c06bad0 2024-05-08T05:28:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.332+0000 7f5c5b7fe700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c540b2880 0x7f5c540b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.332+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c540b2880 0x7f5c540b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.332+0000 7f5c7b2efb80 1 Processor -- start 2024-05-08T05:28:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.332+0000 7f5c5b7fe700 1 -- 172.21.15.59:0/3585557742 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5c600e1ce0 con 0x7f5c6c06bad0 2024-05-08T05:28:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.332+0000 7f5c7b2efb80 1 -- start start 2024-05-08T05:28:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.332+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c540b2880 0x7f5c540b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c004630 tx=0x7f5c5c00b3c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.332+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 0x558b869d46a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d50b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b86922810 con 0x558b869d4c30 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b868c1770 con 0x558b869d0da0 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558b8691e7d0 con 0x558b869d0480 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d50b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6a59c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:55026/0 (socket says 172.21.15.59:55026) 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d50b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45118/0 (socket says 172.21.15.59:45118) 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6a59c700 1 -- 172.21.15.59:0/3037940340 learned_addr learned my addr 172.21.15.59:0/3037940340 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/3037940340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 0x558b869d46a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6a59c700 1 -- 172.21.15.59:0/3037940340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 msgr2=0x558b869d46a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/3037940340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 0x558b869d46a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6a59c700 1 -- 172.21.15.59:0/3037940340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 msgr2=0x558b869d50b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/3037940340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d50b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6a59c700 1 -- 172.21.15.59:0/3037940340 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b8686f880 con 0x558b869d0480 2024-05-08T05:28:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6ad9d700 1 --2- 172.21.15.59:0/3037940340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:28:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.333+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/3037940340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869d0860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c003b10 tx=0x7f5c5c013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=507a54c7dcc032f6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c58ff9700 1 -- 172.21.15.59:0/3037940340 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c0118d0 con 0x558b869d0480 2024-05-08T05:28:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c58ff9700 1 -- 172.21.15.59:0/3037940340 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c5c01e420 con 0x558b869d0480 2024-05-08T05:28:32.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c58ff9700 1 -- 172.21.15.59:0/3037940340 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c5c01c610 con 0x558b869d0480 2024-05-08T05:28:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/3037940340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 msgr2=0x558b869d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/3037940340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869d0860 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c003b10 tx=0x7f5c5c013000 comp rx=0 tx=0).stop 2024-05-08T05:28:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/3037940340 shutdown_connections 2024-05-08T05:28:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/3037940340 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d50b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/3037940340 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869d0860 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/3037940340 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 0x558b869d46a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.334+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/3037940340 >> 172.21.15.59:0/3037940340 conn(0x558b868fc0b0 msgr2=0x558b869cac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/3037940340 shutdown_connections 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/3037940340 wait complete. 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 -- start start 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 Processor -- start 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869dc030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 0x558b869d93d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b869dce40 con 0x558b869d4c30 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b86888be0 con 0x558b869d0da0 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558b869dcfc0 con 0x558b869d0480 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.335+0000 7f5c7b2efb80 10 client.0 _ll_register_callbacks cb 0x558b868ba8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:32.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d9910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45128/0 (socket says 172.21.15.59:45128) 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6ad9d700 1 -- 172.21.15.59:0/2465682366 learned_addr learned my addr 172.21.15.59:0/2465682366 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869dc030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 0x558b869d93d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6ad9d700 1 -- 172.21.15.59:0/2465682366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 msgr2=0x558b869dc030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6ad9d700 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869dc030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6ad9d700 1 -- 172.21.15.59:0/2465682366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 msgr2=0x558b869d93d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6ad9d700 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 0x558b869d93d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6ad9d700 1 -- 172.21.15.59:0/2465682366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b868be640 con 0x558b869d4c30 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 0x558b869d93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c6ad9d700 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d9910 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5c64015000 tx=0x7f5c64004da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77406fac71c74fee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:32.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2465682366 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c64007a30 con 0x558b869d4c30 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2465682366 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5c64007bd0 con 0x558b869d4c30 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2465682366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c5c01e420 con 0x558b869d4c30 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.336+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2465682366 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c64007dd0 con 0x558b869d4c30 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c7b2efb80 10 client.15045 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2465682366 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b869e7660 con 0x558b869d4c30 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2465682366 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c5c01e420 con 0x558b869d4c30 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2465682366 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5c64011070 con 0x558b869d4c30 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c7b2efb80 10 client.15045 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c7b2efb80 10 client.15045 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c7b2efb80 20 client.15045 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x558b869e9c90 0x558b869eb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c7b2efb80 4 client.15045 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:28:32.343 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2465682366 --> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b8699c600 con 0x558b869e9c90 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.337+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x558b869e9c90 0x558b869eb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.338+0000 7f5c51ffb700 10 client.15045 ms_handle_connect on v2:172.21.15.143:6836/555225813 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.338+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x558b869e9c90 0x558b869eb100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b22153a10b705372 server_cookie=b38861e7af8888c3 in_seq=0 out_seq=0 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2465682366 <== mds.3 v2:172.21.15.143:6836/555225813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558b8699c600 con 0x558b869e9c90 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c51ffb700 10 client.15045 handle_command_reply: tid=0 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 client.15045 shutdown 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2465682366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 msgr2=0x558b869d9910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d9910 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f5c64015000 tx=0x7f5c64004da0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2465682366 shutdown_connections 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d4c30 0x558b869d9910 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x558b869e9c90 0x558b869eb100 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869dc030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2465682366 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0da0 0x558b869d93d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.344 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c51ffb700 0 client.15045 ms_handle_reset on v2:172.21.15.143:6836/555225813 2024-05-08T05:28:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2465682366 >> 172.21.15.59:0/2465682366 conn(0x558b868fc0b0 msgr2=0x558b869c56d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2465682366 shutdown_connections 2024-05-08T05:28:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2465682366 wait complete. 2024-05-08T05:28:32.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.339+0000 7f5c7b2efb80 20 client.15045 trim_cache size 0 max 16384 2024-05-08T05:28:32.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.360+0000 7f5c7b2efb80 1 Processor -- start 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.360+0000 7f5c7b2efb80 1 -- start start 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869d9390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869d98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d4c30 0x558b869d7d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b869e9440 con 0x558b869d0da0 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b869e9b10 con 0x558b869d4c30 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558b869ea1e0 con 0x558b869d0480 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c69d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869d98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c69d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869d98d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45130/0 (socket says 172.21.15.59:45130) 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c69d9b700 1 -- 172.21.15.59:0/650267901 learned_addr learned my addr 172.21.15.59:0/650267901 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c6ad9d700 1 --2- 172.21.15.59:0/650267901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d4c30 0x558b869d7d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c69d9b700 1 -- 172.21.15.59:0/650267901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 msgr2=0x558b869d9390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:32.363 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/650267901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869d9390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c69d9b700 1 -- 172.21.15.59:0/650267901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d4c30 msgr2=0x558b869d7d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/650267901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d4c30 0x558b869d7d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c69d9b700 1 -- 172.21.15.59:0/650267901 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b869cc4d0 con 0x558b869d0da0 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c6ad9d700 1 --2- 172.21.15.59:0/650267901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d4c30 0x558b869d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/650267901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869d98d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5c600a4f90 tx=0x7f5c60010870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5d38d95509d356a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c53fff700 1 -- 172.21.15.59:0/650267901 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600a8f10 con 0x558b869d0da0 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c53fff700 1 -- 172.21.15.59:0/650267901 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c600ad2c0 con 0x558b869d0da0 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.361+0000 7f5c53fff700 1 -- 172.21.15.59:0/650267901 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600a83d0 con 0x558b869d0da0 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/650267901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 msgr2=0x558b869d98d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/650267901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869d98d0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f5c600a4f90 tx=0x7f5c60010870 comp rx=0 tx=0).stop 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/650267901 shutdown_connections 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/650267901 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d4c30 0x558b869d7d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/650267901 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d0480 0x558b869d9390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/650267901 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869d98d0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/650267901 >> 172.21.15.59:0/650267901 conn(0x558b868fc0b0 msgr2=0x558b869d8990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/650267901 shutdown_connections 2024-05-08T05:28:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/650267901 wait complete. 2024-05-08T05:28:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 -- start start 2024-05-08T05:28:32.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.362+0000 7f5c7b2efb80 1 Processor -- start 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0480 0x558b869ee4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869eea30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c7b2efb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d4c30 0x558b869f2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558b869eaf80 con 0x558b869d0da0 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558b869ea8b0 con 0x558b869d0480 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c7b2efb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558b869eb650 con 0x558b869d4c30 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c7b2efb80 10 client.0 _ll_register_callbacks cb 0x558b868ba8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c69d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869eea30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c69d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869eea30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45136/0 (socket says 172.21.15.59:45136) 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c69d9b700 1 -- 172.21.15.59:0/2298164768 learned_addr learned my addr 172.21.15.59:0/2298164768 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0480 0x558b869ee4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c6ad9d700 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d4c30 0x558b869f2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c69d9b700 1 -- 172.21.15.59:0/2298164768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d4c30 msgr2=0x558b869f2cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d4c30 0x558b869f2cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c69d9b700 1 -- 172.21.15.59:0/2298164768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0480 msgr2=0x558b869ee4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0480 0x558b869ee4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c69d9b700 1 -- 172.21.15.59:0/2298164768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b869f31f0 con 0x558b869d0da0 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c6ad9d700 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d4c30 0x558b869f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c69d9b700 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869eea30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5c600a8c10 tx=0x7f5c600106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77445a95741e724c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c51ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2298164768 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c600a8f10 con 0x558b869d0da0 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2298164768 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5c600b1af0 con 0x558b869d0da0 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.363+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2298164768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c600ad2c0 con 0x558b869d0da0 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2298164768 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c60011830 con 0x558b869d0da0 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c7b2efb80 10 client.15051 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2298164768 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b86a557b0 con 0x558b869d0da0 2024-05-08T05:28:32.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2298164768 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c600ad2c0 con 0x558b869d0da0 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2298164768 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5c600a7be0 con 0x558b869d0da0 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c7b2efb80 10 client.15051 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c7b2efb80 10 client.15051 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c7b2efb80 20 client.15051 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x558b86a59d90 0x558b86a58aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c7b2efb80 4 client.15051 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2298164768 --> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] -- command(tid 0: {"prefix": "status"}) -- 0x558b869301f0 con 0x558b86a59d90 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.364+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x558b86a59d90 0x558b86a58aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c6a59c700 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x558b86a59d90 0x558b86a58aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6287ec6e09484f1c server_cookie=6765a379660a1fed in_seq=0 out_seq=0 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c51ffb700 10 client.15051 ms_handle_connect on v2:172.21.15.143:6836/555225813 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c51ffb700 1 -- 172.21.15.59:0/2298164768 <== mds.3 v2:172.21.15.143:6836/555225813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x558b869301f0 con 0x558b86a59d90 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c51ffb700 10 client.15051 handle_command_reply: tid=0 2024-05-08T05:28:32.370 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 client.15051 shutdown 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2298164768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 msgr2=0x558b869eea30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869eea30 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f5c600a8c10 tx=0x7f5c600106b0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2298164768 shutdown_connections 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x558b86a59d90 0x558b86a58aa0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558b869d4c30 0x558b869f2cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558b869d0480 0x558b869ee4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 --2- 172.21.15.59:0/2298164768 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558b869d0da0 0x558b869eea30 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2298164768 >> 172.21.15.59:0/2298164768 conn(0x558b868fc0b0 msgr2=0x558b869d9390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c51ffb700 0 client.15051 ms_handle_reset on v2:172.21.15.143:6836/555225813 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2298164768 shutdown_connections 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 1 -- 172.21.15.59:0/2298164768 wait complete. 2024-05-08T05:28:32.371 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.365+0000 7f5c7b2efb80 20 client.15051 trim_cache size 0 max 16384 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 3, 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24421, 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 125.96497544499999, 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 216.481386456 2024-05-08T05:28:32.372 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:28:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c540b2880 msgr2=0x7f5c540b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c540b2880 0x7f5c540b4d40 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f5c5c004630 tx=0x7f5c5c00b3c0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c06bad0 msgr2=0x7f5c6c19c3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c06bad0 0x7f5c6c19c3a0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f5c6000b530 tx=0x7f5c6000b840 comp rx=0 tx=0).stop 2024-05-08T05:28:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3585557742 shutdown_connections 2024-05-08T05:28:32.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5c6c10ff50 0x7f5c6c1a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c540b2880 0x7f5c540b4d40 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5c6c06b1b0 0x7f5c6c19be60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 --2- 172.21.15.59:0/3585557742 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5c6c06bad0 0x7f5c6c19c3a0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3585557742 >> 172.21.15.59:0/3585557742 conn(0x7f5c6c079350 msgr2=0x7f5c6c1114a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3585557742 shutdown_connections 2024-05-08T05:28:32.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:32.366+0000 7f5c70a5c700 1 -- 172.21.15.59:0/3585557742 wait complete. 2024-05-08T05:28:33.083 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T05:28:33.340 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:33.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:33 smithi143 ceph-mon[34337]: pgmap v394: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:33.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:33 smithi059 ceph-mon[30304]: pgmap v394: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:33.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:33 smithi096 ceph-mon[34253]: pgmap v394: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.017+0000 7f5cb0dba700 1 -- 172.21.15.59:0/1795312402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac103750 msgr2=0x7f5cac103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.017+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/1795312402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac103750 0x7f5cac103b30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00b0d0 tx=0x7f5c9c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.017+0000 7f5cb0dba700 1 -- 172.21.15.59:0/1795312402 shutdown_connections 2024-05-08T05:28:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.017+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/1795312402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cac108850 0x7f5cac1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.017+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/1795312402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac103750 0x7f5cac103b30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.017+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/1795312402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac104100 0x7f5cac108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.017+0000 7f5cb0dba700 1 -- 172.21.15.59:0/1795312402 >> 172.21.15.59:0/1795312402 conn(0x7f5cac07a600 msgr2=0x7f5cac07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.017+0000 7f5cb0dba700 1 -- 172.21.15.59:0/1795312402 shutdown_connections 2024-05-08T05:28:35.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.017+0000 7f5cb0dba700 1 -- 172.21.15.59:0/1795312402 wait complete. 2024-05-08T05:28:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5cb0dba700 1 Processor -- start 2024-05-08T05:28:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5cb0dba700 1 -- start start 2024-05-08T05:28:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5cb0dba700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cac103750 0x7f5cac19f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5cb0dba700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac104100 0x7f5cac19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5cb0dba700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac108850 0x7f5cac199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5cb0dba700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac113620 con 0x7f5cac108850 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5cb0dba700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1134a0 con 0x7f5cac104100 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5cb0dba700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5cac1137a0 con 0x7f5cac103750 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5caad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac108850 0x7f5cac199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5caad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac108850 0x7f5cac199920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45146/0 (socket says 172.21.15.59:45146) 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5caad9d700 1 -- 172.21.15.59:0/3001958367 learned_addr learned my addr 172.21.15.59:0/3001958367 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5ca9d9b700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac104100 0x7f5cac19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.018+0000 7f5caa59c700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cac103750 0x7f5cac19f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5caad9d700 1 -- 172.21.15.59:0/3001958367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cac103750 msgr2=0x7f5cac19f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5caad9d700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cac103750 0x7f5cac19f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5caad9d700 1 -- 172.21.15.59:0/3001958367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac104100 msgr2=0x7f5cac19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5caad9d700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac104100 0x7f5cac19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5caad9d700 1 -- 172.21.15.59:0/3001958367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5cac19a080 con 0x7f5cac108850 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5caad9d700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac108850 0x7f5cac199920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c900 tx=0x7f5ca400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5c9b7fe700 1 -- 172.21.15.59:0/3001958367 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4007b00 con 0x7f5cac108850 2024-05-08T05:28:35.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5c9b7fe700 1 -- 172.21.15.59:0/3001958367 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca400ce50 con 0x7f5cac108850 2024-05-08T05:28:35.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5cb0dba700 1 -- 172.21.15.59:0/3001958367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5cac19a370 con 0x7f5cac108850 2024-05-08T05:28:35.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.019+0000 7f5c9b7fe700 1 -- 172.21.15.59:0/3001958367 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca4018660 con 0x7f5cac108850 2024-05-08T05:28:35.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.020+0000 7f5cb0dba700 1 -- 172.21.15.59:0/3001958367 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5cac1a65e0 con 0x7f5cac108850 2024-05-08T05:28:35.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.022+0000 7f5c9b7fe700 1 -- 172.21.15.59:0/3001958367 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f5ca400f450 con 0x7f5cac108850 2024-05-08T05:28:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.023+0000 7f5c9b7fe700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c940b2900 0x7f5c940b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.023+0000 7f5caa59c700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c940b2900 0x7f5c940b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.023+0000 7f5c9b7fe700 1 -- 172.21.15.59:0/3001958367 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5ca40e24b0 con 0x7f5cac108850 2024-05-08T05:28:35.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.023+0000 7f5caa59c700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c940b2900 0x7f5c940b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009fd0 tx=0x7f5c9c00b040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:35.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5cbb64db80 1 Processor -- start 2024-05-08T05:28:35.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5cbb64db80 1 -- start start 2024-05-08T05:28:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142540 0x55e365142920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 0x55e36513cbd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e36513d160 0x55e36513d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e36508a8f0 con 0x55e365142540 2024-05-08T05:28:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e364fefd50 con 0x55e365142e60 2024-05-08T05:28:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e3650868b0 con 0x55e36513d160 2024-05-08T05:28:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5caa59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142540 0x55e365142920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5caa59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142540 0x55e365142920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45154/0 (socket says 172.21.15.59:45154) 2024-05-08T05:28:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5caa59c700 1 -- 172.21.15.59:0/325892365 learned_addr learned my addr 172.21.15.59:0/325892365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5caad9d700 1 --2- 172.21.15.59:0/325892365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e36513d160 0x55e36513d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.024+0000 7f5ca9d9b700 1 --2- 172.21.15.59:0/325892365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 0x55e36513cbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.025+0000 7f5caa59c700 1 -- 172.21.15.59:0/325892365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e36513d160 msgr2=0x55e36513d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:35.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.025+0000 7f5caa59c700 1 --2- 172.21.15.59:0/325892365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e36513d160 0x55e36513d5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.025+0000 7f5caa59c700 1 -- 172.21.15.59:0/325892365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 msgr2=0x55e36513cbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.025+0000 7f5caa59c700 1 --2- 172.21.15.59:0/325892365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 0x55e36513cbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.025+0000 7f5caa59c700 1 -- 172.21.15.59:0/325892365 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e364fd7880 con 0x55e365142540 2024-05-08T05:28:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.025+0000 7f5caa59c700 1 --2- 172.21.15.59:0/325892365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142540 0x55e365142920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00b0d0 tx=0x7f5c9c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac897066b441af4c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.025+0000 7f5c98ff9700 1 -- 172.21.15.59:0/325892365 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c007b40 con 0x55e365142540 2024-05-08T05:28:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.025+0000 7f5c98ff9700 1 -- 172.21.15.59:0/325892365 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c9c02e070 con 0x55e365142540 2024-05-08T05:28:35.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.025+0000 7f5c98ff9700 1 -- 172.21.15.59:0/325892365 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c01c440 con 0x55e365142540 2024-05-08T05:28:35.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 -- 172.21.15.59:0/325892365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142540 msgr2=0x55e365142920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:35.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/325892365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142540 0x55e365142920 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c00b0d0 tx=0x7f5c9c01c000 comp rx=0 tx=0).stop 2024-05-08T05:28:35.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 -- 172.21.15.59:0/325892365 shutdown_connections 2024-05-08T05:28:35.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/325892365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e36513d160 0x55e36513d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/325892365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142540 0x55e365142920 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/325892365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 0x55e36513cbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 -- 172.21.15.59:0/325892365 >> 172.21.15.59:0/325892365 conn(0x55e365064b20 msgr2=0x55e365133250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 -- 172.21.15.59:0/325892365 shutdown_connections 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 -- 172.21.15.59:0/325892365 wait complete. 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 -- start start 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.026+0000 7f5cbb64db80 1 Processor -- start 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e36513d160 0x55e365148af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e365145e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 0x55e3651463d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e365149830 con 0x55e36513d160 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e365043e80 con 0x55e365142e60 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e3651499b0 con 0x55e365142540 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5cbb64db80 10 client.0 _ll_register_callbacks cb 0x55e36501e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caa59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e36513d160 0x55e365148af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caa59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e36513d160 0x55e365148af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45158/0 (socket says 172.21.15.59:45158) 2024-05-08T05:28:35.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caa59c700 1 -- 172.21.15.59:0/1409347295 learned_addr learned my addr 172.21.15.59:0/1409347295 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5ca9d9b700 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e365145e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caad9d700 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 0x55e3651463d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caa59c700 1 -- 172.21.15.59:0/1409347295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 msgr2=0x55e365145e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caa59c700 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e365145e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caa59c700 1 -- 172.21.15.59:0/1409347295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 msgr2=0x55e3651463d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caa59c700 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 0x55e3651463d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caa59c700 1 -- 172.21.15.59:0/1409347295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e365041710 con 0x55e36513d160 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5caa59c700 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e36513d160 0x55e365148af0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c029cc0 tx=0x7f5c9c029760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=360d22405856f86 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5c91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.027+0000 7f5c91ffb700 1 -- 172.21.15.59:0/1409347295 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c9c037070 con 0x55e36513d160 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5c91ffb700 1 -- 172.21.15.59:0/1409347295 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5c9c01c650 con 0x55e36513d160 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5cbb64db80 1 -- 172.21.15.59:0/1409347295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c9c02e070 con 0x55e36513d160 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5c91ffb700 1 -- 172.21.15.59:0/1409347295 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c9c01c850 con 0x55e36513d160 2024-05-08T05:28:35.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5cbb64db80 10 client.15063 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5cbb64db80 1 -- 172.21.15.59:0/1409347295 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3651540e0 con 0x55e36513d160 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5c91ffb700 1 -- 172.21.15.59:0/1409347295 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c9c02e070 con 0x55e36513d160 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5c91ffb700 1 -- 172.21.15.59:0/1409347295 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5c9c011070 con 0x55e36513d160 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5cbb64db80 10 client.15063 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5cbb64db80 10 client.15063 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5cbb64db80 20 client.15063 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x55e365155430 0x55e3651575c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5cbb64db80 4 client.15063 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.028+0000 7f5cbb64db80 1 -- 172.21.15.59:0/1409347295 --> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e3650982d0 con 0x55e365155430 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.029+0000 7f5ca9d9b700 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x55e365155430 0x55e3651575c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.029+0000 7f5ca9d9b700 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x55e365155430 0x55e3651575c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3989db24acfe0604 server_cookie=d8523a489b9ca92 in_seq=0 out_seq=0 2024-05-08T05:28:35.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.029+0000 7f5c91ffb700 10 client.15063 ms_handle_connect on v2:172.21.15.143:6838/2166229809 2024-05-08T05:28:35.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.030+0000 7f5c91ffb700 1 -- 172.21.15.59:0/1409347295 <== mds.2 v2:172.21.15.143:6838/2166229809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e3650982d0 con 0x55e365155430 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.030+0000 7f5c91ffb700 10 client.15063 handle_command_reply: tid=0 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.030+0000 7f5cbb64db80 1 client.15063 shutdown 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.030+0000 7f5cbb64db80 1 -- 172.21.15.59:0/1409347295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e36513d160 msgr2=0x55e365148af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.030+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e36513d160 0x55e365148af0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c029cc0 tx=0x7f5c9c029760 comp rx=0 tx=0).stop 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5cbb64db80 1 -- 172.21.15.59:0/1409347295 shutdown_connections 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e365142e60 0x55e3651463d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x55e365155430 0x55e3651575c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e36513d160 0x55e365148af0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/1409347295 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e365145e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5cbb64db80 1 -- 172.21.15.59:0/1409347295 >> 172.21.15.59:0/1409347295 conn(0x55e365064b20 msgr2=0x55e36512dae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5c91ffb700 0 client.15063 ms_handle_reset on v2:172.21.15.143:6838/2166229809 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5cbb64db80 1 -- 172.21.15.59:0/1409347295 shutdown_connections 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5cbb64db80 1 -- 172.21.15.59:0/1409347295 wait complete. 2024-05-08T05:28:35.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.031+0000 7f5cbb64db80 20 client.15063 trim_cache size 0 max 16384 2024-05-08T05:28:35.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5cbb64db80 1 Processor -- start 2024-05-08T05:28:35.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5cbb64db80 1 -- start start 2024-05-08T05:28:35.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 0x55e365155a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e365156ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e365134d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e365155900 con 0x55e365142e60 2024-05-08T05:28:35.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e365155fd0 con 0x55e36513d160 2024-05-08T05:28:35.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e3651566a0 con 0x55e365142540 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e365134d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e365134d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45166/0 (socket says 172.21.15.59:45166) 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caad9d700 1 -- 172.21.15.59:0/215401966 learned_addr learned my addr 172.21.15.59:0/215401966 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caa59c700 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 0x55e365155a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5ca9d9b700 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e365156ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caad9d700 1 -- 172.21.15.59:0/215401966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 msgr2=0x55e365156ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caad9d700 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e365156ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caad9d700 1 -- 172.21.15.59:0/215401966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 msgr2=0x55e365155a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caad9d700 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 0x55e365155a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caad9d700 1 -- 172.21.15.59:0/215401966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e3651499c0 con 0x55e365142e60 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5ca9d9b700 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e365156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.052+0000 7f5caad9d700 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e365134d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40a5760 tx=0x7f5ca40a9030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f90c888368c33a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5c93fff700 1 -- 172.21.15.59:0/215401966 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40c3070 con 0x55e365142e60 2024-05-08T05:28:35.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5c93fff700 1 -- 172.21.15.59:0/215401966 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca40a91f0 con 0x55e365142e60 2024-05-08T05:28:35.055 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5c93fff700 1 -- 172.21.15.59:0/215401966 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca400f740 con 0x55e365142e60 2024-05-08T05:28:35.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5cbb64db80 1 -- 172.21.15.59:0/215401966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 msgr2=0x55e365134d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:35.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e365134d40 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40a5760 tx=0x7f5ca40a9030 comp rx=0 tx=0).stop 2024-05-08T05:28:35.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5cbb64db80 1 -- 172.21.15.59:0/215401966 shutdown_connections 2024-05-08T05:28:35.058 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e365134d40 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 0x55e365155a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/215401966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e365156ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5cbb64db80 1 -- 172.21.15.59:0/215401966 >> 172.21.15.59:0/215401966 conn(0x55e365064b20 msgr2=0x55e365143300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5cbb64db80 1 -- 172.21.15.59:0/215401966 shutdown_connections 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.053+0000 7f5cbb64db80 1 -- 172.21.15.59:0/215401966 wait complete. 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5cbb64db80 1 -- start start 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5cbb64db80 1 Processor -- start 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 0x55e36515ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e36515f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5cbb64db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e36515f660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e365157440 con 0x55e365142e60 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e365156d70 con 0x55e36513d160 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5cbb64db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e365157b10 con 0x55e365142540 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5cbb64db80 10 client.0 _ll_register_callbacks cb 0x55e36501e560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e36515f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.059 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e36515f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45174/0 (socket says 172.21.15.59:45174) 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caad9d700 1 -- 172.21.15.59:0/258612219 learned_addr learned my addr 172.21.15.59:0/258612219 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caa59c700 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 0x55e36515ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5ca9d9b700 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e36515f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caad9d700 1 -- 172.21.15.59:0/258612219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 msgr2=0x55e36515f120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caad9d700 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e36515f120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caad9d700 1 -- 172.21.15.59:0/258612219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 msgr2=0x55e36515ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caad9d700 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 0x55e36515ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caad9d700 1 -- 172.21.15.59:0/258612219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e36512cfc0 con 0x55e365142e60 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5ca9d9b700 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e36515f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.054+0000 7f5caa59c700 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 0x55e36515ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5caad9d700 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e36515f660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40074a0 tx=0x7f5ca40a9750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6239310c28a2fe77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5c91ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5c91ffb700 1 -- 172.21.15.59:0/258612219 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ca40c8060 con 0x55e365142e60 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5cbb64db80 1 -- 172.21.15.59:0/258612219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ca40a91f0 con 0x55e365142e60 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5c91ffb700 1 -- 172.21.15.59:0/258612219 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f5ca40a5cd0 con 0x55e365142e60 2024-05-08T05:28:35.060 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5c91ffb700 1 -- 172.21.15.59:0/258612219 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ca4088050 con 0x55e365142e60 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5cbb64db80 10 client.15069 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5cbb64db80 1 -- 172.21.15.59:0/258612219 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e3651c16b0 con 0x55e365142e60 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5c91ffb700 1 -- 172.21.15.59:0/258612219 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ca40a91f0 con 0x55e365142e60 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5c91ffb700 1 -- 172.21.15.59:0/258612219 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5ca40b20b0 con 0x55e365142e60 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5cbb64db80 10 client.15069 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5cbb64db80 10 client.15069 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5cbb64db80 20 client.15069 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x55e3651c5ce0 0x55e3651c49f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5cbb64db80 4 client.15069 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.055+0000 7f5cbb64db80 1 -- 172.21.15.59:0/258612219 --> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] -- command(tid 0: {"prefix": "status"}) -- 0x55e365092370 con 0x55e3651c5ce0 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.056+0000 7f5caa59c700 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x55e3651c5ce0 0x55e3651c49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.056+0000 7f5caa59c700 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x55e3651c5ce0 0x55e3651c49f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=822f9dc77f830340 server_cookie=ef85432e6dec0c0 in_seq=0 out_seq=0 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.056+0000 7f5c91ffb700 10 client.15069 ms_handle_connect on v2:172.21.15.143:6838/2166229809 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.056+0000 7f5c91ffb700 1 -- 172.21.15.59:0/258612219 <== mds.2 v2:172.21.15.143:6838/2166229809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e365092370 con 0x55e3651c5ce0 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.056+0000 7f5c91ffb700 10 client.15069 handle_command_reply: tid=0 2024-05-08T05:28:35.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 client.15069 shutdown 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 -- 172.21.15.59:0/258612219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 msgr2=0x55e36515f660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e36515f660 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f5ca40074a0 tx=0x7f5ca40a9750 comp rx=0 tx=0).stop 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 -- 172.21.15.59:0/258612219 shutdown_connections 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x55e3651c5ce0 0x55e3651c49f0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e365142e60 0x55e36515f660 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e36513d160 0x55e36515ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 --2- 172.21.15.59:0/258612219 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e365142540 0x55e36515f120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 -- 172.21.15.59:0/258612219 >> 172.21.15.59:0/258612219 conn(0x55e365064b20 msgr2=0x55e3651377b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5c91ffb700 0 client.15069 ms_handle_reset on v2:172.21.15.143:6838/2166229809 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 -- 172.21.15.59:0/258612219 shutdown_connections 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 1 -- 172.21.15.59:0/258612219 wait complete. 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cbb64db80 20 client.15069 trim_cache size 0 max 16384 2024-05-08T05:28:35.062 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 2, 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24433, 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 129.66413000099999, 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 213.52693958500001 2024-05-08T05:28:35.063 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:28:35.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cb0dba700 1 -- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c940b2900 msgr2=0x7f5c940b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:35.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c940b2900 0x7f5c940b4dc0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f5c9c009fd0 tx=0x7f5c9c00b040 comp rx=0 tx=0).stop 2024-05-08T05:28:35.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cb0dba700 1 -- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac108850 msgr2=0x7f5cac199920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:35.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac108850 0x7f5cac199920 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f5ca400c900 tx=0x7f5ca400cc10 comp rx=0 tx=0).stop 2024-05-08T05:28:35.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cb0dba700 1 -- 172.21.15.59:0/3001958367 shutdown_connections 2024-05-08T05:28:35.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5cac108850 0x7f5cac199920 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5cac103750 0x7f5cac19f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f5c940b2900 0x7f5c940b4dc0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.057+0000 7f5cb0dba700 1 --2- 172.21.15.59:0/3001958367 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5cac104100 0x7f5cac19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:35.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.058+0000 7f5cb0dba700 1 -- 172.21.15.59:0/3001958367 >> 172.21.15.59:0/3001958367 conn(0x7f5cac07a600 msgr2=0x7f5cac1017c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:35.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.058+0000 7f5cb0dba700 1 -- 172.21.15.59:0/3001958367 shutdown_connections 2024-05-08T05:28:35.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:35.058+0000 7f5cb0dba700 1 -- 172.21.15.59:0/3001958367 wait complete. 2024-05-08T05:28:35.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:35 smithi143 ceph-mon[34337]: pgmap v395: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:35.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:35 smithi096 ceph-mon[34253]: pgmap v395: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:35.783 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:35 smithi059 ceph-mon[30304]: pgmap v395: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:35.817 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-08T05:28:36.076 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:37.737 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:37 smithi059 ceph-mon[30304]: pgmap v396: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:37.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:37 smithi143 ceph-mon[34337]: pgmap v396: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:37.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.749+0000 7f8470ad9700 1 -- 172.21.15.59:0/2824001653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102630 msgr2=0x7f846c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:37.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.749+0000 7f8470ad9700 1 --2- 172.21.15.59:0/2824001653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102630 0x7f846c102a10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f845c0099c0 tx=0x7f845c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.750+0000 7f8470ad9700 1 -- 172.21.15.59:0/2824001653 shutdown_connections 2024-05-08T05:28:37.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.750+0000 7f8470ad9700 1 --2- 172.21.15.59:0/2824001653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f846c1076c0 0x7f846c10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.750+0000 7f8470ad9700 1 --2- 172.21.15.59:0/2824001653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102630 0x7f846c102a10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.750+0000 7f8470ad9700 1 --2- 172.21.15.59:0/2824001653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f846c102fe0 0x7f846c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.750+0000 7f8470ad9700 1 -- 172.21.15.59:0/2824001653 >> 172.21.15.59:0/2824001653 conn(0x7f846c079350 msgr2=0x7f846c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:37.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.750+0000 7f8470ad9700 1 -- 172.21.15.59:0/2824001653 shutdown_connections 2024-05-08T05:28:37.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.750+0000 7f8470ad9700 1 -- 172.21.15.59:0/2824001653 wait complete. 2024-05-08T05:28:37.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.750+0000 7f8470ad9700 1 Processor -- start 2024-05-08T05:28:37.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.750+0000 7f8470ad9700 1 -- start start 2024-05-08T05:28:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8470ad9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f846c102630 0x7f846c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8470ad9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102fe0 0x7f846c19afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8470ad9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f846c1076c0 0x7f846c1a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8470ad9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f846c112580 con 0x7f846c102fe0 2024-05-08T05:28:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8470ad9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f846c112400 con 0x7f846c102630 2024-05-08T05:28:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8470ad9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f846c112700 con 0x7f846c1076c0 2024-05-08T05:28:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8469d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102fe0 0x7f846c19afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8469d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102fe0 0x7f846c19afa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45196/0 (socket says 172.21.15.59:45196) 2024-05-08T05:28:37.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8469d9b700 1 -- 172.21.15.59:0/1251366382 learned_addr learned my addr 172.21.15.59:0/1251366382 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f846a59c700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f846c102630 0x7f846c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f846ad9d700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f846c1076c0 0x7f846c1a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8469d9b700 1 -- 172.21.15.59:0/1251366382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f846c1076c0 msgr2=0x7f846c1a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8469d9b700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f846c1076c0 0x7f846c1a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8469d9b700 1 -- 172.21.15.59:0/1251366382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f846c102630 msgr2=0x7f846c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8469d9b700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f846c102630 0x7f846c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8469d9b700 1 -- 172.21.15.59:0/1251366382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f846c1a6020 con 0x7f846c102fe0 2024-05-08T05:28:37.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f846ad9d700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f846c1076c0 0x7f846c1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:37.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f8469d9b700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102fe0 0x7f846c19afa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f846000c8f0 tx=0x7f846000cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:37.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.751+0000 7f845b7fe700 1 -- 172.21.15.59:0/1251366382 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460007940 con 0x7f846c102fe0 2024-05-08T05:28:37.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.752+0000 7f8470ad9700 1 -- 172.21.15.59:0/1251366382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f846c1a62b0 con 0x7f846c102fe0 2024-05-08T05:28:37.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.752+0000 7f8470ad9700 1 -- 172.21.15.59:0/1251366382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f846c1a6810 con 0x7f846c102fe0 2024-05-08T05:28:37.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.752+0000 7f845b7fe700 1 -- 172.21.15.59:0/1251366382 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8460018400 con 0x7f846c102fe0 2024-05-08T05:28:37.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.752+0000 7f845b7fe700 1 -- 172.21.15.59:0/1251366382 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8460021650 con 0x7f846c102fe0 2024-05-08T05:28:37.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.754+0000 7f845b7fe700 1 -- 172.21.15.59:0/1251366382 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f8460007ae0 con 0x7f846c102fe0 2024-05-08T05:28:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.755+0000 7f845b7fe700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84540b2900 0x7f84540b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.755+0000 7f845b7fe700 1 -- 172.21.15.59:0/1251366382 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f84600e2240 con 0x7f846c102fe0 2024-05-08T05:28:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.755+0000 7f846a59c700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84540b2900 0x7f84540b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.755+0000 7f847b36cb80 1 Processor -- start 2024-05-08T05:28:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.755+0000 7f847b36cb80 1 -- start start 2024-05-08T05:28:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.755+0000 7f846a59c700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84540b2900 0x7f84540b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f845c009990 tx=0x7f845c00b790 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bb890 0x5628c84bbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84bfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84c0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f847b36cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5628c8409810 con 0x5628c84bfff0 2024-05-08T05:28:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f847b36cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5628c83a8770 con 0x5628c84bb890 2024-05-08T05:28:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f847b36cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5628c84057d0 con 0x5628c84bc1b0 2024-05-08T05:28:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84c0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84c0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45200/0 (socket says 172.21.15.59:45200) 2024-05-08T05:28:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846ad9d700 1 -- 172.21.15.59:0/2175539273 learned_addr learned my addr 172.21.15.59:0/2175539273 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846a59c700 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bb890 0x5628c84bbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f8469d9b700 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84bfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846ad9d700 1 -- 172.21.15.59:0/2175539273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 msgr2=0x5628c84bfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846ad9d700 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84bfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846ad9d700 1 -- 172.21.15.59:0/2175539273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bb890 msgr2=0x5628c84bbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846ad9d700 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bb890 0x5628c84bbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846ad9d700 1 -- 172.21.15.59:0/2175539273 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c8356880 con 0x5628c84bfff0 2024-05-08T05:28:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846a59c700 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bb890 0x5628c84bbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f846ad9d700 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84c0430 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8464009dd0 tx=0x7f84640077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1a86b9c89e3859 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.756+0000 7f8458ff9700 1 -- 172.21.15.59:0/2175539273 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464009e60 con 0x5628c84bfff0 2024-05-08T05:28:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f8458ff9700 1 -- 172.21.15.59:0/2175539273 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8464004bb0 con 0x5628c84bfff0 2024-05-08T05:28:37.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f8458ff9700 1 -- 172.21.15.59:0/2175539273 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464014670 con 0x5628c84bfff0 2024-05-08T05:28:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f847b36cb80 1 -- 172.21.15.59:0/2175539273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 msgr2=0x5628c84c0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84c0430 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f8464009dd0 tx=0x7f84640077e0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f847b36cb80 1 -- 172.21.15.59:0/2175539273 shutdown_connections 2024-05-08T05:28:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84c0430 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bb890 0x5628c84bbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2175539273 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84bfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f847b36cb80 1 -- 172.21.15.59:0/2175539273 >> 172.21.15.59:0/2175539273 conn(0x5628c83e30b0 msgr2=0x5628c84b1c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:37.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f847b36cb80 1 -- 172.21.15.59:0/2175539273 shutdown_connections 2024-05-08T05:28:37.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.757+0000 7f847b36cb80 1 -- 172.21.15.59:0/2175539273 wait complete. 2024-05-08T05:28:37.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f847b36cb80 1 -- start start 2024-05-08T05:28:37.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f847b36cb80 1 Processor -- start 2024-05-08T05:28:37.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84c7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84c4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 0x5628c84c4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f847b36cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5628c84c8220 con 0x5628c84bb890 2024-05-08T05:28:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f847b36cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5628c836fbe0 con 0x5628c84bfff0 2024-05-08T05:28:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f847b36cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5628c84c83a0 con 0x5628c84bc1b0 2024-05-08T05:28:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f847b36cb80 10 client.0 _ll_register_callbacks cb 0x5628c83a18d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:37.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f846a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84c7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f846a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84c7380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45202/0 (socket says 172.21.15.59:45202) 2024-05-08T05:28:37.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f846a59c700 1 -- 172.21.15.59:0/4271416965 learned_addr learned my addr 172.21.15.59:0/4271416965 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:37.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f8469d9b700 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84c4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.758+0000 7f846ad9d700 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 0x5628c84c4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f846a59c700 1 -- 172.21.15.59:0/4271416965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 msgr2=0x5628c84c4720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:37.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f846a59c700 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84c4720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f846a59c700 1 -- 172.21.15.59:0/4271416965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 msgr2=0x5628c84c4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:37.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f846a59c700 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 0x5628c84c4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f846a59c700 1 -- 172.21.15.59:0/4271416965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c83a5640 con 0x5628c84bb890 2024-05-08T05:28:37.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f846a59c700 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84c7380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f845c0099c0 tx=0x7f845c009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa1f110432b40b80 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:37.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f8451ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:37.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f8451ffb700 1 -- 172.21.15.59:0/4271416965 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c034060 con 0x5628c84bb890 2024-05-08T05:28:37.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f8451ffb700 1 -- 172.21.15.59:0/4271416965 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f845c00f460 con 0x5628c84bb890 2024-05-08T05:28:37.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f847b36cb80 1 -- 172.21.15.59:0/4271416965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8464004bb0 con 0x5628c84bb890 2024-05-08T05:28:37.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f8451ffb700 1 -- 172.21.15.59:0/4271416965 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f845c03b050 con 0x5628c84bb890 2024-05-08T05:28:37.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f847b36cb80 10 client.15081 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:37.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f847b36cb80 1 -- 172.21.15.59:0/4271416965 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5628c84d29e0 con 0x5628c84bb890 2024-05-08T05:28:37.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.759+0000 7f8451ffb700 1 -- 172.21.15.59:0/4271416965 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8464004bb0 con 0x5628c84bb890 2024-05-08T05:28:37.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.760+0000 7f8451ffb700 1 -- 172.21.15.59:0/4271416965 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f845c02f640 con 0x5628c84bb890 2024-05-08T05:28:37.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.760+0000 7f847b36cb80 10 client.15081 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:37.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.760+0000 7f847b36cb80 10 client.15081 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T05:28:37.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.760+0000 7f847b36cb80 20 client.15081 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:37.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.760+0000 7f847b36cb80 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5628c84d50c0 0x5628c84d6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.760+0000 7f847b36cb80 4 client.15081 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:28:37.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.760+0000 7f847b36cb80 1 -- 172.21.15.59:0/4271416965 --> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5628c8483600 con 0x5628c84d50c0 2024-05-08T05:28:37.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.760+0000 7f8469d9b700 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5628c84d50c0 0x5628c84d6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.761+0000 7f8469d9b700 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5628c84d50c0 0x5628c84d6530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7b777e05560904fb server_cookie=f59ac1632fb59f54 in_seq=0 out_seq=0 2024-05-08T05:28:37.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.761+0000 7f8451ffb700 10 client.15081 ms_handle_connect on v2:172.21.15.96:6836/780978870 2024-05-08T05:28:37.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.761+0000 7f8451ffb700 1 -- 172.21.15.59:0/4271416965 <== mds.1 v2:172.21.15.96:6836/780978870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5628c8483600 con 0x5628c84d50c0 2024-05-08T05:28:37.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.761+0000 7f8451ffb700 10 client.15081 handle_command_reply: tid=0 2024-05-08T05:28:37.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 client.15081 shutdown 2024-05-08T05:28:37.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 -- 172.21.15.59:0/4271416965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 msgr2=0x5628c84c7380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:37.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84c7380 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f845c0099c0 tx=0x7f845c009310 comp rx=0 tx=0).stop 2024-05-08T05:28:37.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 -- 172.21.15.59:0/4271416965 shutdown_connections 2024-05-08T05:28:37.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5628c84d50c0 0x5628c84d6530 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 0x5628c84c4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84c7380 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f8451ffb700 0 client.15081 ms_handle_reset on v2:172.21.15.96:6836/780978870 2024-05-08T05:28:37.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 --2- 172.21.15.59:0/4271416965 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84c4720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 -- 172.21.15.59:0/4271416965 >> 172.21.15.59:0/4271416965 conn(0x5628c83e30b0 msgr2=0x5628c84ac6d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:37.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 -- 172.21.15.59:0/4271416965 shutdown_connections 2024-05-08T05:28:37.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 1 -- 172.21.15.59:0/4271416965 wait complete. 2024-05-08T05:28:37.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.762+0000 7f847b36cb80 20 client.15081 trim_cache size 0 max 16384 2024-05-08T05:28:37.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:37 smithi096 ceph-mon[34253]: pgmap v396: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:37.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f847b36cb80 1 Processor -- start 2024-05-08T05:28:37.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f847b36cb80 1 -- start start 2024-05-08T05:28:37.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bb890 0x5628c84c3030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bc1b0 0x5628c84c3570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84abe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f847b36cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5628c84d4870 con 0x5628c84bfff0 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f847b36cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5628c84d4f40 con 0x5628c84bc1b0 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f847b36cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5628c84d5610 con 0x5628c84bb890 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f846ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84abe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f846ad9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84abe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45216/0 (socket says 172.21.15.59:45216) 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f846ad9d700 1 -- 172.21.15.59:0/1624565843 learned_addr learned my addr 172.21.15.59:0/1624565843 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f846a59c700 1 --2- 172.21.15.59:0/1624565843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bb890 0x5628c84c3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f8469d9b700 1 --2- 172.21.15.59:0/1624565843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bc1b0 0x5628c84c3570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f846ad9d700 1 -- 172.21.15.59:0/1624565843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bb890 msgr2=0x5628c84c3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f846ad9d700 1 --2- 172.21.15.59:0/1624565843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bb890 0x5628c84c3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f846ad9d700 1 -- 172.21.15.59:0/1624565843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bc1b0 msgr2=0x5628c84c3570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:37.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f846ad9d700 1 --2- 172.21.15.59:0/1624565843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bc1b0 0x5628c84c3570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.791+0000 7f846ad9d700 1 -- 172.21.15.59:0/1624565843 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5628c84c7690 con 0x5628c84bfff0 2024-05-08T05:28:37.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.792+0000 7f846ad9d700 1 --2- 172.21.15.59:0/1624565843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84abe40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8464007be0 tx=0x7f84640096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce56ee2a2103ba48 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:37.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.792+0000 7f8453fff700 1 -- 172.21.15.59:0/1624565843 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464009e60 con 0x5628c84bfff0 2024-05-08T05:28:37.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.792+0000 7f8453fff700 1 -- 172.21.15.59:0/1624565843 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8464006000 con 0x5628c84bfff0 2024-05-08T05:28:37.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.792+0000 7f8453fff700 1 -- 172.21.15.59:0/1624565843 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8464014410 con 0x5628c84bfff0 2024-05-08T05:28:37.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 -- 172.21.15.59:0/1624565843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 msgr2=0x5628c84abe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:37.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 --2- 172.21.15.59:0/1624565843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84abe40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f8464007be0 tx=0x7f84640096c0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 -- 172.21.15.59:0/1624565843 shutdown_connections 2024-05-08T05:28:37.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 --2- 172.21.15.59:0/1624565843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bfff0 0x5628c84abe40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 --2- 172.21.15.59:0/1624565843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bb890 0x5628c84c3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 --2- 172.21.15.59:0/1624565843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bc1b0 0x5628c84c3570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 -- 172.21.15.59:0/1624565843 >> 172.21.15.59:0/1624565843 conn(0x5628c83e30b0 msgr2=0x5628c84d3ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 -- 172.21.15.59:0/1624565843 shutdown_connections 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 -- 172.21.15.59:0/1624565843 wait complete. 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 -- start start 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 Processor -- start 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84be260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84df9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 0x5628c84dfee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5628c84d63b0 con 0x5628c84bb890 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5628c84d5ce0 con 0x5628c84bfff0 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.793+0000 7f847b36cb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5628c84d6a80 con 0x5628c84bc1b0 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f847b36cb80 10 client.0 _ll_register_callbacks cb 0x5628c83a18d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:37.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84be260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84be260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45232/0 (socket says 172.21.15.59:45232) 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846a59c700 1 -- 172.21.15.59:0/2370279398 learned_addr learned my addr 172.21.15.59:0/2370279398 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f8469d9b700 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84df9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846ad9d700 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 0x5628c84dfee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846a59c700 1 -- 172.21.15.59:0/2370279398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 msgr2=0x5628c84df9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846a59c700 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84df9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846a59c700 1 -- 172.21.15.59:0/2370279398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 msgr2=0x5628c84dfee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846a59c700 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 0x5628c84dfee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846a59c700 1 -- 172.21.15.59:0/2370279398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5628c84c1850 con 0x5628c84bb890 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f846a59c700 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84be260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f845c0098e0 tx=0x7f845c01f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ca85e65f0f9da88 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f8451ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f8451ffb700 1 -- 172.21.15.59:0/2370279398 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f845c045020 con 0x5628c84bb890 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f847b36cb80 1 -- 172.21.15.59:0/2370279398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8464006000 con 0x5628c84bb890 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.794+0000 7f8451ffb700 1 -- 172.21.15.59:0/2370279398 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f845c00f460 con 0x5628c84bb890 2024-05-08T05:28:37.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f8451ffb700 1 -- 172.21.15.59:0/2370279398 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f845c050050 con 0x5628c84bb890 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f847b36cb80 10 client.15087 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f847b36cb80 1 -- 172.21.15.59:0/2370279398 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5628c8540970 con 0x5628c84bb890 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f8451ffb700 1 -- 172.21.15.59:0/2370279398 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8464006000 con 0x5628c84bb890 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f8451ffb700 1 -- 172.21.15.59:0/2370279398 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f845c041440 con 0x5628c84bb890 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f847b36cb80 10 client.15087 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f847b36cb80 10 client.15087 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f847b36cb80 20 client.15087 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5628c85455a0 0x5628c85442b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f847b36cb80 4 client.15087 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f847b36cb80 1 -- 172.21.15.59:0/2370279398 --> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] -- command(tid 0: {"prefix": "status"}) -- 0x5628c84171f0 con 0x5628c85455a0 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.795+0000 7f8469d9b700 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5628c85455a0 0x5628c85442b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f8469d9b700 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5628c85455a0 0x5628c85442b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=abb6376f610cd20e server_cookie=85c3fdf5759e5e38 in_seq=0 out_seq=0 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f8451ffb700 10 client.15087 ms_handle_connect on v2:172.21.15.96:6836/780978870 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f8451ffb700 1 -- 172.21.15.59:0/2370279398 <== mds.1 v2:172.21.15.96:6836/780978870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5628c84171f0 con 0x5628c85455a0 2024-05-08T05:28:37.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f8451ffb700 10 client.15087 handle_command_reply: tid=0 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 client.15087 shutdown 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 -- 172.21.15.59:0/2370279398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 msgr2=0x5628c84be260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84be260 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f845c0098e0 tx=0x7f845c01f000 comp rx=0 tx=0).stop 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 -- 172.21.15.59:0/2370279398 shutdown_connections 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5628c85455a0 0x5628c85442b0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5628c84bfff0 0x5628c84dfee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f8451ffb700 0 client.15087 ms_handle_reset on v2:172.21.15.96:6836/780978870 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5628c84bb890 0x5628c84be260 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 --2- 172.21.15.59:0/2370279398 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5628c84bc1b0 0x5628c84df9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 -- 172.21.15.59:0/2370279398 >> 172.21.15.59:0/2370279398 conn(0x5628c83e30b0 msgr2=0x5628c84abe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 -- 172.21.15.59:0/2370279398 shutdown_connections 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.796+0000 7f847b36cb80 1 -- 172.21.15.59:0/2370279398 wait complete. 2024-05-08T05:28:37.802 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f847b36cb80 20 client.15087 trim_cache size 0 max 16384 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 1, 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24443, 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 133.40801600099999, 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:28:37.803 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 218.994876111 2024-05-08T05:28:37.804 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:28:37.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 -- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84540b2900 msgr2=0x7f84540b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:37.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84540b2900 0x7f84540b4dc0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f845c009990 tx=0x7f845c00b790 comp rx=0 tx=0).stop 2024-05-08T05:28:37.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 -- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102fe0 msgr2=0x7f846c19afa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:37.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102fe0 0x7f846c19afa0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f846000c8f0 tx=0x7f846000cc00 comp rx=0 tx=0).stop 2024-05-08T05:28:37.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 -- 172.21.15.59:0/1251366382 shutdown_connections 2024-05-08T05:28:37.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f846c1076c0 0x7f846c1a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84540b2900 0x7f84540b4dc0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f846c102630 0x7f846c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 --2- 172.21.15.59:0/1251366382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f846c102fe0 0x7f846c19afa0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:37.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 -- 172.21.15.59:0/1251366382 >> 172.21.15.59:0/1251366382 conn(0x7f846c079350 msgr2=0x7f846c100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:37.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 -- 172.21.15.59:0/1251366382 shutdown_connections 2024-05-08T05:28:37.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:37.797+0000 7f8470ad9700 1 -- 172.21.15.59:0/1251366382 wait complete. 2024-05-08T05:28:38.599 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-08T05:28:38.860 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:39 smithi143 ceph-mon[34337]: pgmap v397: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:39.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:39 smithi096 ceph-mon[34253]: pgmap v397: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:39.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:39 smithi059 ceph-mon[30304]: pgmap v397: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:40.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.620+0000 7fa0a198c700 1 -- 172.21.15.59:0/1510340046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c108400 msgr2=0x7fa09c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:40.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.620+0000 7fa0a198c700 1 --2- 172.21.15.59:0/1510340046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c108400 0x7fa09c1087e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7fa08c00b0a0 tx=0x7fa08c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.620+0000 7fa0a198c700 1 -- 172.21.15.59:0/1510340046 shutdown_connections 2024-05-08T05:28:40.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.620+0000 7fa0a198c700 1 --2- 172.21.15.59:0/1510340046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09c102d50 0x7fa09c10f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.621 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.620+0000 7fa0a198c700 1 --2- 172.21.15.59:0/1510340046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c108400 0x7fa09c1087e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.620+0000 7fa0a198c700 1 --2- 172.21.15.59:0/1510340046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa09c102390 0x7fa09c102810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.620+0000 7fa0a198c700 1 -- 172.21.15.59:0/1510340046 >> 172.21.15.59:0/1510340046 conn(0x7fa09c079310 msgr2=0x7fa09c079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.620+0000 7fa0a198c700 1 -- 172.21.15.59:0/1510340046 shutdown_connections 2024-05-08T05:28:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.620+0000 7fa0a198c700 1 -- 172.21.15.59:0/1510340046 wait complete. 2024-05-08T05:28:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.621+0000 7fa0a198c700 1 Processor -- start 2024-05-08T05:28:40.622 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.621+0000 7fa0a198c700 1 -- start start 2024-05-08T05:28:40.623 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.621+0000 7fa0a198c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa09c102390 0x7fa09c10e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.621+0000 7fa0a198c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c102d50 0x7fa09c10eb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.621+0000 7fa0a198c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09c108400 0x7fa09c1a49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.621+0000 7fa0a198c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c113670 con 0x7fa09c102d50 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.621+0000 7fa0a198c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c1134f0 con 0x7fa09c102390 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.621+0000 7fa0a198c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa09c1137f0 con 0x7fa09c108400 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.621+0000 7fa09a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c102d50 0x7fa09c10eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c102d50 0x7fa09c10eb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49820/0 (socket says 172.21.15.59:49820) 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09a7fc700 1 -- 172.21.15.59:0/3373011045 learned_addr learned my addr 172.21.15.59:0/3373011045 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09b7fe700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09c108400 0x7fa09c1a49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09affd700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa09c102390 0x7fa09c10e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09a7fc700 1 -- 172.21.15.59:0/3373011045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09c108400 msgr2=0x7fa09c1a49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09c108400 0x7fa09c1a49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09a7fc700 1 -- 172.21.15.59:0/3373011045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa09c102390 msgr2=0x7fa09c10e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:40.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa09c102390 0x7fa09c10e5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09a7fc700 1 -- 172.21.15.59:0/3373011045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa09c1a50f0 con 0x7fa09c102d50 2024-05-08T05:28:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09affd700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa09c102390 0x7fa09c10e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c102d50 0x7fa09c10eb10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa09000c920 tx=0x7fa09000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa0a098a700 1 -- 172.21.15.59:0/3373011045 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090007b00 con 0x7fa09c102d50 2024-05-08T05:28:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa0a098a700 1 -- 172.21.15.59:0/3373011045 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa090004ce0 con 0x7fa09c102d50 2024-05-08T05:28:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa0a198c700 1 -- 172.21.15.59:0/3373011045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa09c1a53e0 con 0x7fa09c102d50 2024-05-08T05:28:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa0a098a700 1 -- 172.21.15.59:0/3373011045 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa090005550 con 0x7fa09c102d50 2024-05-08T05:28:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.622+0000 7fa0a198c700 1 -- 172.21.15.59:0/3373011045 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa09c1acc60 con 0x7fa09c102d50 2024-05-08T05:28:40.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.625+0000 7fa0a098a700 1 -- 172.21.15.59:0/3373011045 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa090007ca0 con 0x7fa09c102d50 2024-05-08T05:28:40.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.626+0000 7fa0a098a700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0840b2880 0x7fa0840b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.626+0000 7fa09affd700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0840b2880 0x7fa0840b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.626+0000 7fa0a098a700 1 -- 172.21.15.59:0/3373011045 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa0900e24a0 con 0x7fa09c102d50 2024-05-08T05:28:40.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.626+0000 7fa09affd700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0840b2880 0x7fa0840b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0062a0 tx=0x7fa08c006230 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:40.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.626+0000 7fa0ac21fb80 1 Processor -- start 2024-05-08T05:28:40.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.626+0000 7fa0ac21fb80 1 -- start start 2024-05-08T05:28:40.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9bf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 0x55b34e9b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9b9d00 0x55b34e9ba140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b34e90b8f0 con 0x55b34e9bf160 2024-05-08T05:28:40.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b34e870d50 con 0x55b34e9bfa80 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b34e9078b0 con 0x55b34e9b9d00 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9bf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9bf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49830/0 (socket says 172.21.15.59:49830) 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09affd700 1 -- 172.21.15.59:0/4193344608 learned_addr learned my addr 172.21.15.59:0/4193344608 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/4193344608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 0x55b34e9b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09b7fe700 1 --2- 172.21.15.59:0/4193344608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9b9d00 0x55b34e9ba140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09affd700 1 -- 172.21.15.59:0/4193344608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9b9d00 msgr2=0x55b34e9ba140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09affd700 1 --2- 172.21.15.59:0/4193344608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9b9d00 0x55b34e9ba140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09affd700 1 -- 172.21.15.59:0/4193344608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 msgr2=0x55b34e9b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09affd700 1 --2- 172.21.15.59:0/4193344608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 0x55b34e9b97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09affd700 1 -- 172.21.15.59:0/4193344608 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b34e858880 con 0x55b34e9bf160 2024-05-08T05:28:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa09affd700 1 --2- 172.21.15.59:0/4193344608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9bf540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa08c015000 tx=0x7fa08c01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d71048427fb257d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.627+0000 7fa081ffb700 1 -- 172.21.15.59:0/4193344608 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c007d00 con 0x55b34e9bf160 2024-05-08T05:28:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa081ffb700 1 -- 172.21.15.59:0/4193344608 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08c02e070 con 0x55b34e9bf160 2024-05-08T05:28:40.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa081ffb700 1 -- 172.21.15.59:0/4193344608 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c0052a0 con 0x55b34e9bf160 2024-05-08T05:28:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/4193344608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 msgr2=0x55b34e9bf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/4193344608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9bf540 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa08c015000 tx=0x7fa08c01c000 comp rx=0 tx=0).stop 2024-05-08T05:28:40.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/4193344608 shutdown_connections 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/4193344608 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9b9d00 0x55b34e9ba140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/4193344608 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9bf540 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/4193344608 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 0x55b34e9b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/4193344608 >> 172.21.15.59:0/4193344608 conn(0x55b34e8e5b20 msgr2=0x55b34e9b4250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/4193344608 shutdown_connections 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.628+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/4193344608 wait complete. 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa0ac21fb80 1 -- start start 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa0ac21fb80 1 Processor -- start 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9b9d00 0x55b34e9c56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bf160 0x55b34e9c2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 0x55b34e9c3000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b34e9c63a0 con 0x55b34e9b9d00 2024-05-08T05:28:40.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b34e8c4e80 con 0x55b34e9bfa80 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b34e9c6520 con 0x55b34e9bf160 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa0ac21fb80 10 client.0 _ll_register_callbacks cb 0x55b34e89f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa09affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9b9d00 0x55b34e9c56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa09affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9b9d00 0x55b34e9c56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49844/0 (socket says 172.21.15.59:49844) 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.629+0000 7fa09affd700 1 -- 172.21.15.59:0/576739978 learned_addr learned my addr 172.21.15.59:0/576739978 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bf160 0x55b34e9c2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa09b7fe700 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 0x55b34e9c3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa09affd700 1 -- 172.21.15.59:0/576739978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bf160 msgr2=0x55b34e9c2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa09affd700 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bf160 0x55b34e9c2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa09affd700 1 -- 172.21.15.59:0/576739978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 msgr2=0x55b34e9c3000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa09affd700 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 0x55b34e9c3000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa09affd700 1 -- 172.21.15.59:0/576739978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b34e8c2710 con 0x55b34e9b9d00 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa09affd700 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9b9d00 0x55b34e9c56d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009d10 tx=0x7fa08c01cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95096f02688ac2a4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa07affd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:40.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa07affd700 1 -- 172.21.15.59:0/576739978 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa08c038020 con 0x55b34e9b9d00 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa07affd700 1 -- 172.21.15.59:0/576739978 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa08c034460 con 0x55b34e9b9d00 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/576739978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa08c02e070 con 0x55b34e9b9d00 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa07affd700 1 -- 172.21.15.59:0/576739978 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa08c043050 con 0x55b34e9b9d00 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa0ac21fb80 10 client.15099 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.630+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/576739978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b34e9d0c30 con 0x55b34e9b9d00 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.631+0000 7fa07affd700 1 -- 172.21.15.59:0/576739978 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa08c02e070 con 0x55b34e9b9d00 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.631+0000 7fa07affd700 1 -- 172.21.15.59:0/576739978 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa08c011070 con 0x55b34e9b9d00 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.631+0000 7fa0ac21fb80 10 client.15099 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.631+0000 7fa0ac21fb80 10 client.15099 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.631+0000 7fa0ac21fb80 20 client.15099 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.631+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b34e9d1f80 0x55b34e9d4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.631+0000 7fa0ac21fb80 4 client.15099 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.631+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/576739978 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b34e9192d0 con 0x55b34e9d1f80 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.631+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b34e9d1f80 0x55b34e9d4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.636 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.632+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b34e9d1f80 0x55b34e9d4130 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b694f25c7f162cb6 server_cookie=7b598ee6fc054a8b in_seq=0 out_seq=0 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.632+0000 7fa07affd700 10 client.15099 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.632+0000 7fa07affd700 1 -- 172.21.15.59:0/576739978 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b34e9192d0 con 0x55b34e9d1f80 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.632+0000 7fa07affd700 10 client.15099 handle_command_reply: tid=0 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.632+0000 7fa0ac21fb80 1 client.15099 shutdown 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.632+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/576739978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9b9d00 msgr2=0x55b34e9c56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.632+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9b9d00 0x55b34e9c56d0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fa08c009d10 tx=0x7fa08c01cbb0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/576739978 shutdown_connections 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b34e9d1f80 0x55b34e9d4130 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9bfa80 0x55b34e9c3000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9b9d00 0x55b34e9c56d0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/576739978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bf160 0x55b34e9c2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa07affd700 0 client.15099 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/576739978 >> 172.21.15.59:0/576739978 conn(0x55b34e8e5b20 msgr2=0x55b34e9aeae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/576739978 shutdown_connections 2024-05-08T05:28:40.637 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/576739978 wait complete. 2024-05-08T05:28:40.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.633+0000 7fa0ac21fb80 20 client.15099 trim_cache size 0 max 16384 2024-05-08T05:28:40.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa0ac21fb80 1 Processor -- start 2024-05-08T05:28:40.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa0ac21fb80 1 -- start start 2024-05-08T05:28:40.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 0x55b34e9b97c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9c0570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 0x55b34e9aebf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b34e9d2470 con 0x55b34e9bf160 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b34e9d2b40 con 0x55b34e9b9d00 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b34e9d3210 con 0x55b34e9bfa80 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa09a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9c0570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa09a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9c0570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49850/0 (socket says 172.21.15.59:49850) 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa09a7fc700 1 -- 172.21.15.59:0/164448265 learned_addr learned my addr 172.21.15.59:0/164448265 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa09affd700 1 --2- 172.21.15.59:0/164448265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 0x55b34e9b97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.654+0000 7fa09a7fc700 1 -- 172.21.15.59:0/164448265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 msgr2=0x55b34e9aebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.655+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/164448265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 0x55b34e9aebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.655+0000 7fa09a7fc700 1 -- 172.21.15.59:0/164448265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 msgr2=0x55b34e9b97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.655+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/164448265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 0x55b34e9b97c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.655+0000 7fa09a7fc700 1 -- 172.21.15.59:0/164448265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b34e9c1030 con 0x55b34e9bf160 2024-05-08T05:28:40.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.655+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/164448265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9c0570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa0900a9430 tx=0x7fa090091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f00e474eb2b50de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:40.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.655+0000 7fa0817fa700 1 -- 172.21.15.59:0/164448265 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0900c2070 con 0x55b34e9bf160 2024-05-08T05:28:40.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.655+0000 7fa0817fa700 1 -- 172.21.15.59:0/164448265 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0900a3e50 con 0x55b34e9bf160 2024-05-08T05:28:40.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.655+0000 7fa0817fa700 1 -- 172.21.15.59:0/164448265 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0900b0e60 con 0x55b34e9bf160 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/164448265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 msgr2=0x55b34e9c0570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/164448265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9c0570 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fa0900a9430 tx=0x7fa090091040 comp rx=0 tx=0).stop 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/164448265 shutdown_connections 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/164448265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 0x55b34e9aebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/164448265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 0x55b34e9b97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/164448265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9c0570 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/164448265 >> 172.21.15.59:0/164448265 conn(0x55b34e8e5b20 msgr2=0x55b34e9bfe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/164448265 shutdown_connections 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/164448265 wait complete. 2024-05-08T05:28:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 -- start start 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.656+0000 7fa0ac21fb80 1 Processor -- start 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 0x55b34e9dd5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9ddaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa0ac21fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 0x55b34e9d8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b34e9d3fb0 con 0x55b34e9bf160 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b34e9d38e0 con 0x55b34e9b9d00 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa0ac21fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b34e9d4680 con 0x55b34e9bfa80 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa0ac21fb80 10 client.0 _ll_register_callbacks cb 0x55b34e89f560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9ddaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09a7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9ddaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49856/0 (socket says 172.21.15.59:49856) 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09a7fc700 1 -- 172.21.15.59:0/986320575 learned_addr learned my addr 172.21.15.59:0/986320575 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09b7fe700 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 0x55b34e9d8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09affd700 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 0x55b34e9dd5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09a7fc700 1 -- 172.21.15.59:0/986320575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 msgr2=0x55b34e9d8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 0x55b34e9d8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09a7fc700 1 -- 172.21.15.59:0/986320575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 msgr2=0x55b34e9dd5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 0x55b34e9dd5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09a7fc700 1 -- 172.21.15.59:0/986320575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b34e9d8b50 con 0x55b34e9bf160 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09b7fe700 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 0x55b34e9d8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09affd700 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 0x55b34e9dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa09a7fc700 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9ddaf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa090005950 tx=0x7fa0900a92f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed71b579d2f74eb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa07affd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa07affd700 1 -- 172.21.15.59:0/986320575 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0900c2040 con 0x55b34e9bf160 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa07affd700 1 -- 172.21.15.59:0/986320575 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa0900a5d20 con 0x55b34e9bf160 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.657+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/986320575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0900a3e50 con 0x55b34e9bf160 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa07affd700 1 -- 172.21.15.59:0/986320575 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0900b0e60 con 0x55b34e9bf160 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa0ac21fb80 10 client.15105 fetch_fsmap learned FSMap version 29 2024-05-08T05:28:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/986320575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b34ea3e9f0 con 0x55b34e9bf160 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa07affd700 1 -- 172.21.15.59:0/986320575 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0900a3e50 con 0x55b34e9bf160 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa07affd700 1 -- 172.21.15.59:0/986320575 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa0900b2100 con 0x55b34e9bf160 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa0ac21fb80 10 client.15105 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa0ac21fb80 10 client.15105 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa0ac21fb80 20 client.15105 populate_metadata read hostname 'smithi059' 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b34ea43170 0x55b34ea41e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa0ac21fb80 4 client.15105 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/986320575 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "status"}) -- 0x55b34e913370 con 0x55b34ea43170 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.658+0000 7fa09affd700 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b34ea43170 0x55b34ea41e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa09affd700 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b34ea43170 0x55b34ea41e80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14b4914e628d8315 server_cookie=110db1b86fe71bfa in_seq=0 out_seq=0 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa07affd700 10 client.15105 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa07affd700 1 -- 172.21.15.59:0/986320575 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x7fa08c021160 con 0x55b34ea43170 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa07affd700 10 client.15105 handle_command_reply: tid=0 2024-05-08T05:28:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 client.15105 shutdown 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/986320575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 msgr2=0x55b34e9ddaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9ddaf0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fa090005950 tx=0x7fa0900a92f0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/986320575 shutdown_connections 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b34ea43170 0x55b34ea41e80 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b34e9bfa80 0x55b34e9d8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b34e9b9d00 0x55b34e9dd5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa07affd700 0 client.15105 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 --2- 172.21.15.59:0/986320575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b34e9bf160 0x55b34e9ddaf0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/986320575 >> 172.21.15.59:0/986320575 conn(0x55b34e8e5b20 msgr2=0x55b34e9c3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/986320575 shutdown_connections 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 1 -- 172.21.15.59:0/986320575 wait complete. 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.659+0000 7fa0ac21fb80 20 client.15105 trim_cache size 0 max 16384 2024-05-08T05:28:40.665 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24473, 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 192.99640710899999, 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 208.91541105600001 2024-05-08T05:28:40.666 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:28:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 -- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0840b2880 msgr2=0x7fa0840b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0840b2880 0x7fa0840b4d40 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fa08c0062a0 tx=0x7fa08c006230 comp rx=0 tx=0).stop 2024-05-08T05:28:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 -- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c102d50 msgr2=0x7fa09c10eb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c102d50 0x7fa09c10eb10 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fa09000c920 tx=0x7fa09000cc30 comp rx=0 tx=0).stop 2024-05-08T05:28:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 -- 172.21.15.59:0/3373011045 shutdown_connections 2024-05-08T05:28:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa09c108400 0x7fa09c1a49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa0840b2880 0x7fa0840b4d40 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa09c102390 0x7fa09c10e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 --2- 172.21.15.59:0/3373011045 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa09c102d50 0x7fa09c10eb10 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 -- 172.21.15.59:0/3373011045 >> 172.21.15.59:0/3373011045 conn(0x7fa09c079310 msgr2=0x7fa09c101110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 -- 172.21.15.59:0/3373011045 shutdown_connections 2024-05-08T05:28:40.668 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:40.660+0000 7fa0a198c700 1 -- 172.21.15.59:0/3373011045 wait complete. 2024-05-08T05:28:41.450 INFO:tasks.cephadm:Setup complete, yielding 2024-05-08T05:28:41.450 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T05:28:41.461 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:28:41.461 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- bash -c 'ceph orch status' 2024-05-08T05:28:41.717 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:41.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:41 smithi143 ceph-mon[34337]: pgmap v398: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:41.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:41 smithi096 ceph-mon[34253]: pgmap v398: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:41.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:41 smithi059 ceph-mon[30304]: pgmap v398: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.444+0000 7f58f44d0700 1 -- 172.21.15.59:0/2925847215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec1068c0 msgr2=0x7f58ec108cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.444+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2925847215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec1068c0 0x7f58ec108cb0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f58dc009990 tx=0x7f58dc009ca0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.444+0000 7f58f44d0700 1 -- 172.21.15.59:0/2925847215 shutdown_connections 2024-05-08T05:28:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.444+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2925847215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ec1091f0 0x7f58ec10b660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.445 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.444+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2925847215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58ec103f90 0x7f58ec106380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.444+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2925847215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec1068c0 0x7f58ec108cb0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.444+0000 7f58f44d0700 1 -- 172.21.15.59:0/2925847215 >> 172.21.15.59:0/2925847215 conn(0x7f58ec0fd9a0 msgr2=0x7f58ec0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.444+0000 7f58f44d0700 1 -- 172.21.15.59:0/2925847215 shutdown_connections 2024-05-08T05:28:43.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.445+0000 7f58f44d0700 1 -- 172.21.15.59:0/2925847215 wait complete. 2024-05-08T05:28:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.445+0000 7f58f44d0700 1 Processor -- start 2024-05-08T05:28:43.447 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.445+0000 7f58f44d0700 1 -- start start 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f44d0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec103f90 0x7f58ec1996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f44d0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58ec1068c0 0x7f58ec199bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f44d0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ec1091f0 0x7f58ec1a0c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f44d0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec10efc0 con 0x7f58ec103f90 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f44d0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec10ee40 con 0x7f58ec1091f0 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f44d0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f58ec10f140 con 0x7f58ec1068c0 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f226c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec103f90 0x7f58ec1996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f226c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec103f90 0x7f58ec1996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49884/0 (socket says 172.21.15.59:49884) 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f226c700 1 -- 172.21.15.59:0/2206458783 learned_addr learned my addr 172.21.15.59:0/2206458783 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f1a6b700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58ec1068c0 0x7f58ec199bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f2a6d700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ec1091f0 0x7f58ec1a0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f226c700 1 -- 172.21.15.59:0/2206458783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58ec1068c0 msgr2=0x7f58ec199bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:43.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f226c700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58ec1068c0 0x7f58ec199bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f226c700 1 -- 172.21.15.59:0/2206458783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ec1091f0 msgr2=0x7f58ec1a0c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f226c700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ec1091f0 0x7f58ec1a0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f226c700 1 -- 172.21.15.59:0/2206458783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58ec1a1310 con 0x7f58ec103f90 2024-05-08T05:28:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.446+0000 7f58f226c700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec103f90 0x7f58ec1996b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f58e800c9a0 tx=0x7f58e800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.447+0000 7f58e37fe700 1 -- 172.21.15.59:0/2206458783 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8007a60 con 0x7f58ec103f90 2024-05-08T05:28:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.447+0000 7f58e37fe700 1 -- 172.21.15.59:0/2206458783 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58e8004ce0 con 0x7f58ec103f90 2024-05-08T05:28:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.447+0000 7f58e37fe700 1 -- 172.21.15.59:0/2206458783 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e8005560 con 0x7f58ec103f90 2024-05-08T05:28:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.447+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58ec1a1600 con 0x7f58ec103f90 2024-05-08T05:28:43.449 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.447+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58ec1a1af0 con 0x7f58ec103f90 2024-05-08T05:28:43.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.448+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58ec06a190 con 0x7f58ec103f90 2024-05-08T05:28:43.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.449+0000 7f58e37fe700 1 -- 172.21.15.59:0/2206458783 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f58e8007c00 con 0x7f58ec103f90 2024-05-08T05:28:43.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.450+0000 7f58e37fe700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f58d80b2880 0x7f58d80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.450+0000 7f58f1a6b700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f58d80b2880 0x7f58d80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.451+0000 7f58e37fe700 1 -- 172.21.15.59:0/2206458783 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f58e80e2350 con 0x7f58ec103f90 2024-05-08T05:28:43.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.451+0000 7f58f1a6b700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f58d80b2880 0x7f58d80b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b5c0 tx=0x7f58dc005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:43.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.455+0000 7f58e37fe700 1 -- 172.21.15.59:0/2206458783 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f58e80a9d10 con 0x7f58ec103f90 2024-05-08T05:28:43.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.691+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7f58ec0638e0 con 0x7f58d80b2880 2024-05-08T05:28:43.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.692+0000 7f58e37fe700 1 -- 172.21.15.59:0/2206458783 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7f58ec0638e0 con 0x7f58d80b2880 2024-05-08T05:28:43.693 INFO:teuthology.orchestra.run.smithi059.stdout:Backend: cephadm 2024-05-08T05:28:43.693 INFO:teuthology.orchestra.run.smithi059.stdout:Available: Yes 2024-05-08T05:28:43.693 INFO:teuthology.orchestra.run.smithi059.stdout:Paused: No 2024-05-08T05:28:43.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f58d80b2880 msgr2=0x7f58d80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:43.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f58d80b2880 0x7f58d80b4d40 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f58dc00b5c0 tx=0x7f58dc005fb0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec103f90 msgr2=0x7f58ec1996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:43.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec103f90 0x7f58ec1996b0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f58e800c9a0 tx=0x7f58e800ccb0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.696 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:43 smithi059 ceph-mon[30304]: pgmap v399: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:43.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 shutdown_connections 2024-05-08T05:28:43.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f58d80b2880 0x7f58d80b4d40 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58ec1091f0 0x7f58ec1a0c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58ec103f90 0x7f58ec1996b0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 --2- 172.21.15.59:0/2206458783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58ec1068c0 0x7f58ec199bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:43.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 >> 172.21.15.59:0/2206458783 conn(0x7f58ec0fd9a0 msgr2=0x7f58ec105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:43.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 shutdown_connections 2024-05-08T05:28:43.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:43.695+0000 7f58f44d0700 1 -- 172.21.15.59:0/2206458783 wait complete. 2024-05-08T05:28:43.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:43 smithi143 ceph-mon[34337]: pgmap v399: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:43.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:43 smithi096 ceph-mon[34253]: pgmap v399: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:44.460 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ps' 2024-05-08T05:28:44.718 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:45.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:45 smithi096 ceph-mon[34253]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:45.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:45 smithi096 ceph-mon[34253]: pgmap v400: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:45.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:45 smithi059 ceph-mon[30304]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:45.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:45 smithi059 ceph-mon[30304]: pgmap v400: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:45.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:45 smithi143 ceph-mon[34337]: from='client.15111 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:45.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:45 smithi143 ceph-mon[34337]: pgmap v400: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.455+0000 7fc166e29700 1 -- 172.21.15.59:0/1184605168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1601050c0 msgr2=0x7fc16010b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.455+0000 7fc166e29700 1 --2- 172.21.15.59:0/1184605168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1601050c0 0x7fc16010b970 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fc15800b0d0 tx=0x7fc15800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.455+0000 7fc166e29700 1 -- 172.21.15.59:0/1184605168 shutdown_connections 2024-05-08T05:28:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.455+0000 7fc166e29700 1 --2- 172.21.15.59:0/1184605168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc1601050c0 0x7fc16010b970 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.455+0000 7fc166e29700 1 --2- 172.21.15.59:0/1184605168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc160103520 0x7fc160103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.455+0000 7fc166e29700 1 --2- 172.21.15.59:0/1184605168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc160104720 0x7fc160104b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.455+0000 7fc166e29700 1 -- 172.21.15.59:0/1184605168 >> 172.21.15.59:0/1184605168 conn(0x7fc1600feac0 msgr2=0x7fc160100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.455+0000 7fc166e29700 1 -- 172.21.15.59:0/1184605168 shutdown_connections 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.455+0000 7fc166e29700 1 -- 172.21.15.59:0/1184605168 wait complete. 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc166e29700 1 Processor -- start 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc166e29700 1 -- start start 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc166e29700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc160103520 0x7fc160077240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc166e29700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc160104720 0x7fc160075890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc166e29700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1601050c0 0x7fc160075dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc166e29700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc16010efa0 con 0x7fc160103520 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc166e29700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc16010ee20 con 0x7fc1601050c0 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc166e29700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc16010f120 con 0x7fc160104720 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc164bc5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc160103520 0x7fc160077240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc164bc5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc160103520 0x7fc160077240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49916/0 (socket says 172.21.15.59:49916) 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.456+0000 7fc164bc5700 1 -- 172.21.15.59:0/706270708 learned_addr learned my addr 172.21.15.59:0/706270708 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc1653c6700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1601050c0 0x7fc160075dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:46.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc15ffff700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc160104720 0x7fc160075890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc164bc5700 1 -- 172.21.15.59:0/706270708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc160104720 msgr2=0x7fc160075890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc164bc5700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc160104720 0x7fc160075890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc164bc5700 1 -- 172.21.15.59:0/706270708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1601050c0 msgr2=0x7fc160075dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc164bc5700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1601050c0 0x7fc160075dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc164bc5700 1 -- 172.21.15.59:0/706270708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc160076680 con 0x7fc160103520 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc1653c6700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1601050c0 0x7fc160075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc164bc5700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc160103520 0x7fc160077240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c970 tx=0x7fc15400cd30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc15dffb700 1 -- 172.21.15.59:0/706270708 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154004220 con 0x7fc160103520 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1601a3720 con 0x7fc160103520 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc15dffb700 1 -- 172.21.15.59:0/706270708 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc154007d50 con 0x7fc160103520 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc15dffb700 1 -- 172.21.15.59:0/706270708 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154003cc0 con 0x7fc160103520 2024-05-08T05:28:46.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.457+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1601a3c10 con 0x7fc160103520 2024-05-08T05:28:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.458+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc16006a190 con 0x7fc160103520 2024-05-08T05:28:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.460+0000 7fc15dffb700 1 -- 172.21.15.59:0/706270708 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc1540043c0 con 0x7fc160103520 2024-05-08T05:28:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.460+0000 7fc15dffb700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc1500b2880 0x7fc1500b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:46.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.460+0000 7fc15ffff700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc1500b2880 0x7fc1500b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:46.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.460+0000 7fc15dffb700 1 -- 172.21.15.59:0/706270708 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc1540e21f0 con 0x7fc160103520 2024-05-08T05:28:46.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.461+0000 7fc15ffff700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc1500b2880 0x7fc1500b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc14c005950 tx=0x7fc14c0058e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:46.466 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.465+0000 7fc15dffb700 1 -- 172.21.15.59:0/706270708 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc154014030 con 0x7fc160103520 2024-05-08T05:28:46.688 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.687+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fc1600638e0 con 0x7fc1500b2880 2024-05-08T05:28:46.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.700+0000 7fc15dffb700 1 -- 172.21.15.59:0/706270708 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4123 (secure 0 0 0) 0x7fc1600638e0 con 0x7fc1500b2880 2024-05-08T05:28:46.702 INFO:teuthology.orchestra.run.smithi059.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-05-08T05:28:46.702 INFO:teuthology.orchestra.run.smithi059.stdout:mds.a smithi059 running (3m) 42s ago 3m 18.1M - 19.0.0-3538-gbee299f4 352576a2df49 d02388b60e7b 2024-05-08T05:28:46.702 INFO:teuthology.orchestra.run.smithi059.stdout:mds.b smithi096 running (3m) 77s ago 3m 28.9M - 19.0.0-3538-gbee299f4 352576a2df49 19a46f73e739 2024-05-08T05:28:46.702 INFO:teuthology.orchestra.run.smithi059.stdout:mds.c smithi143 running (3m) 76s ago 3m 17.5M - 19.0.0-3538-gbee299f4 352576a2df49 8388243eb823 2024-05-08T05:28:46.702 INFO:teuthology.orchestra.run.smithi059.stdout:mds.d smithi059 running (3m) 42s ago 3m 16.0M - 19.0.0-3538-gbee299f4 352576a2df49 16357abb3bc0 2024-05-08T05:28:46.702 INFO:teuthology.orchestra.run.smithi059.stdout:mds.e smithi096 running (3m) 77s ago 3m 17.9M - 19.0.0-3538-gbee299f4 352576a2df49 e2474414a206 2024-05-08T05:28:46.702 INFO:teuthology.orchestra.run.smithi059.stdout:mds.f smithi143 running (3m) 76s ago 3m 18.0M - 19.0.0-3538-gbee299f4 352576a2df49 33cb737861da 2024-05-08T05:28:46.702 INFO:teuthology.orchestra.run.smithi059.stdout:mds.g smithi059 running (3m) 42s ago 3m 17.1M - 19.0.0-3538-gbee299f4 352576a2df49 bbacb2744df2 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mds.h smithi096 running (3m) 77s ago 3m 20.2M - 19.0.0-3538-gbee299f4 352576a2df49 e5e7468a1174 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mds.i smithi143 running (3m) 76s ago 3m 18.6M - 19.0.0-3538-gbee299f4 352576a2df49 dcd9b13ab8a7 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mds.j smithi059 running (4m) 42s ago 4m 16.1M - 19.0.0-3538-gbee299f4 352576a2df49 283f0a6ceacb 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mds.k smithi096 running (4m) 77s ago 4m 18.9M - 19.0.0-3538-gbee299f4 352576a2df49 e8fde80f3d55 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mds.l smithi143 running (4m) 76s ago 4m 17.7M - 19.0.0-3538-gbee299f4 352576a2df49 d840ca7eb67b 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.x smithi059 *:9283,8765 running (11m) 42s ago 11m 531M - 19.0.0-3538-gbee299f4 352576a2df49 f894f8c5035a 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.y smithi096 *:8443,8765 running (9m) 77s ago 9m 466M - 19.0.0-3538-gbee299f4 352576a2df49 02b28c2448fd 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mgr.z smithi143 *:8443,8765 running (9m) 76s ago 9m 468M - 19.0.0-3538-gbee299f4 352576a2df49 8ac990e3137b 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mon.a smithi059 running (11m) 42s ago 11m 50.7M 2048M 19.0.0-3538-gbee299f4 352576a2df49 cc0f25ef8e85 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mon.b smithi096 running (9m) 77s ago 9m 41.0M 2048M 19.0.0-3538-gbee299f4 352576a2df49 f209783c0e38 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:mon.c smithi143 running (9m) 76s ago 9m 39.3M 2048M 19.0.0-3538-gbee299f4 352576a2df49 9a75cd2c5c25 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 smithi059 running (9m) 42s ago 9m 45.5M 4273M 19.0.0-3538-gbee299f4 352576a2df49 bf6fa0040e0f 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 smithi096 running (8m) 77s ago 8m 46.7M 4272M 19.0.0-3538-gbee299f4 352576a2df49 f8890f900aeb 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 smithi143 running (8m) 76s ago 8m 57.5M 4272M 19.0.0-3538-gbee299f4 352576a2df49 67ed697002e7 2024-05-08T05:28:46.703 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 smithi059 running (7m) 42s ago 7m 55.7M 4273M 19.0.0-3538-gbee299f4 352576a2df49 4bec9bbb0851 2024-05-08T05:28:46.704 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 smithi096 running (7m) 77s ago 7m 53.9M 4272M 19.0.0-3538-gbee299f4 352576a2df49 b576282a60ce 2024-05-08T05:28:46.704 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 smithi143 running (7m) 76s ago 7m 59.6M 4272M 19.0.0-3538-gbee299f4 352576a2df49 8054591672b3 2024-05-08T05:28:46.704 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6 smithi059 running (6m) 42s ago 6m 45.5M 4273M 19.0.0-3538-gbee299f4 352576a2df49 df4dc58411b5 2024-05-08T05:28:46.704 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7 smithi096 running (6m) 77s ago 6m 47.2M 4272M 19.0.0-3538-gbee299f4 352576a2df49 c7fc6de95e03 2024-05-08T05:28:46.704 INFO:teuthology.orchestra.run.smithi059.stdout:osd.8 smithi143 running (5m) 76s ago 5m 55.1M 4272M 19.0.0-3538-gbee299f4 352576a2df49 a15af0860e33 2024-05-08T05:28:46.704 INFO:teuthology.orchestra.run.smithi059.stdout:osd.9 smithi059 running (5m) 42s ago 5m 44.1M 4273M 19.0.0-3538-gbee299f4 352576a2df49 bfc811b77a45 2024-05-08T05:28:46.704 INFO:teuthology.orchestra.run.smithi059.stdout:osd.10 smithi096 running (5m) 77s ago 4m 46.9M 4272M 19.0.0-3538-gbee299f4 352576a2df49 c11cd9e27599 2024-05-08T05:28:46.704 INFO:teuthology.orchestra.run.smithi059.stdout:osd.11 smithi143 running (4m) 76s ago 4m 44.5M 4272M 19.0.0-3538-gbee299f4 352576a2df49 b383deb01f3d 2024-05-08T05:28:46.705 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc1500b2880 msgr2=0x7fc1500b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc1500b2880 0x7fc1500b4d40 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc14c005950 tx=0x7fc14c0058e0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc160103520 msgr2=0x7fc160077240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc160103520 0x7fc160077240 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc15400c970 tx=0x7fc15400cd30 comp rx=0 tx=0).stop 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 shutdown_connections 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc1500b2880 0x7fc1500b4d40 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc1601050c0 0x7fc160075dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc160103520 0x7fc160077240 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 --2- 172.21.15.59:0/706270708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc160104720 0x7fc160075890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 >> 172.21.15.59:0/706270708 conn(0x7fc1600feac0 msgr2=0x7fc160107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 shutdown_connections 2024-05-08T05:28:46.706 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:46.704+0000 7fc166e29700 1 -- 172.21.15.59:0/706270708 wait complete. 2024-05-08T05:28:47.401 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- bash -c 'ceph orch ls' 2024-05-08T05:28:47.659 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:47.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:47 smithi096 ceph-mon[34253]: pgmap v401: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:47.782 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:47 smithi059 ceph-mon[30304]: pgmap v401: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:47.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:47 smithi143 ceph-mon[34337]: pgmap v401: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:48.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:48 smithi096 ceph-mon[34253]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:48.802 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:48 smithi059 ceph-mon[30304]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:48.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:48 smithi143 ceph-mon[34337]: from='client.15117 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:49.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.349+0000 7f1b02e2c700 1 -- 172.21.15.59:0/2057715785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 msgr2=0x7f1afc102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.349+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/2057715785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 0x7f1afc102820 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f1ae40099c0 tx=0x7f1ae4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.349+0000 7f1b02e2c700 1 -- 172.21.15.59:0/2057715785 shutdown_connections 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.349+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/2057715785 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1afc103fa0 0x7f1afc10a850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.349+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/2057715785 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 0x7f1afc102820 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.349+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/2057715785 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1afc103600 0x7f1afc103a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.349+0000 7f1b02e2c700 1 -- 172.21.15.59:0/2057715785 >> 172.21.15.59:0/2057715785 conn(0x7f1afc0fd9a0 msgr2=0x7f1afc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.349+0000 7f1b02e2c700 1 -- 172.21.15.59:0/2057715785 shutdown_connections 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.349+0000 7f1b02e2c700 1 -- 172.21.15.59:0/2057715785 wait complete. 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b02e2c700 1 Processor -- start 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b02e2c700 1 -- start start 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b02e2c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 0x7f1afc199740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:49.353 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b02e2c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1afc103600 0x7f1afc199c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b02e2c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1afc103fa0 0x7f1afc1a0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b02e2c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10def0 con 0x7f1afc102400 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b02e2c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10dd70 con 0x7f1afc103600 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b02e2c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1afc10e070 con 0x7f1afc103fa0 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b00bc8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 0x7f1afc199740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b00bc8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 0x7f1afc199740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50176/0 (socket says 172.21.15.59:50176) 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b00bc8700 1 -- 172.21.15.59:0/428466110 learned_addr learned my addr 172.21.15.59:0/428466110 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1afbfff700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1afc103600 0x7f1afc199c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.350+0000 7f1b013c9700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1afc103fa0 0x7f1afc1a0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1b00bc8700 1 -- 172.21.15.59:0/428466110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1afc103fa0 msgr2=0x7f1afc1a0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1b00bc8700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1afc103fa0 0x7f1afc1a0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1b00bc8700 1 -- 172.21.15.59:0/428466110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1afc103600 msgr2=0x7f1afc199c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1b00bc8700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1afc103600 0x7f1afc199c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.354 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1b00bc8700 1 -- 172.21.15.59:0/428466110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1afc1a1350 con 0x7f1afc102400 2024-05-08T05:28:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1b013c9700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1afc103fa0 0x7f1afc1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1b00bc8700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 0x7f1afc199740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b580 tx=0x7f1ae4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1af9ffb700 1 -- 172.21.15.59:0/428466110 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4027070 con 0x7f1afc102400 2024-05-08T05:28:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1af9ffb700 1 -- 172.21.15.59:0/428466110 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ae40040a0 con 0x7f1afc102400 2024-05-08T05:28:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1afc1a15e0 con 0x7f1afc102400 2024-05-08T05:28:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1af9ffb700 1 -- 172.21.15.59:0/428466110 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ae4022da0 con 0x7f1afc102400 2024-05-08T05:28:49.355 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.351+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1afc1a1b10 con 0x7f1afc102400 2024-05-08T05:28:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.352+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1afc06a190 con 0x7f1afc102400 2024-05-08T05:28:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.354+0000 7f1af9ffb700 1 -- 172.21.15.59:0/428466110 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f1ae402a070 con 0x7f1afc102400 2024-05-08T05:28:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.354+0000 7f1af9ffb700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ae80b2900 0x7f1ae80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.354+0000 7f1afbfff700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ae80b2900 0x7f1ae80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.354+0000 7f1af9ffb700 1 -- 172.21.15.59:0/428466110 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1ae40e3910 con 0x7f1afc102400 2024-05-08T05:28:49.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.355+0000 7f1afbfff700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ae80b2900 0x7f1ae80b4dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005950 tx=0x7f1aec00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:49.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.358+0000 7f1af9ffb700 1 -- 172.21.15.59:0/428466110 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1ae4031020 con 0x7f1afc102400 2024-05-08T05:28:49.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.582+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f1afc06b890 con 0x7f1ae80b2900 2024-05-08T05:28:49.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.588+0000 7f1af9ffb700 1 -- 172.21.15.59:0/428466110 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f1afc06b890 con 0x7f1ae80b2900 2024-05-08T05:28:49.590 INFO:teuthology.orchestra.run.smithi059.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-05-08T05:28:49.590 INFO:teuthology.orchestra.run.smithi059.stdout:mds.all 12/12 79s ago 4m smithi059=a;smithi059=d;smithi059=g;smithi059=j;smithi096=b;smithi096=e;smithi096=h;smithi096=k;smithi143=c;smithi143=f;smithi143=i;smithi143=l;count:12 2024-05-08T05:28:49.590 INFO:teuthology.orchestra.run.smithi059.stdout:mgr 3/3 79s ago 9m smithi059=x;smithi096=y;smithi143=z;count:3 2024-05-08T05:28:49.590 INFO:teuthology.orchestra.run.smithi059.stdout:mon 3/3 79s ago 10m smithi059:172.21.15.59=a;smithi096:172.21.15.96=b;smithi143:172.21.15.143=c;count:3 2024-05-08T05:28:49.590 INFO:teuthology.orchestra.run.smithi059.stdout:osd 12 79s ago - 2024-05-08T05:28:49.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ae80b2900 msgr2=0x7f1ae80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:49.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ae80b2900 0x7f1ae80b4dc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f1aec005950 tx=0x7f1aec00b410 comp rx=0 tx=0).stop 2024-05-08T05:28:49.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 msgr2=0x7f1afc199740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:49.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 0x7f1afc199740 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f1ae400b580 tx=0x7f1ae4009580 comp rx=0 tx=0).stop 2024-05-08T05:28:49.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 shutdown_connections 2024-05-08T05:28:49.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1afc103fa0 0x7f1afc1a0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1afc102400 0x7f1afc199740 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f1ae80b2900 0x7f1ae80b4dc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 --2- 172.21.15.59:0/428466110 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1afc103600 0x7f1afc199c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:49.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 >> 172.21.15.59:0/428466110 conn(0x7f1afc0fd9a0 msgr2=0x7f1afc06b180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:49.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 shutdown_connections 2024-05-08T05:28:49.593 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:49.591+0000 7f1b02e2c700 1 -- 172.21.15.59:0/428466110 wait complete. 2024-05-08T05:28:49.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:49 smithi096 ceph-mon[34253]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:49.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:49 smithi059 ceph-mon[30304]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:49.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:49 smithi143 ceph-mon[34337]: pgmap v402: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:50.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- bash -c 'ceph orch host ls' 2024-05-08T05:28:50.542 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:50.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:50 smithi096 ceph-mon[34253]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:50.794 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:50 smithi059 ceph-mon[30304]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:50.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:50 smithi143 ceph-mon[34337]: from='client.15123 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:51.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:51 smithi096 ceph-mon[34253]: pgmap v403: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:51.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:51 smithi059 ceph-mon[30304]: pgmap v403: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:51.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:51 smithi143 ceph-mon[34337]: pgmap v403: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.218+0000 7f8311207700 1 -- 172.21.15.59:0/3375936661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 msgr2=0x7f830c108cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.218+0000 7f8311207700 1 --2- 172.21.15.59:0/3375936661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 0x7f830c108cd0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f82fc009990 tx=0x7f82fc009ca0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.218+0000 7f8311207700 1 -- 172.21.15.59:0/3375936661 shutdown_connections 2024-05-08T05:28:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.218+0000 7f8311207700 1 --2- 172.21.15.59:0/3375936661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f830c109210 0x7f830c10b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.218+0000 7f8311207700 1 --2- 172.21.15.59:0/3375936661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f830c103fb0 0x7f830c1063a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.218+0000 7f8311207700 1 --2- 172.21.15.59:0/3375936661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 0x7f830c108cd0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.218+0000 7f8311207700 1 -- 172.21.15.59:0/3375936661 >> 172.21.15.59:0/3375936661 conn(0x7f830c0fda00 msgr2=0x7f830c0ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.218+0000 7f8311207700 1 -- 172.21.15.59:0/3375936661 shutdown_connections 2024-05-08T05:28:52.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.218+0000 7f8311207700 1 -- 172.21.15.59:0/3375936661 wait complete. 2024-05-08T05:28:52.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f8311207700 1 Processor -- start 2024-05-08T05:28:52.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f8311207700 1 -- start start 2024-05-08T05:28:52.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f8311207700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f830c103fb0 0x7f830c199660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:52.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f8311207700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 0x7f830c199ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:52.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f8311207700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f830c109210 0x7f830c1a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:52.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f8311207700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10efe0 con 0x7f830c1068e0 2024-05-08T05:28:52.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f8311207700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10ee60 con 0x7f830c103fb0 2024-05-08T05:28:52.222 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f8311207700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f830c10f160 con 0x7f830c109210 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f830a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 0x7f830c199ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f830a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 0x7f830c199ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50180/0 (socket says 172.21.15.59:50180) 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f830a59c700 1 -- 172.21.15.59:0/3067388102 learned_addr learned my addr 172.21.15.59:0/3067388102 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.219+0000 7f830ad9d700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f830c103fb0 0x7f830c199660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f830b59e700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f830c109210 0x7f830c1a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f830a59c700 1 -- 172.21.15.59:0/3067388102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f830c109210 msgr2=0x7f830c1a0c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f830a59c700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f830c109210 0x7f830c1a0c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f830a59c700 1 -- 172.21.15.59:0/3067388102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f830c103fb0 msgr2=0x7f830c199660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f830a59c700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f830c103fb0 0x7f830c199660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f830a59c700 1 -- 172.21.15.59:0/3067388102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f830c1a12c0 con 0x7f830c1068e0 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f830a59c700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 0x7f830c199ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0036f0 tx=0x7f82fc0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f8303fff700 1 -- 172.21.15.59:0/3067388102 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc027070 con 0x7f830c1068e0 2024-05-08T05:28:52.223 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f8303fff700 1 -- 172.21.15.59:0/3067388102 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82fc004080 con 0x7f830c1068e0 2024-05-08T05:28:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f830c1a1550 con 0x7f830c1068e0 2024-05-08T05:28:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f8303fff700 1 -- 172.21.15.59:0/3067388102 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82fc022d50 con 0x7f830c1068e0 2024-05-08T05:28:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.220+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f830c1a1a00 con 0x7f830c1068e0 2024-05-08T05:28:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.221+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f830c06a190 con 0x7f830c1068e0 2024-05-08T05:28:52.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.223+0000 7f8303fff700 1 -- 172.21.15.59:0/3067388102 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f82fc02a070 con 0x7f830c1068e0 2024-05-08T05:28:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.223+0000 7f8303fff700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82f80b2900 0x7f82f80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.224+0000 7f830ad9d700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82f80b2900 0x7f82f80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.224+0000 7f8303fff700 1 -- 172.21.15.59:0/3067388102 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f82fc0e3ba0 con 0x7f830c1068e0 2024-05-08T05:28:52.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.224+0000 7f830ad9d700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82f80b2900 0x7f82f80b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005950 tx=0x7f82f400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:52.228 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.227+0000 7f8303fff700 1 -- 172.21.15.59:0/3067388102 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f82fc031070 con 0x7f830c1068e0 2024-05-08T05:28:52.446 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.445+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f830c0638e0 con 0x7f82f80b2900 2024-05-08T05:28:52.448 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.446+0000 7f8303fff700 1 -- 172.21.15.59:0/3067388102 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f830c0638e0 con 0x7f82f80b2900 2024-05-08T05:28:52.448 INFO:teuthology.orchestra.run.smithi059.stdout:HOST ADDR LABELS STATUS 2024-05-08T05:28:52.448 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059 172.21.15.59 2024-05-08T05:28:52.448 INFO:teuthology.orchestra.run.smithi059.stdout:smithi096 172.21.15.96 2024-05-08T05:28:52.448 INFO:teuthology.orchestra.run.smithi059.stdout:smithi143 172.21.15.143 2024-05-08T05:28:52.448 INFO:teuthology.orchestra.run.smithi059.stdout:3 hosts in cluster 2024-05-08T05:28:52.450 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82f80b2900 msgr2=0x7f82f80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82f80b2900 0x7f82f80b4dc0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f82f4005950 tx=0x7f82f400a300 comp rx=0 tx=0).stop 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 msgr2=0x7f830c199ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 0x7f830c199ba0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f82fc0036f0 tx=0x7f82fc0047b0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 shutdown_connections 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f830c109210 0x7f830c1a0c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f82f80b2900 0x7f82f80b4dc0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f830c103fb0 0x7f830c199660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 --2- 172.21.15.59:0/3067388102 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f830c1068e0 0x7f830c199ba0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 >> 172.21.15.59:0/3067388102 conn(0x7f830c0fda00 msgr2=0x7f830c104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 shutdown_connections 2024-05-08T05:28:52.451 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:52.449+0000 7f8311207700 1 -- 172.21.15.59:0/3067388102 wait complete. 2024-05-08T05:28:53.248 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- bash -c 'ceph orch device ls' 2024-05-08T05:28:53.510 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:53.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:53 smithi059 ceph-mon[30304]: pgmap v404: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:53.769 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:53 smithi059 ceph-mon[30304]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:53.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:53 smithi096 ceph-mon[34253]: pgmap v404: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:53.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:53 smithi096 ceph-mon[34253]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:53.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:53 smithi143 ceph-mon[34337]: pgmap v404: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:28:53.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:53 smithi143 ceph-mon[34337]: from='client.15129 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:55.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.250+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2560122483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 msgr2=0x7fa9bc108d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.250+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2560122483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 0x7fa9bc108d70 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00b0a0 tx=0x7fa9ac00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.250+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2560122483 shutdown_connections 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.250+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2560122483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9bc1092b0 0x7fa9bc10b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.250+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2560122483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9bc104050 0x7fa9bc106440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.250+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2560122483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 0x7fa9bc108d70 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.250+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2560122483 >> 172.21.15.59:0/2560122483 conn(0x7fa9bc0fd9a0 msgr2=0x7fa9bc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.250+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2560122483 shutdown_connections 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.250+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2560122483 wait complete. 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9c2ef6700 1 Processor -- start 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9c2ef6700 1 -- start start 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9c2ef6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9bc104050 0x7fa9bc1996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9c2ef6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 0x7fa9bc199c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:55.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9c2ef6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9bc1092b0 0x7fa9bc1a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9c2ef6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10eff0 con 0x7fa9bc106980 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9c2ef6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10ee70 con 0x7fa9bc1092b0 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9c2ef6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa9bc10f170 con 0x7fa9bc104050 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9bbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 0x7fa9bc199c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9bbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 0x7fa9bc199c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50206/0 (socket says 172.21.15.59:50206) 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9bbfff700 1 -- 172.21.15.59:0/2393913482 learned_addr learned my addr 172.21.15.59:0/2393913482 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.251+0000 7fa9c0c92700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9bc104050 0x7fa9bc1996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9c1493700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9bc1092b0 0x7fa9bc1a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9bbfff700 1 -- 172.21.15.59:0/2393913482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9bc104050 msgr2=0x7fa9bc1996c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9bbfff700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9bc104050 0x7fa9bc1996c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9bbfff700 1 -- 172.21.15.59:0/2393913482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9bc1092b0 msgr2=0x7fa9bc1a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9bbfff700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9bc1092b0 0x7fa9bc1a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9bbfff700 1 -- 172.21.15.59:0/2393913482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9bc101e40 con 0x7fa9bc106980 2024-05-08T05:28:55.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9c1493700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9bc1092b0 0x7fa9bc1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:28:55.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9bbfff700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 0x7fa9bc199c00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00b070 tx=0x7fa9ac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:55.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9b9ffb700 1 -- 172.21.15.59:0/2393913482 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac021070 con 0x7fa9bc106980 2024-05-08T05:28:55.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9b9ffb700 1 -- 172.21.15.59:0/2393913482 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ac007de0 con 0x7fa9bc106980 2024-05-08T05:28:55.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9bc1020d0 con 0x7fa9bc106980 2024-05-08T05:28:55.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9b9ffb700 1 -- 172.21.15.59:0/2393913482 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ac01d450 con 0x7fa9bc106980 2024-05-08T05:28:55.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.252+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9bc1026e0 con 0x7fa9bc106980 2024-05-08T05:28:55.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.253+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa9bc06a190 con 0x7fa9bc106980 2024-05-08T05:28:55.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.255+0000 7fa9b9ffb700 1 -- 172.21.15.59:0/2393913482 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fa9ac003a00 con 0x7fa9bc106980 2024-05-08T05:28:55.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.255+0000 7fa9b9ffb700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa9a80b2900 0x7fa9a80b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:55.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.256+0000 7fa9b9ffb700 1 -- 172.21.15.59:0/2393913482 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fa9ac00f020 con 0x7fa9bc106980 2024-05-08T05:28:55.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.256+0000 7fa9c0c92700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa9a80b2900 0x7fa9a80b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:55.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.256+0000 7fa9c0c92700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa9a80b2900 0x7fa9a80b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4005950 tx=0x7fa9a400a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:55.260 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.259+0000 7fa9b9ffb700 1 -- 172.21.15.59:0/2393913482 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa9ac02a020 con 0x7fa9bc106980 2024-05-08T05:28:55.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.482+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7fa9bc0ff010 con 0x7fa9a80b2900 2024-05-08T05:28:55.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.484+0000 7fa9b9ffb700 1 -- 172.21.15.59:0/2393913482 <== mgr.14152 v2:172.21.15.59:6800/546809822 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7fa9bc0ff010 con 0x7fa9a80b2900 2024-05-08T05:28:55.486 INFO:teuthology.orchestra.run.smithi059.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-05-08T05:28:55.486 INFO:teuthology.orchestra.run.smithi059.stdout:smithi059 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT5330009L400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-08T05:28:55.486 INFO:teuthology.orchestra.run.smithi059.stdout:smithi096 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400NH400BGN 372G No 5m ago Has a FileSystem, LVM detected 2024-05-08T05:28:55.486 INFO:teuthology.orchestra.run.smithi059.stdout:smithi143 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT62330073400BGN 372G No 4m ago Has a FileSystem, LVM detected 2024-05-08T05:28:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa9a80b2900 msgr2=0x7fa9a80b4dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa9a80b2900 0x7fa9a80b4dc0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa9a4005950 tx=0x7fa9a400a300 comp rx=0 tx=0).stop 2024-05-08T05:28:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 msgr2=0x7fa9bc199c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 0x7fa9bc199c00 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fa9ac00b070 tx=0x7fa9ac00cf60 comp rx=0 tx=0).stop 2024-05-08T05:28:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 shutdown_connections 2024-05-08T05:28:55.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fa9a80b2900 0x7fa9a80b4dc0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa9bc1092b0 0x7fa9bc1a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa9bc104050 0x7fa9bc1996c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 --2- 172.21.15.59:0/2393913482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa9bc106980 0x7fa9bc199c00 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 >> 172.21.15.59:0/2393913482 conn(0x7fa9bc0fd9a0 msgr2=0x7fa9bc104ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 shutdown_connections 2024-05-08T05:28:55.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:55.487+0000 7fa9c2ef6700 1 -- 172.21.15.59:0/2393913482 wait complete. 2024-05-08T05:28:55.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:55 smithi096 ceph-mon[34253]: pgmap v405: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:55.789 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:55 smithi059 ceph-mon[30304]: pgmap v405: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:55.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:55 smithi143 ceph-mon[34337]: pgmap v405: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:28:56.219 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-05-08T05:28:56.228 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:28:56.228 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- bash -c 'ceph fs dump' 2024-05-08T05:28:56.495 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:56.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:56 smithi096 ceph-mon[34253]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:56.804 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:56 smithi059 ceph-mon[30304]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:56.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:56 smithi143 ceph-mon[34337]: from='client.15135 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:28:57.648 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:57 smithi059 ceph-mon[30304]: pgmap v406: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:57.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:57 smithi096 ceph-mon[34253]: pgmap v406: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:57.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:57 smithi143 ceph-mon[34337]: pgmap v406: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:28:58.181 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.178+0000 7fb58e3f5700 1 -- 172.21.15.59:0/953819542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 msgr2=0x7fb588104b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.178+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/953819542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 0x7fb588104b80 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7fb57c0099c0 tx=0x7fb57c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.178+0000 7fb58e3f5700 1 -- 172.21.15.59:0/953819542 shutdown_connections 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.178+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/953819542 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb5881050c0 0x7fb58810b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.178+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/953819542 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb588103520 0x7fb588103940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.178+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/953819542 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 0x7fb588104b80 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.178+0000 7fb58e3f5700 1 -- 172.21.15.59:0/953819542 >> 172.21.15.59:0/953819542 conn(0x7fb5880feac0 msgr2=0x7fb588100f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.178+0000 7fb58e3f5700 1 -- 172.21.15.59:0/953819542 shutdown_connections 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.178+0000 7fb58e3f5700 1 -- 172.21.15.59:0/953819542 wait complete. 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb58e3f5700 1 Processor -- start 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb58e3f5700 1 -- start start 2024-05-08T05:28:58.182 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb58e3f5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb588103520 0x7fb588199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb58e3f5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 0x7fb588199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb58e3f5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5881050c0 0x7fb5881a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb58e3f5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810f010 con 0x7fb588104720 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb58e3f5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810ee90 con 0x7fb588103520 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb58e3f5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb58810f190 con 0x7fb5881050c0 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb5877fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 0x7fb588199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb5877fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 0x7fb588199b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50228/0 (socket says 172.21.15.59:50228) 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb5877fe700 1 -- 172.21.15.59:0/3801807377 learned_addr learned my addr 172.21.15.59:0/3801807377 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb587fff700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb588103520 0x7fb588199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb58c992700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5881050c0 0x7fb5881a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.179+0000 7fb5877fe700 1 -- 172.21.15.59:0/3801807377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5881050c0 msgr2=0x7fb5881a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb5877fe700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5881050c0 0x7fb5881a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.183 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb5877fe700 1 -- 172.21.15.59:0/3801807377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb588103520 msgr2=0x7fb588199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb5877fe700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb588103520 0x7fb588199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb5877fe700 1 -- 172.21.15.59:0/3801807377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5881a1280 con 0x7fb588104720 2024-05-08T05:28:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb5877fe700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 0x7fb588199b60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005910 tx=0x7fb57c005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb5857fa700 1 -- 172.21.15.59:0/3801807377 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c027070 con 0x7fb588104720 2024-05-08T05:28:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5881a1510 con 0x7fb588104720 2024-05-08T05:28:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb5857fa700 1 -- 172.21.15.59:0/3801807377 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb57c022470 con 0x7fb588104720 2024-05-08T05:28:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb5857fa700 1 -- 172.21.15.59:0/3801807377 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb57c00f6c0 con 0x7fb588104720 2024-05-08T05:28:58.184 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.180+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5881a19f0 con 0x7fb588104720 2024-05-08T05:28:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.181+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb58806a190 con 0x7fb588104720 2024-05-08T05:28:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.182+0000 7fb5857fa700 1 -- 172.21.15.59:0/3801807377 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fb57c02a070 con 0x7fb588104720 2024-05-08T05:28:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.183+0000 7fb5857fa700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb5700b2880 0x7fb5700b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:28:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.183+0000 7fb587fff700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb5700b2880 0x7fb5700b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:28:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.183+0000 7fb5857fa700 1 -- 172.21.15.59:0/3801807377 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fb57c0e38b0 con 0x7fb588104720 2024-05-08T05:28:58.185 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.184+0000 7fb587fff700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb5700b2880 0x7fb5700b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb588104580 tx=0x7fb57800a3b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:28:58.188 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.187+0000 7fb5857fa700 1 -- 172.21.15.59:0/3801807377 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb57c031080 con 0x7fb588104720 2024-05-08T05:28:58.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.457+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7fb58802e510 con 0x7fb588104720 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.457+0000 7fb5857fa700 1 -- 172.21.15.59:0/3801807377 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 29 v29) ==== 76+0+3090 (secure 0 0 0) 0x7fb57c013070 con 0x7fb588104720 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stdout:e29 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stdout:legacy client fscid: 1 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stdout:Filesystem 'cephfs' (1) 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stdout:fs_name cephfs 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 29 2024-05-08T05:28:58.459 INFO:teuthology.orchestra.run.smithi059.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-05-08T05:25:27.601131+0000 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2024-05-08T05:26:31.316568+0000 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:tableserver 0 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:root 0 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:session_timeout 300 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:session_autoclose 300 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:max_file_size 1099511627776 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:max_xattr_size 65536 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:required_client_features {} 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:last_failure 0 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:last_failure_osd_epoch 0 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:max_mds 5 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:in 0,1,2,3,4 2024-05-08T05:28:58.460 INFO:teuthology.orchestra.run.smithi059.stdout:up {0=24473,1=24443,2=24433,3=24421,4=24419} 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:failed 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:damaged 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:stopped 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:data_pools [3,4] 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:metadata_pool 2 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:inline_data disabled 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:balancer 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:bal_rank_mask -1 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:standby_count_wanted 0 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:qdb_cluster leader: 24473 members: 24419,24433,24421,24443,24473 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.b{0:24473} state up:active seq 5 addr [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.f{0:24415} state up:standby-replay seq 1 addr [v2:172.21.15.143:6834/2504523054,v1:172.21.15.143:6835/2504523054] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.h{1:24443} state up:active seq 23 addr [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.461 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.k{1:24401} state up:standby-replay seq 1 addr [v2:172.21.15.96:6832/2593895617,v1:172.21.15.96:6833/2593895617] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.i{2:24433} state up:active seq 22 addr [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.l{2:24385} state up:standby-replay seq 1 addr [v2:172.21.15.143:6832/1879950931,v1:172.21.15.143:6833/1879950931] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.c{3:24421} state up:active seq 24 addr [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.a{3:14571} state up:standby-replay seq 1 addr [v2:172.21.15.59:6840/2496793439,v1:172.21.15.59:6841/2496793439] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.e{4:24419} state up:active seq 26 addr [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.g{4:14565} state up:standby-replay seq 1 addr [v2:172.21.15.59:6838/2756426260,v1:172.21.15.59:6839/2756426260] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout:Standby daemons: 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.j{-1:14517} state up:standby seq 1 addr [v2:172.21.15.59:6834/3565849854,v1:172.21.15.59:6835/3565849854] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.462 INFO:teuthology.orchestra.run.smithi059.stdout:[mds.d{-1:14529} state up:standby seq 1 addr [v2:172.21.15.59:6836/4212370259,v1:172.21.15.59:6837/4212370259] compat {c=[1],r=[1],i=[1fff]}] 2024-05-08T05:28:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb5700b2880 msgr2=0x7fb5700b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb5700b2880 0x7fb5700b4d40 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fb588104580 tx=0x7fb57800a3b0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 msgr2=0x7fb588199b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:28:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 0x7fb588199b60 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb57c005910 tx=0x7fb57c005e80 comp rx=0 tx=0).stop 2024-05-08T05:28:58.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 shutdown_connections 2024-05-08T05:28:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb5881050c0 0x7fb5881a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fb5700b2880 0x7fb5700b4d40 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb588103520 0x7fb588199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 --2- 172.21.15.59:0/3801807377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb588104720 0x7fb588199b60 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:28:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 >> 172.21.15.59:0/3801807377 conn(0x7fb5880feac0 msgr2=0x7fb588107a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:28:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 shutdown_connections 2024-05-08T05:28:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:28:58.460+0000 7fb58e3f5700 1 -- 172.21.15.59:0/3801807377 wait complete. 2024-05-08T05:28:58.464 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 29 2024-05-08T05:28:58.832 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:58 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3801807377' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T05:28:58.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:58 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3801807377' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T05:28:59.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:58 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3801807377' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-05-08T05:28:59.195 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- bash -c 'ceph osd dump' 2024-05-08T05:28:59.451 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:28:59.819 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:28:59 smithi059 ceph-mon[30304]: pgmap v407: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:28:59.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:28:59 smithi143 ceph-mon[34337]: pgmap v407: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:29:00.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:28:59 smithi096 ceph-mon[34253]: pgmap v407: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:29:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.158+0000 7f6f3ce72700 1 -- 172.21.15.59:0/2565203009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 msgr2=0x7f6f3810ba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.158+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/2565203009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 0x7f6f3810ba20 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6f30009990 tx=0x7f6f30009ca0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.158+0000 7f6f3ce72700 1 -- 172.21.15.59:0/2565203009 shutdown_connections 2024-05-08T05:29:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.158+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/2565203009 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 0x7f6f3810ba20 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.158+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/2565203009 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f381017f0 0x7f6f38101c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.158+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/2565203009 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f381047d0 0x7f6f38104c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.160 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.158+0000 7f6f3ce72700 1 -- 172.21.15.59:0/2565203009 >> 172.21.15.59:0/2565203009 conn(0x7f6f380fd150 msgr2=0x7f6f380ff590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.158+0000 7f6f3ce72700 1 -- 172.21.15.59:0/2565203009 shutdown_connections 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.159+0000 7f6f3ce72700 1 -- 172.21.15.59:0/2565203009 wait complete. 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f3ce72700 1 Processor -- start 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f3ce72700 1 -- start start 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f3ce72700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f381017f0 0x7f6f38104130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f3ce72700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f381047d0 0x7f6f38102780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f3ce72700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 0x7f6f38102cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f3ce72700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3810efa0 con 0x7f6f38105170 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f3ce72700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3810ee20 con 0x7f6f381017f0 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f3ce72700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6f3810f120 con 0x7f6f381047d0 2024-05-08T05:29:01.162 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f36d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 0x7f6f38102cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f36d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 0x7f6f38102cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51774/0 (socket says 172.21.15.59:51774) 2024-05-08T05:29:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f36d9d700 1 -- 172.21.15.59:0/1764966280 learned_addr learned my addr 172.21.15.59:0/1764966280 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.160+0000 7f6f3659c700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f381017f0 0x7f6f38104130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f35d9b700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f381047d0 0x7f6f38102780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f36d9d700 1 -- 172.21.15.59:0/1764966280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f381047d0 msgr2=0x7f6f38102780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f36d9d700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f381047d0 0x7f6f38102780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f36d9d700 1 -- 172.21.15.59:0/1764966280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f381017f0 msgr2=0x7f6f38104130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f36d9d700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f381017f0 0x7f6f38104130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.163 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f36d9d700 1 -- 172.21.15.59:0/1764966280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f38103570 con 0x7f6f38105170 2024-05-08T05:29:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f36d9d700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 0x7f6f38102cc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6f300036f0 tx=0x7f6f300047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f277fe700 1 -- 172.21.15.59:0/1764966280 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30027070 con 0x7f6f38105170 2024-05-08T05:29:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f277fe700 1 -- 172.21.15.59:0/1764966280 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f300040c0 con 0x7f6f38105170 2024-05-08T05:29:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f381a36c0 con 0x7f6f38105170 2024-05-08T05:29:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f277fe700 1 -- 172.21.15.59:0/1764966280 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f30022e50 con 0x7f6f38105170 2024-05-08T05:29:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.161+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f381a3ac0 con 0x7f6f38105170 2024-05-08T05:29:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.162+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6f3806a190 con 0x7f6f38105170 2024-05-08T05:29:01.164 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.163+0000 7f6f277fe700 1 -- 172.21.15.59:0/1764966280 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f6f3002a070 con 0x7f6f38105170 2024-05-08T05:29:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.164+0000 7f6f277fe700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6f200b2a20 0x7f6f200b4ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.165+0000 7f6f3659c700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6f200b2a20 0x7f6f200b4ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.165+0000 7f6f277fe700 1 -- 172.21.15.59:0/1764966280 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f6f300e38a0 con 0x7f6f38105170 2024-05-08T05:29:01.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.165+0000 7f6f3659c700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6f200b2a20 0x7f6f200b4ee0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009550 tx=0x7f6f280091b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:01.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.168+0000 7f6f277fe700 1 -- 172.21.15.59:0/1764966280 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6f30013070 con 0x7f6f38105170 2024-05-08T05:29:01.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.382+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f6f380640b0 con 0x7f6f38105170 2024-05-08T05:29:01.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.383+0000 7f6f277fe700 1 -- 172.21.15.59:0/1764966280 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v90) ==== 56+0+5632 (secure 0 0 0) 0x7f6f300ab2c0 con 0x7f6f38105170 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:epoch 90 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:created 2024-05-08T05:17:00.521511+0000 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:modified 2024-05-08T05:27:05.550089+0000 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:crush_version 27 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:full_ratio 0.95 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:backfillfull_ratio 0.9 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:nearfull_ratio 0.85 2024-05-08T05:29:01.385 INFO:teuthology.orchestra.run.smithi059.stdout:require_min_compat_client luminous 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:min_compat_client luminous 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:require_osd_release squid 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:stretch_mode_enabled false 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 22 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 79 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 86 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:max_osd 12 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:osd.0 up in weight 1 up_from 8 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6802/4120729833,v1:172.21.15.59:6803/4120729833] [v2:172.21.15.59:6804/4120729833,v1:172.21.15.59:6805/4120729833] exists,up 2bd987ee-e978-4d9b-89b6-6acca697cac4 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:osd.1 up in weight 1 up_from 13 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6800/2776039768,v1:172.21.15.96:6801/2776039768] [v2:172.21.15.96:6802/2776039768,v1:172.21.15.96:6803/2776039768] exists,up 855aa3c7-8818-419d-8a7d-078e3bd10535 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6800/815254096,v1:172.21.15.143:6801/815254096] [v2:172.21.15.143:6802/815254096,v1:172.21.15.143:6803/815254096] exists,up a05037ce-9958-4ac3-9377-9a6bef15510d 2024-05-08T05:29:01.386 INFO:teuthology.orchestra.run.smithi059.stdout:osd.3 up in weight 1 up_from 25 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6810/3135413737,v1:172.21.15.59:6811/3135413737] [v2:172.21.15.59:6812/3135413737,v1:172.21.15.59:6813/3135413737] exists,up 40198b3b-3215-4d59-9a89-4c18fa5271b1 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:osd.4 up in weight 1 up_from 31 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6808/2351393060,v1:172.21.15.96:6809/2351393060] [v2:172.21.15.96:6810/2351393060,v1:172.21.15.96:6811/2351393060] exists,up 328d5640-5ab1-4baf-b141-039dce5211c9 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:osd.5 up in weight 1 up_from 36 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6808/1078449086,v1:172.21.15.143:6809/1078449086] [v2:172.21.15.143:6810/1078449086,v1:172.21.15.143:6811/1078449086] exists,up 592fc3c9-1310-441d-b834-bec5021dba8e 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:osd.6 up in weight 1 up_from 42 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6818/126911389,v1:172.21.15.59:6819/126911389] [v2:172.21.15.59:6820/126911389,v1:172.21.15.59:6821/126911389] exists,up bce63f2d-f8c7-4304-bec8-8f1d803e2791 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:osd.7 up in weight 1 up_from 47 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6816/2845324468,v1:172.21.15.96:6817/2845324468] [v2:172.21.15.96:6818/2845324468,v1:172.21.15.96:6819/2845324468] exists,up f10748de-9037-46bd-9db3-1d88f6f93af8 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:osd.8 up in weight 1 up_from 54 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6816/2760087187,v1:172.21.15.143:6817/2760087187] [v2:172.21.15.143:6818/2760087187,v1:172.21.15.143:6819/2760087187] exists,up 6e84c2e9-f309-4779-8e07-0b13fc05d3c1 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:osd.9 up in weight 1 up_from 60 up_thru 82 down_at 0 last_clean_interval [0,0) [v2:172.21.15.59:6826/440517197,v1:172.21.15.59:6827/440517197] [v2:172.21.15.59:6828/440517197,v1:172.21.15.59:6829/440517197] exists,up d6035037-1d0b-49dd-ae6c-2708beba9c2b 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:osd.10 up in weight 1 up_from 65 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.96:6824/3187941389,v1:172.21.15.96:6825/3187941389] [v2:172.21.15.96:6826/3187941389,v1:172.21.15.96:6827/3187941389] exists,up 18a53f85-31e5-4969-b393-9dfc48d9aaa3 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:osd.11 up in weight 1 up_from 71 up_thru 89 down_at 0 last_clean_interval [0,0) [v2:172.21.15.143:6824/1379436601,v1:172.21.15.143:6825/1379436601] [v2:172.21.15.143:6826/1379436601,v1:172.21.15.143:6827/1379436601] exists,up c1c3b9ed-8849-4a99-a885-fed187c8aaf7 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.5 [8,11] 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.e [2,1] 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.10 [2,4] 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 2.1d [8,11] 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.3 [4,9] 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.1d [4,8] 2024-05-08T05:29:01.387 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.21 [5,9] 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 3.2c [4,9] 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.2 [9,1] 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.9 [5,8] 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.12 [9,10] 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:pg_upmap_items 4.18 [9,8] 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/1451301612 expires 2024-05-09T05:18:03.888613+0000 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/3771264184 expires 2024-05-09T05:18:03.888613+0000 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6801/95297313 expires 2024-05-09T05:18:03.888613+0000 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2518610768 expires 2024-05-09T05:17:35.412203+0000 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/2478322707 expires 2024-05-09T05:17:35.412203+0000 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/95297313 expires 2024-05-09T05:18:03.888613+0000 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6800/817743540 expires 2024-05-09T05:17:35.412203+0000 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:6801/817743540 expires 2024-05-09T05:17:35.412203+0000 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/3865726924 expires 2024-05-09T05:17:35.412203+0000 2024-05-08T05:29:01.388 INFO:teuthology.orchestra.run.smithi059.stdout:blocklist 172.21.15.59:0/362628113 expires 2024-05-09T05:18:03.888613+0000 2024-05-08T05:29:01.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6f200b2a20 msgr2=0x7f6f200b4ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:01.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6f200b2a20 0x7f6f200b4ee0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6f28009550 tx=0x7f6f280091b0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 msgr2=0x7f6f38102cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:01.389 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 0x7f6f38102cc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6f300036f0 tx=0x7f6f300047b0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 shutdown_connections 2024-05-08T05:29:01.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6f38105170 0x7f6f38102cc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f6f200b2a20 0x7f6f200b4ee0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6f381017f0 0x7f6f38104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 --2- 172.21.15.59:0/1764966280 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6f381047d0 0x7f6f38102780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:01.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 >> 172.21.15.59:0/1764966280 conn(0x7f6f380fd150 msgr2=0x7f6f38107ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:01.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 shutdown_connections 2024-05-08T05:29:01.390 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:01.386+0000 7f6f3ce72700 1 -- 172.21.15.59:0/1764966280 wait complete. 2024-05-08T05:29:01.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:01 smithi059 ceph-mon[30304]: pgmap v408: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:01.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:01 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1764966280' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T05:29:01.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:01 smithi143 ceph-mon[34337]: pgmap v408: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:01.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:01 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1764966280' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T05:29:02.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:01 smithi096 ceph-mon[34253]: pgmap v408: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:02.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:01 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1764966280' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-05-08T05:29:02.088 INFO:teuthology.run_tasks:Running task fs.ready... 2024-05-08T05:29:02.100 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:29:02.354 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:03.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:03 smithi059 ceph-mon[30304]: pgmap v409: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:29:03.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:03 smithi143 ceph-mon[34337]: pgmap v409: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:29:04.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:03 smithi096 ceph-mon[34253]: pgmap v409: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:29:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.118+0000 7f98cd332700 1 -- 172.21.15.59:0/1702548163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 msgr2=0x7f98c8102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.118+0000 7f98cd332700 1 --2- 172.21.15.59:0/1702548163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 0x7f98c8102820 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009990 tx=0x7f98bc009ca0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.118+0000 7f98cd332700 1 -- 172.21.15.59:0/1702548163 shutdown_connections 2024-05-08T05:29:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.118+0000 7f98cd332700 1 --2- 172.21.15.59:0/1702548163 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c8102d60 0x7f98c810f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.118+0000 7f98cd332700 1 --2- 172.21.15.59:0/1702548163 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c8108410 0x7f98c81087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.118+0000 7f98cd332700 1 --2- 172.21.15.59:0/1702548163 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 0x7f98c8102820 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.118+0000 7f98cd332700 1 -- 172.21.15.59:0/1702548163 >> 172.21.15.59:0/1702548163 conn(0x7f98c8079350 msgr2=0x7f98c8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.118+0000 7f98cd332700 1 -- 172.21.15.59:0/1702548163 shutdown_connections 2024-05-08T05:29:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.118+0000 7f98cd332700 1 -- 172.21.15.59:0/1702548163 wait complete. 2024-05-08T05:29:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98cd332700 1 Processor -- start 2024-05-08T05:29:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98cd332700 1 -- start start 2024-05-08T05:29:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98cd332700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 0x7f98c819f930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98cd332700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c8102d60 0x7f98c819fe70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98cd332700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c8108410 0x7f98c8199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98cd332700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98c8113730 con 0x7f98c81023a0 2024-05-08T05:29:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98cd332700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81135b0 con 0x7f98c8102d60 2024-05-08T05:29:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98cd332700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f98c81138b0 con 0x7f98c8108410 2024-05-08T05:29:04.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98c6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 0x7f98c819f930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.119+0000 7f98c6ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 0x7f98c819f930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51802/0 (socket says 172.21.15.59:51802) 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c6ffd700 1 -- 172.21.15.59:0/3747286618 learned_addr learned my addr 172.21.15.59:0/3747286618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c67fc700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c8102d60 0x7f98c819fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c77fe700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c8108410 0x7f98c8199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c6ffd700 1 -- 172.21.15.59:0/3747286618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c8108410 msgr2=0x7f98c8199af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c6ffd700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c8108410 0x7f98c8199af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c6ffd700 1 -- 172.21.15.59:0/3747286618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c8102d60 msgr2=0x7f98c819fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c6ffd700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c8102d60 0x7f98c819fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c6ffd700 1 -- 172.21.15.59:0/3747286618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98c819a250 con 0x7f98c81023a0 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c77fe700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c8108410 0x7f98c8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98c6ffd700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 0x7f98c819f930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f98b8007f10 tx=0x7f98b800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98affff700 1 -- 172.21.15.59:0/3747286618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b800dc90 con 0x7f98c81023a0 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98c819a540 con 0x7f98c81023a0 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98affff700 1 -- 172.21.15.59:0/3747286618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98b800de30 con 0x7f98c81023a0 2024-05-08T05:29:04.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98c81a6800 con 0x7f98c81023a0 2024-05-08T05:29:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.120+0000 7f98affff700 1 -- 172.21.15.59:0/3747286618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b80133f0 con 0x7f98c81023a0 2024-05-08T05:29:04.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.121+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98c806a190 con 0x7f98c81023a0 2024-05-08T05:29:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.124+0000 7f98affff700 1 -- 172.21.15.59:0/3747286618 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f98b801a020 con 0x7f98c81023a0 2024-05-08T05:29:04.125 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.124+0000 7f98affff700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f98b00b2880 0x7f98b00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.124+0000 7f98c67fc700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f98b00b2880 0x7f98b00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.124+0000 7f98affff700 1 -- 172.21.15.59:0/3747286618 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f98b80e2540 con 0x7f98c81023a0 2024-05-08T05:29:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.125+0000 7f98c67fc700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f98b00b2880 0x7f98b00b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b5c0 tx=0x7f98bc01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.128+0000 7f98affff700 1 -- 172.21.15.59:0/3747286618 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98b8010070 con 0x7f98c81023a0 2024-05-08T05:29:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.392+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98c802e510 con 0x7f98c81023a0 2024-05-08T05:29:04.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.393+0000 7f98affff700 1 -- 172.21.15.59:0/3747286618 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7f98b80a9e70 con 0x7f98c81023a0 2024-05-08T05:29:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:29:04.395 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:26:31.316568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:29:04.396 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f98b00b2880 msgr2=0x7f98b00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f98b00b2880 0x7f98b00b4d40 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f98bc00b5c0 tx=0x7f98bc01a040 comp rx=0 tx=0).stop 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 msgr2=0x7f98c819f930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 0x7f98c819f930 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f98b8007f10 tx=0x7f98b800d3b0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 shutdown_connections 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98c8108410 0x7f98c8199af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98c81023a0 0x7f98c819f930 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f98b00b2880 0x7f98b00b4d40 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 --2- 172.21.15.59:0/3747286618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98c8102d60 0x7f98c819fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 >> 172.21.15.59:0/3747286618 conn(0x7f98c8079350 msgr2=0x7f98c8100fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 shutdown_connections 2024-05-08T05:29:04.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:04.395+0000 7f98cd332700 1 -- 172.21.15.59:0/3747286618 wait complete. 2024-05-08T05:29:04.398 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 29 2024-05-08T05:29:04.734 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:04 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3747286618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:04.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:04 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3747286618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:05.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:04 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3747286618' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:05.092 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:29:05.348 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:05.755 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:05 smithi059 ceph-mon[30304]: pgmap v410: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:05.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:05 smithi143 ceph-mon[34337]: pgmap v410: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:06.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:05 smithi096 ceph-mon[34253]: pgmap v410: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:06.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:29:06.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:06 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:29:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:06.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:06 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:29:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:06 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:07.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.139+0000 7fc715cf4700 1 -- 172.21.15.59:0/3389889824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 msgr2=0x7fc7101125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.139+0000 7fc715cf4700 1 --2- 172.21.15.59:0/3389889824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 0x7fc7101125b0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc70800b0a0 tx=0x7fc70800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.139+0000 7fc715cf4700 1 -- 172.21.15.59:0/3389889824 shutdown_connections 2024-05-08T05:29:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.139+0000 7fc715cf4700 1 --2- 172.21.15.59:0/3389889824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 0x7fc7101125b0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.139+0000 7fc715cf4700 1 --2- 172.21.15.59:0/3389889824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc71010b6e0 0x7fc71010bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.139+0000 7fc715cf4700 1 --2- 172.21.15.59:0/3389889824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc7101056e0 0x7fc710105b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.139+0000 7fc715cf4700 1 -- 172.21.15.59:0/3389889824 >> 172.21.15.59:0/3389889824 conn(0x7fc7101011d0 msgr2=0x7fc7101035f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.139+0000 7fc715cf4700 1 -- 172.21.15.59:0/3389889824 shutdown_connections 2024-05-08T05:29:07.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.139+0000 7fc715cf4700 1 -- 172.21.15.59:0/3389889824 wait complete. 2024-05-08T05:29:07.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc715cf4700 1 Processor -- start 2024-05-08T05:29:07.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc715cf4700 1 -- start start 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc715cf4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101056e0 0x7fc7101a1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc715cf4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 0x7fc7101a1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc715cf4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71010b6e0 0x7fc71019bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc715cf4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc710115930 con 0x7fc7101060a0 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc715cf4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc7101157b0 con 0x7fc7101056e0 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc715cf4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc710115ab0 con 0x7fc71010b6e0 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc6fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 0x7fc7101a1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc6fffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 0x7fc7101a1f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:51818/0 (socket says 172.21.15.59:51818) 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc6fffff700 1 -- 172.21.15.59:0/1285419424 learned_addr learned my addr 172.21.15.59:0/1285419424 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc70f7fe700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101056e0 0x7fc7101a1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.140+0000 7fc70ffff700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71010b6e0 0x7fc71019bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc6fffff700 1 -- 172.21.15.59:0/1285419424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71010b6e0 msgr2=0x7fc71019bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc6fffff700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71010b6e0 0x7fc71019bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc6fffff700 1 -- 172.21.15.59:0/1285419424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101056e0 msgr2=0x7fc7101a1a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:07.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc6fffff700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101056e0 0x7fc7101a1a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc6fffff700 1 -- 172.21.15.59:0/1285419424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc71019c3f0 con 0x7fc7101060a0 2024-05-08T05:29:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc6fffff700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 0x7fc7101a1f80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d980 tx=0x7fc70400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc70d7fa700 1 -- 172.21.15.59:0/1285419424 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc704017070 con 0x7fc7101060a0 2024-05-08T05:29:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc71019c6e0 con 0x7fc7101060a0 2024-05-08T05:29:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc70d7fa700 1 -- 172.21.15.59:0/1285419424 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc704005020 con 0x7fc7101060a0 2024-05-08T05:29:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.141+0000 7fc70d7fa700 1 -- 172.21.15.59:0/1285419424 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc70400fa10 con 0x7fc7101060a0 2024-05-08T05:29:07.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.142+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7101a89c0 con 0x7fc7101060a0 2024-05-08T05:29:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.143+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc710106e20 con 0x7fc7101060a0 2024-05-08T05:29:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.144+0000 7fc70d7fa700 1 -- 172.21.15.59:0/1285419424 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc704004a30 con 0x7fc7101060a0 2024-05-08T05:29:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.144+0000 7fc70d7fa700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc6f80b2880 0x7fc6f80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.145+0000 7fc70f7fe700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc6f80b2880 0x7fc6f80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:07.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.145+0000 7fc70d7fa700 1 -- 172.21.15.59:0/1285419424 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc7040e2460 con 0x7fc7101060a0 2024-05-08T05:29:07.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.145+0000 7fc70f7fe700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc6f80b2880 0x7fc6f80b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc70000abd0 tx=0x7fc70000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:07.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.149+0000 7fc70d7fa700 1 -- 172.21.15.59:0/1285419424 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc704020020 con 0x7fc7101060a0 2024-05-08T05:29:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.413+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc71019c9e0 con 0x7fc7101060a0 2024-05-08T05:29:07.414 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.414+0000 7fc70d7fa700 1 -- 172.21.15.59:0/1285419424 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11929 (secure 0 0 0) 0x7fc7040b0e20 con 0x7fc7101060a0 2024-05-08T05:29:07.415 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:29:07.415 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:26:31.316568+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:29:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc6f80b2880 msgr2=0x7fc6f80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc6f80b2880 0x7fc6f80b4d40 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fc70000abd0 tx=0x7fc70000a380 comp rx=0 tx=0).stop 2024-05-08T05:29:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 msgr2=0x7fc7101a1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 0x7fc7101a1f80 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc70400d980 tx=0x7fc70400dc90 comp rx=0 tx=0).stop 2024-05-08T05:29:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 shutdown_connections 2024-05-08T05:29:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc71010b6e0 0x7fc71019bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.417 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc6f80b2880 0x7fc6f80b4d40 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc7101056e0 0x7fc7101a1a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 --2- 172.21.15.59:0/1285419424 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc7101060a0 0x7fc7101a1f80 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 >> 172.21.15.59:0/1285419424 conn(0x7fc7101011d0 msgr2=0x7fc7101016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.416+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 shutdown_connections 2024-05-08T05:29:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:07.417+0000 7fc715cf4700 1 -- 172.21.15.59:0/1285419424 wait complete. 2024-05-08T05:29:07.418 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 29 2024-05-08T05:29:07.781 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:07 smithi059 ceph-mon[30304]: pgmap v411: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:29:07.781 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:07 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:29:07.781 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:07 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:29:07.781 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:07 smithi059 ceph-mon[30304]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:07.781 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:07 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1285419424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:07 smithi143 ceph-mon[34337]: pgmap v411: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:29:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:07 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:29:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:07 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:29:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:07 smithi143 ceph-mon[34337]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:07.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:07 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1285419424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:08.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:07 smithi096 ceph-mon[34253]: pgmap v411: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:29:08.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:07 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:29:08.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:07 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:29:08.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:07 smithi096 ceph-mon[34253]: from='mgr.14152 172.21.15.59:0/745353230' entity='mgr.x' 2024-05-08T05:29:08.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:07 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1285419424' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:08.049 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:29:08.305 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:09.822 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:09 smithi059 ceph-mon[30304]: pgmap v412: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:29:09.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:09 smithi143 ceph-mon[34337]: pgmap v412: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:29:10.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:09 smithi096 ceph-mon[34253]: pgmap v412: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:29:10.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.062+0000 7f36c0b9e700 1 -- 172.21.15.59:0/4224963549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc1033d0 msgr2=0x7f36bc103850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:10.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.062+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/4224963549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc1033d0 0x7f36bc103850 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f36b0009990 tx=0x7f36b0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.062+0000 7f36c0b9e700 1 -- 172.21.15.59:0/4224963549 shutdown_connections 2024-05-08T05:29:10.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.062+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/4224963549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36bc103d90 0x7f36bc1102a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.062+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/4224963549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36bc109460 0x7f36bc109840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.062+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/4224963549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc1033d0 0x7f36bc103850 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.062+0000 7f36c0b9e700 1 -- 172.21.15.59:0/4224963549 >> 172.21.15.59:0/4224963549 conn(0x7f36bc0fef20 msgr2=0x7f36bc101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:10.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.062+0000 7f36c0b9e700 1 -- 172.21.15.59:0/4224963549 shutdown_connections 2024-05-08T05:29:10.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.062+0000 7f36c0b9e700 1 -- 172.21.15.59:0/4224963549 wait complete. 2024-05-08T05:29:10.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.063+0000 7f36c0b9e700 1 Processor -- start 2024-05-08T05:29:10.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.063+0000 7f36c0b9e700 1 -- start start 2024-05-08T05:29:10.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.063+0000 7f36c0b9e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36bc1033d0 0x7f36bc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:10.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.063+0000 7f36c0b9e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc103d90 0x7f36bc075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:10.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.063+0000 7f36c0b9e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36bc109460 0x7f36bc076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36c0b9e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc1135a0 con 0x7f36bc103d90 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36c0b9e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc113420 con 0x7f36bc109460 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36c0b9e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f36bc113720 con 0x7f36bc1033d0 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36baffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc103d90 0x7f36bc075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36baffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc103d90 0x7f36bc075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47660/0 (socket says 172.21.15.59:47660) 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36baffd700 1 -- 172.21.15.59:0/1416423260 learned_addr learned my addr 172.21.15.59:0/1416423260 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36bb7fe700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36bc1033d0 0x7f36bc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36bbfff700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36bc109460 0x7f36bc076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36baffd700 1 -- 172.21.15.59:0/1416423260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36bc1033d0 msgr2=0x7f36bc0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36baffd700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36bc1033d0 0x7f36bc0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36baffd700 1 -- 172.21.15.59:0/1416423260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36bc109460 msgr2=0x7f36bc076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36baffd700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36bc109460 0x7f36bc076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36baffd700 1 -- 172.21.15.59:0/1416423260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36bc1a7fb0 con 0x7f36bc103d90 2024-05-08T05:29:10.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36baffd700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc103d90 0x7f36bc075ce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f36b00057c0 tx=0x7f36b00057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:10.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36b8ff9700 1 -- 172.21.15.59:0/1416423260 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0027070 con 0x7f36bc103d90 2024-05-08T05:29:10.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36b8ff9700 1 -- 172.21.15.59:0/1416423260 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36b0003f20 con 0x7f36bc103d90 2024-05-08T05:29:10.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.064+0000 7f36b8ff9700 1 -- 172.21.15.59:0/1416423260 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b0022b50 con 0x7f36bc103d90 2024-05-08T05:29:10.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.065+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36bc1a8240 con 0x7f36bc103d90 2024-05-08T05:29:10.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.065+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36bc1a86d0 con 0x7f36bc103d90 2024-05-08T05:29:10.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.065+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36bc06a190 con 0x7f36bc103d90 2024-05-08T05:29:10.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.067+0000 7f36b8ff9700 1 -- 172.21.15.59:0/1416423260 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f36b002a070 con 0x7f36bc103d90 2024-05-08T05:29:10.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.068+0000 7f36b8ff9700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f36a40b2880 0x7f36a40b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:10.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.068+0000 7f36b8ff9700 1 -- 172.21.15.59:0/1416423260 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f36b00e3cb0 con 0x7f36bc103d90 2024-05-08T05:29:10.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.068+0000 7f36bb7fe700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f36a40b2880 0x7f36a40b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:10.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.068+0000 7f36bb7fe700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f36a40b2880 0x7f36a40b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f36ac005950 tx=0x7f36ac00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:10.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.072+0000 7f36b8ff9700 1 -- 172.21.15.59:0/1416423260 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f36b0031020 con 0x7f36bc103d90 2024-05-08T05:29:10.287 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.287+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36bc02e510 con 0x7f36bc103d90 2024-05-08T05:29:10.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.289+0000 7f36b8ff9700 1 -- 172.21.15.59:0/1416423260 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v90) ==== 74+0+24695 (secure 0 0 0) 0x7f36b0013070 con 0x7f36bc103d90 2024-05-08T05:29:10.289 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:29:10.290 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":90,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:27:05.550089+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f36a40b2880 msgr2=0x7f36a40b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f36a40b2880 0x7f36a40b4d40 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f36ac005950 tx=0x7f36ac00a300 comp rx=0 tx=0).stop 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc103d90 msgr2=0x7f36bc075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc103d90 0x7f36bc075ce0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f36b00057c0 tx=0x7f36b00057f0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 shutdown_connections 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f36a40b2880 0x7f36a40b4d40 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f36bc109460 0x7f36bc076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f36bc1033d0 0x7f36bc0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 --2- 172.21.15.59:0/1416423260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f36bc103d90 0x7f36bc075ce0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 >> 172.21.15.59:0/1416423260 conn(0x7f36bc0fef20 msgr2=0x7f36bc0ff520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 shutdown_connections 2024-05-08T05:29:10.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:10.291+0000 7f36c0b9e700 1 -- 172.21.15.59:0/1416423260 wait complete. 2024-05-08T05:29:10.955 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:10 smithi059 ceph-mon[30304]: pgmap v413: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:10.955 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:10 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1416423260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:29:10.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T05:25:27.601131+0000', 'modified': '2024-05-08T05:26:31.316568+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24473, 'mds_1': 24443, 'mds_2': 24433, 'mds_3': 24421, 'mds_4': 24419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14565': {'gid': 14565, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6839/2756426260', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 2756426260}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 2756426260}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14571': {'gid': 14571, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6841/2496793439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6840', 'nonce': 2496793439}, {'type': 'v1', 'addr': '172.21.15.59:6841', 'nonce': 2496793439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6833/1879950931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6832', 'nonce': 1879950931}, {'type': 'v1', 'addr': '172.21.15.143:6833', 'nonce': 1879950931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6833/2593895617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 2593895617}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 2593895617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6835/2504523054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 2504523054}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 2504523054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.96:6835/2294126295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 2294126295}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 2294126295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.143:6837/555225813', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 555225813}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 555225813}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.143:6839/2166229809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 2166229809}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 2166229809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.96:6837/780978870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 780978870}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 780978870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6839/126053836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 126053836}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 126053836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24473, 'qdb_cluster': [24419, 24433, 24421, 24443, 24473]} 2024-05-08T05:29:10.987 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T05:29:10.987 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.e status 2024-05-08T05:29:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:10 smithi143 ceph-mon[34337]: pgmap v413: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:10 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1416423260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:29:11.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:10 smithi096 ceph-mon[34253]: pgmap v413: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:11.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:10 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1416423260' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:29:11.244 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:12.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.933+0000 7f9d4e098700 1 -- 172.21.15.59:0/3690501508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 msgr2=0x7f9d481098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.933+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3690501508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 0x7f9d481098f0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009990 tx=0x7f9d30009ca0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.933+0000 7f9d4e098700 1 -- 172.21.15.59:0/3690501508 shutdown_connections 2024-05-08T05:29:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.933+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3690501508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d4806c5f0 0x7f9d48110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.933+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3690501508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 0x7f9d481098f0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.933+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3690501508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4806bc30 0x7f9d4806c0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.933+0000 7f9d4e098700 1 -- 172.21.15.59:0/3690501508 >> 172.21.15.59:0/3690501508 conn(0x7f9d48079350 msgr2=0x7f9d48079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.933+0000 7f9d4e098700 1 -- 172.21.15.59:0/3690501508 shutdown_connections 2024-05-08T05:29:12.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.933+0000 7f9d4e098700 1 -- 172.21.15.59:0/3690501508 wait complete. 2024-05-08T05:29:12.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d4e098700 1 Processor -- start 2024-05-08T05:29:12.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d4e098700 1 -- start start 2024-05-08T05:29:12.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d4e098700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4806bc30 0x7f9d4819f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d4e098700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d4806c5f0 0x7f9d4819fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d4e098700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 0x7f9d481999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d4e098700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9d481136a0 con 0x7f9d48109510 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d4e098700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48113520 con 0x7f9d4806bc30 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d4e098700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9d48113820 con 0x7f9d4806c5f0 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 0x7f9d481999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 0x7f9d481999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47682/0 (socket says 172.21.15.59:47682) 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d47fff700 1 -- 172.21.15.59:0/3627953897 learned_addr learned my addr 172.21.15.59:0/3627953897 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d477fe700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4806bc30 0x7f9d4819f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.934+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d4806c5f0 0x7f9d4819fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d47fff700 1 -- 172.21.15.59:0/3627953897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d4806c5f0 msgr2=0x7f9d4819fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d47fff700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d4806c5f0 0x7f9d4819fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d47fff700 1 -- 172.21.15.59:0/3627953897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4806bc30 msgr2=0x7f9d4819f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d47fff700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4806bc30 0x7f9d4819f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d47fff700 1 -- 172.21.15.59:0/3627953897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9d4819a140 con 0x7f9d48109510 2024-05-08T05:29:12.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d477fe700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4806bc30 0x7f9d4819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d47fff700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 0x7f9d481999e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000b970 tx=0x7f9d4000bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d44ff9700 1 -- 172.21.15.59:0/3627953897 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d4000d860 con 0x7f9d48109510 2024-05-08T05:29:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d44ff9700 1 -- 172.21.15.59:0/3627953897 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d4000e430 con 0x7f9d48109510 2024-05-08T05:29:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d4e098700 1 -- 172.21.15.59:0/3627953897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9d4819a430 con 0x7f9d48109510 2024-05-08T05:29:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d44ff9700 1 -- 172.21.15.59:0/3627953897 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d40017660 con 0x7f9d48109510 2024-05-08T05:29:12.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.935+0000 7f9d4e098700 1 -- 172.21.15.59:0/3627953897 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9d481a66f0 con 0x7f9d48109510 2024-05-08T05:29:12.948 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.937+0000 7f9d44ff9700 1 -- 172.21.15.59:0/3627953897 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f9d4000da00 con 0x7f9d48109510 2024-05-08T05:29:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.938+0000 7f9d44ff9700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9d340b2880 0x7f9d340b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.938+0000 7f9d477fe700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9d340b2880 0x7f9d340b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.938+0000 7f9d44ff9700 1 -- 172.21.15.59:0/3627953897 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9d400e2180 con 0x7f9d48109510 2024-05-08T05:29:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d5892bb80 1 Processor -- start 2024-05-08T05:29:12.949 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d5892bb80 1 -- start start 2024-05-08T05:29:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d477fe700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9d340b2880 0x7f9d340b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b5c0 tx=0x7f9d30005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435cdf30 0x559d435ce310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 0x559d435c90c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559d43516810 con 0x559d435ce850 2024-05-08T05:29:12.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559d434b5770 con 0x559d435c8c40 2024-05-08T05:29:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559d435127d0 con 0x559d435cdf30 2024-05-08T05:29:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d46ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d46ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435c86b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47688/0 (socket says 172.21.15.59:47688) 2024-05-08T05:29:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d46ffd700 1 -- 172.21.15.59:0/2192185809 learned_addr learned my addr 172.21.15.59:0/2192185809 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:12.951 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d477fe700 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435cdf30 0x559d435ce310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d47fff700 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 0x559d435c90c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d46ffd700 1 -- 172.21.15.59:0/2192185809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435cdf30 msgr2=0x559d435ce310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:12.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435cdf30 0x559d435ce310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d46ffd700 1 -- 172.21.15.59:0/2192185809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 msgr2=0x559d435c90c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:12.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 0x559d435c90c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d46ffd700 1 -- 172.21.15.59:0/2192185809 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d43463880 con 0x559d435ce850 2024-05-08T05:29:12.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.939+0000 7f9d47fff700 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 0x559d435c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:12.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.940+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435c86b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800bd30 tx=0x7f9d380106b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73fbec5f71243d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:12.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.940+0000 7f9d3dffb700 1 -- 172.21.15.59:0/2192185809 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38010e10 con 0x559d435ce850 2024-05-08T05:29:12.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.940+0000 7f9d3dffb700 1 -- 172.21.15.59:0/2192185809 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d380044e0 con 0x559d435ce850 2024-05-08T05:29:12.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.940+0000 7f9d3dffb700 1 -- 172.21.15.59:0/2192185809 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38011440 con 0x559d435ce850 2024-05-08T05:29:12.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.940+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2192185809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 msgr2=0x559d435c86b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.940+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435c86b0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800bd30 tx=0x7f9d380106b0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2192185809 shutdown_connections 2024-05-08T05:29:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 0x559d435c90c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435cdf30 0x559d435ce310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2192185809 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435c86b0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2192185809 >> 172.21.15.59:0/2192185809 conn(0x559d434f00b0 msgr2=0x559d435bec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2192185809 shutdown_connections 2024-05-08T05:29:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2192185809 wait complete. 2024-05-08T05:29:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 -- start start 2024-05-08T05:29:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 Processor -- start 2024-05-08T05:29:12.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435c8c40 0x559d435d44a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.941+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435d1840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435d1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559d435d5270 con 0x559d435ce850 2024-05-08T05:29:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559d4347cbe0 con 0x559d435cdf30 2024-05-08T05:29:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559d435d53f0 con 0x559d435c8c40 2024-05-08T05:29:12.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d5892bb80 10 client.0 _ll_register_callbacks cb 0x559d434ae8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:12.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435d1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d47fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435d1d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47692/0 (socket says 172.21.15.59:47692) 2024-05-08T05:29:12.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d47fff700 1 -- 172.21.15.59:0/2644813626 learned_addr learned my addr 172.21.15.59:0/2644813626 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:12.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435d1840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d477fe700 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435c8c40 0x559d435d44a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d47fff700 1 -- 172.21.15.59:0/2644813626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435c8c40 msgr2=0x559d435d44a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d47fff700 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435c8c40 0x559d435d44a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d47fff700 1 -- 172.21.15.59:0/2644813626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 msgr2=0x559d435d1840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d47fff700 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435d1840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d47fff700 1 -- 172.21.15.59:0/2644813626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d434b2640 con 0x559d435ce850 2024-05-08T05:29:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d477fe700 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435c8c40 0x559d435d44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:29:12.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:12.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d47fff700 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435d1d80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9d400a5490 tx=0x7f9d400a9b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cebbfd0674d4dbed server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:12.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:12.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d26ffd700 1 -- 172.21.15.59:0/2644813626 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d400c3060 con 0x559d435ce850 2024-05-08T05:29:12.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d26ffd700 1 -- 172.21.15.59:0/2644813626 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9d4000dcf0 con 0x559d435ce850 2024-05-08T05:29:12.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2644813626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d380044e0 con 0x559d435ce850 2024-05-08T05:29:12.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d26ffd700 1 -- 172.21.15.59:0/2644813626 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d400b2270 con 0x559d435ce850 2024-05-08T05:29:12.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d5892bb80 10 client.15177 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:12.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.942+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2644813626 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d435dfb80 con 0x559d435ce850 2024-05-08T05:29:12.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.943+0000 7f9d26ffd700 1 -- 172.21.15.59:0/2644813626 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d380044e0 con 0x559d435ce850 2024-05-08T05:29:12.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.943+0000 7f9d26ffd700 1 -- 172.21.15.59:0/2644813626 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9d4001f030 con 0x559d435ce850 2024-05-08T05:29:12.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.943+0000 7f9d5892bb80 10 client.15177 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.943+0000 7f9d5892bb80 10 client.15177 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T05:29:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.943+0000 7f9d5892bb80 20 client.15177 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.943+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x559d435e20f0 0x559d435e3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.943+0000 7f9d5892bb80 4 client.15177 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:29:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.943+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2644813626 --> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559d43590600 con 0x559d435e20f0 2024-05-08T05:29:12.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.943+0000 7f9d477fe700 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x559d435e20f0 0x559d435e3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.944+0000 7f9d477fe700 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x559d435e20f0 0x559d435e3560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=17f3683f7ab4fb13 server_cookie=7ffeecd572e381c0 in_seq=0 out_seq=0 2024-05-08T05:29:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.944+0000 7f9d26ffd700 10 client.15177 ms_handle_connect on v2:172.21.15.96:6834/2294126295 2024-05-08T05:29:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.944+0000 7f9d26ffd700 1 -- 172.21.15.59:0/2644813626 <== mds.4 v2:172.21.15.96:6834/2294126295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559d43590600 con 0x559d435e20f0 2024-05-08T05:29:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.944+0000 7f9d26ffd700 10 client.15177 handle_command_reply: tid=0 2024-05-08T05:29:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.944+0000 7f9d5892bb80 1 client.15177 shutdown 2024-05-08T05:29:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2644813626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 msgr2=0x559d435d1d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:12.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435d1d80 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f9d400a5490 tx=0x7f9d400a9b10 comp rx=0 tx=0).stop 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2644813626 shutdown_connections 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x559d435e20f0 0x559d435e3560 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435ce850 0x559d435d1d80 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435c8c40 0x559d435d44a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d26ffd700 0 client.15177 ms_handle_reset on v2:172.21.15.96:6834/2294126295 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/2644813626 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435d1840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2644813626 >> 172.21.15.59:0/2644813626 conn(0x559d434f00b0 msgr2=0x559d435b96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2644813626 shutdown_connections 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 1 -- 172.21.15.59:0/2644813626 wait complete. 2024-05-08T05:29:12.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.945+0000 7f9d5892bb80 20 client.15177 trim_cache size 0 max 16384 2024-05-08T05:29:12.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.966+0000 7f9d5892bb80 1 Processor -- start 2024-05-08T05:29:12.973 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.966+0000 7f9d5892bb80 1 -- start start 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 0x559d435d1800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435cdf30 0x559d435d1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 0x559d435cb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559d435e18a0 con 0x559d435cdf30 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559d435e1f70 con 0x559d435c8c40 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559d435e2640 con 0x559d435ce850 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d46ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435cdf30 0x559d435d1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d46ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435cdf30 0x559d435d1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47704/0 (socket says 172.21.15.59:47704) 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d46ffd700 1 -- 172.21.15.59:0/3190972104 learned_addr learned my addr 172.21.15.59:0/3190972104 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d477fe700 1 --2- 172.21.15.59:0/3190972104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 0x559d435d1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d47fff700 1 --2- 172.21.15.59:0/3190972104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 0x559d435cb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d46ffd700 1 -- 172.21.15.59:0/3190972104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 msgr2=0x559d435cb8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/3190972104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 0x559d435cb8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d46ffd700 1 -- 172.21.15.59:0/3190972104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 msgr2=0x559d435d1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/3190972104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 0x559d435d1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d46ffd700 1 -- 172.21.15.59:0/3190972104 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559d435bb7c0 con 0x559d435cdf30 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.967+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/3190972104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435cdf30 0x559d435d1d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800bd30 tx=0x7f9d3800bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4fc918b2f7f8dba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d3d7fa700 1 -- 172.21.15.59:0/3190972104 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38010c10 con 0x559d435cdf30 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d3d7fa700 1 -- 172.21.15.59:0/3190972104 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d38010db0 con 0x559d435cdf30 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d3d7fa700 1 -- 172.21.15.59:0/3190972104 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d38004d60 con 0x559d435cdf30 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d5892bb80 1 -- 172.21.15.59:0/3190972104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435cdf30 msgr2=0x559d435d1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/3190972104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435cdf30 0x559d435d1d40 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f9d3800bd30 tx=0x7f9d3800bc30 comp rx=0 tx=0).stop 2024-05-08T05:29:12.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d5892bb80 1 -- 172.21.15.59:0/3190972104 shutdown_connections 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/3190972104 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 0x559d435cb8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/3190972104 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435c8c40 0x559d435d1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/3190972104 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435cdf30 0x559d435d1d40 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.968+0000 7f9d5892bb80 1 -- 172.21.15.59:0/3190972104 >> 172.21.15.59:0/3190972104 conn(0x559d434f00b0 msgr2=0x559d435d42c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 -- 172.21.15.59:0/3190972104 shutdown_connections 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 -- 172.21.15.59:0/3190972104 wait complete. 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 -- start start 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 Processor -- start 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435c8c40 0x559d435de500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435ec9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 0x559d435e7990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559d435e33e0 con 0x559d435c8c40 2024-05-08T05:29:12.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559d435e2d10 con 0x559d435cdf30 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559d435e3ab0 con 0x559d435ce850 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d5892bb80 10 client.0 _ll_register_callbacks cb 0x559d434ae8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.969+0000 7f9d477fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435c8c40 0x559d435de500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d477fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435c8c40 0x559d435de500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47714/0 (socket says 172.21.15.59:47714) 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d477fe700 1 -- 172.21.15.59:0/482434664 learned_addr learned my addr 172.21.15.59:0/482434664 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d47fff700 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 0x559d435e7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435ec9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d477fe700 1 -- 172.21.15.59:0/482434664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 msgr2=0x559d435e7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d477fe700 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 0x559d435e7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d477fe700 1 -- 172.21.15.59:0/482434664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 msgr2=0x559d435ec9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d477fe700 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435ec9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d477fe700 1 -- 172.21.15.59:0/482434664 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559d435ce560 con 0x559d435c8c40 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435ec9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d477fe700 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435c8c40 0x559d435de500 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009990 tx=0x7f9d3001a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9dd13b55b005ece server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:12.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d26ffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d26ffd700 1 -- 172.21.15.59:0/482434664 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9d300054c0 con 0x559d435c8c40 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d26ffd700 1 -- 172.21.15.59:0/482434664 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f9d30005660 con 0x559d435c8c40 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d5892bb80 1 -- 172.21.15.59:0/482434664 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9d38010db0 con 0x559d435c8c40 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.970+0000 7f9d26ffd700 1 -- 172.21.15.59:0/482434664 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9d30041050 con 0x559d435c8c40 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d5892bb80 10 client.15183 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d5892bb80 1 -- 172.21.15.59:0/482434664 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559d4364d970 con 0x559d435c8c40 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d26ffd700 1 -- 172.21.15.59:0/482434664 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9d38010db0 con 0x559d435c8c40 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d26ffd700 1 -- 172.21.15.59:0/482434664 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9d3000f4f0 con 0x559d435c8c40 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d5892bb80 10 client.15183 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d5892bb80 10 client.15183 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d5892bb80 20 client.15183 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x559d43652600 0x559d43651310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d5892bb80 4 client.15183 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:29:12.978 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d5892bb80 1 -- 172.21.15.59:0/482434664 --> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] -- command(tid 0: {"prefix": "status"}) -- 0x559d435241f0 con 0x559d43652600 2024-05-08T05:29:12.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.971+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x559d43652600 0x559d43651310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:12.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d26ffd700 10 client.15183 ms_handle_connect on v2:172.21.15.96:6834/2294126295 2024-05-08T05:29:12.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d46ffd700 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x559d43652600 0x559d43651310 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ceac3b4fe026799c server_cookie=7556c0054f5812b2 in_seq=0 out_seq=0 2024-05-08T05:29:12.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d26ffd700 1 -- 172.21.15.59:0/482434664 <== mds.4 v2:172.21.15.96:6834/2294126295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x559d435241f0 con 0x559d43652600 2024-05-08T05:29:12.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d26ffd700 10 client.15183 handle_command_reply: tid=0 2024-05-08T05:29:12.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 client.15183 shutdown 2024-05-08T05:29:12.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 -- 172.21.15.59:0/482434664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435c8c40 msgr2=0x559d435de500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:12.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435c8c40 0x559d435de500 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9d30009990 tx=0x7f9d3001a050 comp rx=0 tx=0).stop 2024-05-08T05:29:12.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 -- 172.21.15.59:0/482434664 shutdown_connections 2024-05-08T05:29:12.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x559d43652600 0x559d43651310 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.981 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d26ffd700 0 client.15183 ms_handle_reset on v2:172.21.15.96:6834/2294126295 2024-05-08T05:29:12.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559d435ce850 0x559d435e7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559d435c8c40 0x559d435de500 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 --2- 172.21.15.59:0/482434664 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559d435cdf30 0x559d435ec9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 -- 172.21.15.59:0/482434664 >> 172.21.15.59:0/482434664 conn(0x559d434f00b0 msgr2=0x559d435d1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:12.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 -- 172.21.15.59:0/482434664 shutdown_connections 2024-05-08T05:29:12.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 1 -- 172.21.15.59:0/482434664 wait complete. 2024-05-08T05:29:12.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.972+0000 7f9d5892bb80 20 client.15183 trim_cache size 0 max 16384 2024-05-08T05:29:12.982 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:29:12.982 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 4, 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24419, 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 165.564008546, 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 262.66861249900001 2024-05-08T05:29:12.983 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:29:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 -- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9d340b2880 msgr2=0x7f9d340b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9d340b2880 0x7f9d340b4d40 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9d3000b5c0 tx=0x7f9d30005960 comp rx=0 tx=0).stop 2024-05-08T05:29:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 -- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 msgr2=0x7f9d481999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 0x7f9d481999e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9d4000b970 tx=0x7f9d4000bc80 comp rx=0 tx=0).stop 2024-05-08T05:29:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 -- 172.21.15.59:0/3627953897 shutdown_connections 2024-05-08T05:29:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9d48109510 0x7f9d481999e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f9d340b2880 0x7f9d340b4d40 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.984 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9d4806bc30 0x7f9d4819f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 --2- 172.21.15.59:0/3627953897 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9d4806c5f0 0x7f9d4819fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 -- 172.21.15.59:0/3627953897 >> 172.21.15.59:0/3627953897 conn(0x7f9d48079350 msgr2=0x7f9d48101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 -- 172.21.15.59:0/3627953897 shutdown_connections 2024-05-08T05:29:12.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:12.973+0000 7f9d4e098700 1 -- 172.21.15.59:0/3627953897 wait complete. 2024-05-08T05:29:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:13 smithi059 ceph-mon[30304]: pgmap v414: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:29:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:13 smithi143 ceph-mon[34337]: pgmap v414: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:29:13.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:13 smithi096 ceph-mon[34253]: pgmap v414: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:29:13.628 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-08T05:29:13.884 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:15.360 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:15 smithi059 ceph-mon[30304]: pgmap v415: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:15 smithi143 ceph-mon[34337]: pgmap v415: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:15 smithi096 ceph-mon[34253]: pgmap v415: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:15.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.763+0000 7f95f6d32700 1 -- 172.21.15.59:0/578271345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0101a90 msgr2=0x7f95f0101f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:15.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.763+0000 7f95f6d32700 1 --2- 172.21.15.59:0/578271345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0101a90 0x7f95f0101f10 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f95dc009990 tx=0x7f95dc009ca0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.763+0000 7f95f6d32700 1 -- 172.21.15.59:0/578271345 shutdown_connections 2024-05-08T05:29:15.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.763+0000 7f95f6d32700 1 --2- 172.21.15.59:0/578271345 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95f0102450 0x7f95f0110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.763+0000 7f95f6d32700 1 --2- 172.21.15.59:0/578271345 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f0109520 0x7f95f0109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.763+0000 7f95f6d32700 1 --2- 172.21.15.59:0/578271345 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0101a90 0x7f95f0101f10 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.763+0000 7f95f6d32700 1 -- 172.21.15.59:0/578271345 >> 172.21.15.59:0/578271345 conn(0x7f95f0078a10 msgr2=0x7f95f00ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:15.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.763+0000 7f95f6d32700 1 -- 172.21.15.59:0/578271345 shutdown_connections 2024-05-08T05:29:15.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.763+0000 7f95f6d32700 1 -- 172.21.15.59:0/578271345 wait complete. 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95f6d32700 1 Processor -- start 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95f6d32700 1 -- start start 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95f6d32700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95f0101a90 0x7f95f019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95f6d32700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0102450 0x7f95f019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95f6d32700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f0109520 0x7f95f01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95f6d32700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0113680 con 0x7f95f0102450 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95f6d32700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0113500 con 0x7f95f0109520 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95f6d32700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95f0113800 con 0x7f95f0101a90 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0102450 0x7f95f019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0102450 0x7f95f019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47734/0 (socket says 172.21.15.59:47734) 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95effff700 1 -- 172.21.15.59:0/4243146206 learned_addr learned my addr 172.21.15.59:0/4243146206 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.764+0000 7f95f4ace700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95f0101a90 0x7f95f019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95f52cf700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f0109520 0x7f95f01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95effff700 1 -- 172.21.15.59:0/4243146206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95f0101a90 msgr2=0x7f95f019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95effff700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95f0101a90 0x7f95f019f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95effff700 1 -- 172.21.15.59:0/4243146206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f0109520 msgr2=0x7f95f01998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:15.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95effff700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f0109520 0x7f95f01998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95effff700 1 -- 172.21.15.59:0/4243146206 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95f019a040 con 0x7f95f0102450 2024-05-08T05:29:15.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95effff700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0102450 0x7f95f019fc60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95dc000c00 tx=0x7f95dc0057c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:15.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95edffb700 1 -- 172.21.15.59:0/4243146206 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc026070 con 0x7f95f0102450 2024-05-08T05:29:15.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95edffb700 1 -- 172.21.15.59:0/4243146206 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95dc003f20 con 0x7f95f0102450 2024-05-08T05:29:15.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95f6d32700 1 -- 172.21.15.59:0/4243146206 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95f019a2d0 con 0x7f95f0102450 2024-05-08T05:29:15.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95edffb700 1 -- 172.21.15.59:0/4243146206 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc00fc30 con 0x7f95f0102450 2024-05-08T05:29:15.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.765+0000 7f95f6d32700 1 -- 172.21.15.59:0/4243146206 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95f01a65f0 con 0x7f95f0102450 2024-05-08T05:29:15.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.768+0000 7f95edffb700 1 -- 172.21.15.59:0/4243146206 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f95dc029070 con 0x7f95f0102450 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.768+0000 7f95edffb700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f95e00b2880 0x7f95e00b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f95edffb700 1 -- 172.21.15.59:0/4243146206 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f95dc0e2f10 con 0x7f95f0102450 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f95f4ace700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f95e00b2880 0x7f95e00b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f96015c5b80 1 Processor -- start 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f96015c5b80 1 -- start start 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f95f4ace700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f95e00b2880 0x7f95e00b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95f019aa20 tx=0x7f95e400a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x5612246006a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612246010b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f96015c5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56122454e810 con 0x561224600c30 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f96015c5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612244ed770 con 0x5612245fc480 2024-05-08T05:29:15.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f96015c5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56122454a7d0 con 0x5612245fcda0 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.769+0000 7f95f52cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612246010b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f52cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612246010b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47738/0 (socket says 172.21.15.59:47738) 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f52cf700 1 -- 172.21.15.59:0/2877415080 learned_addr learned my addr 172.21.15.59:0/2877415080 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f4ace700 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95effff700 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x5612246006a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f52cf700 1 -- 172.21.15.59:0/2877415080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 msgr2=0x5612246006a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f52cf700 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x5612246006a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f52cf700 1 -- 172.21.15.59:0/2877415080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 msgr2=0x5612245fc860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f52cf700 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fc860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f52cf700 1 -- 172.21.15.59:0/2877415080 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56122449b880 con 0x561224600c30 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f4ace700 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:15.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95f52cf700 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612246010b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f95e800f5a0 tx=0x7f95e800f8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4767460a7cec1e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:15.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95daffd700 1 -- 172.21.15.59:0/2877415080 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8018030 con 0x561224600c30 2024-05-08T05:29:15.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95daffd700 1 -- 172.21.15.59:0/2877415080 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e8004bb0 con 0x561224600c30 2024-05-08T05:29:15.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.770+0000 7f95daffd700 1 -- 172.21.15.59:0/2877415080 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8005020 con 0x561224600c30 2024-05-08T05:29:15.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 -- 172.21.15.59:0/2877415080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 msgr2=0x5612246010b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:15.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612246010b0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f95e800f5a0 tx=0x7f95e800f8b0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 -- 172.21.15.59:0/2877415080 shutdown_connections 2024-05-08T05:29:15.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612246010b0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fc860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 --2- 172.21.15.59:0/2877415080 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x5612246006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 -- 172.21.15.59:0/2877415080 >> 172.21.15.59:0/2877415080 conn(0x5612245280b0 msgr2=0x5612245f6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:15.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 -- 172.21.15.59:0/2877415080 shutdown_connections 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 -- 172.21.15.59:0/2877415080 wait complete. 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 -- start start 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.771+0000 7f96015c5b80 1 Processor -- start 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x561224608000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612245fcda0 0x5612246053a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561224600c30 0x5612246058e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f96015c5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561224608e40 con 0x5612245fcda0 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f96015c5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612244b4be0 con 0x5612245fc480 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f96015c5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561224608fc0 con 0x561224600c30 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f96015c5b80 10 client.0 _ll_register_callbacks cb 0x5612244e68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612245fcda0 0x5612246053a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95effff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612245fcda0 0x5612246053a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47748/0 (socket says 172.21.15.59:47748) 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95effff700 1 -- 172.21.15.59:0/741808172 learned_addr learned my addr 172.21.15.59:0/741808172 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95f4ace700 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x561224608000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95f52cf700 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561224600c30 0x5612246058e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95effff700 1 -- 172.21.15.59:0/741808172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561224600c30 msgr2=0x5612246058e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95effff700 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561224600c30 0x5612246058e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95effff700 1 -- 172.21.15.59:0/741808172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 msgr2=0x561224608000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95effff700 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x561224608000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.772+0000 7f95effff700 1 -- 172.21.15.59:0/741808172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612244ea640 con 0x5612245fcda0 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f95effff700 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612245fcda0 0x5612246053a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0e09f0 tx=0x7f95dc009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a31b7946b502d660 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f95cffff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f95cffff700 1 -- 172.21.15.59:0/741808172 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95dc0aa930 con 0x5612245fcda0 2024-05-08T05:29:15.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f95cffff700 1 -- 172.21.15.59:0/741808172 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f95dc0b2d80 con 0x5612245fcda0 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f96015c5b80 1 -- 172.21.15.59:0/741808172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e8004bb0 con 0x5612245fcda0 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f95cffff700 1 -- 172.21.15.59:0/741808172 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95dc06f020 con 0x5612245fcda0 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f96015c5b80 10 client.15195 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f96015c5b80 1 -- 172.21.15.59:0/741808172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561224613640 con 0x5612245fcda0 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f95cffff700 1 -- 172.21.15.59:0/741808172 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e8004bb0 con 0x5612245fcda0 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f95cffff700 1 -- 172.21.15.59:0/741808172 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f95dc0ae120 con 0x5612245fcda0 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f96015c5b80 10 client.15195 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f96015c5b80 10 client.15195 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f96015c5b80 20 client.15195 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f96015c5b80 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x561224615cc0 0x561224617130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f96015c5b80 4 client.15195 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.773+0000 7f96015c5b80 1 -- 172.21.15.59:0/741808172 --> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612245c8600 con 0x561224615cc0 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.774+0000 7f95f4ace700 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x561224615cc0 0x561224617130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.774+0000 7f95f4ace700 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x561224615cc0 0x561224617130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=efe63287a469b9e1 server_cookie=5ac37da645cfe954 in_seq=0 out_seq=0 2024-05-08T05:29:15.780 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.774+0000 7f95cffff700 10 client.15195 ms_handle_connect on v2:172.21.15.143:6836/555225813 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f95cffff700 1 -- 172.21.15.59:0/741808172 <== mds.3 v2:172.21.15.143:6836/555225813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5612245c8600 con 0x561224615cc0 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f95cffff700 10 client.15195 handle_command_reply: tid=0 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 client.15195 shutdown 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 -- 172.21.15.59:0/741808172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612245fcda0 msgr2=0x5612246053a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612245fcda0 0x5612246053a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f95dc0e09f0 tx=0x7f95dc009310 comp rx=0 tx=0).stop 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 -- 172.21.15.59:0/741808172 shutdown_connections 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x561224615cc0 0x561224617130 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f95cffff700 0 client.15195 ms_handle_reset on v2:172.21.15.143:6836/555225813 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561224600c30 0x5612246058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x561224608000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 --2- 172.21.15.59:0/741808172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5612245fcda0 0x5612246053a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 -- 172.21.15.59:0/741808172 >> 172.21.15.59:0/741808172 conn(0x5612245280b0 msgr2=0x561224601030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 -- 172.21.15.59:0/741808172 shutdown_connections 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 1 -- 172.21.15.59:0/741808172 wait complete. 2024-05-08T05:29:15.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.775+0000 7f96015c5b80 20 client.15195 trim_cache size 0 max 16384 2024-05-08T05:29:15.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f96015c5b80 1 Processor -- start 2024-05-08T05:29:15.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f96015c5b80 1 -- start start 2024-05-08T05:29:15.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245f86f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x561224620980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612245f1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f96015c5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561224615470 con 0x561224600c30 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f96015c5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561224615b40 con 0x5612245fc480 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f96015c5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561224616210 con 0x5612245fcda0 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f95f52cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612245f1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f95f52cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612245f1550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47754/0 (socket says 172.21.15.59:47754) 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f95f52cf700 1 -- 172.21.15.59:0/2121927390 learned_addr learned my addr 172.21.15.59:0/2121927390 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f95f4ace700 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245f86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.797+0000 7f95effff700 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x561224620980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95f52cf700 1 -- 172.21.15.59:0/2121927390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 msgr2=0x561224620980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95f52cf700 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x561224620980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95f52cf700 1 -- 172.21.15.59:0/2121927390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 msgr2=0x5612245f86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:15.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95f52cf700 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245f86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95f52cf700 1 -- 172.21.15.59:0/2121927390 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612245f84d0 con 0x561224600c30 2024-05-08T05:29:15.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95f4ace700 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:15.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95f52cf700 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612245f1550 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f95e800fc00 tx=0x7f95e80079c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99d359d29e115953 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:15.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95da7fc700 1 -- 172.21.15.59:0/2121927390 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8004bb0 con 0x561224600c30 2024-05-08T05:29:15.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95da7fc700 1 -- 172.21.15.59:0/2121927390 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e8004d50 con 0x561224600c30 2024-05-08T05:29:15.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.798+0000 7f95da7fc700 1 -- 172.21.15.59:0/2121927390 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8004420 con 0x561224600c30 2024-05-08T05:29:15.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 -- 172.21.15.59:0/2121927390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 msgr2=0x5612245f1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:15.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612245f1550 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f95e800fc00 tx=0x7f95e80079c0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 -- 172.21.15.59:0/2121927390 shutdown_connections 2024-05-08T05:29:15.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x5612245f1550 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245f86f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 --2- 172.21.15.59:0/2121927390 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x561224620980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 -- 172.21.15.59:0/2121927390 >> 172.21.15.59:0/2121927390 conn(0x5612245280b0 msgr2=0x5612245f0e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:15.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 -- 172.21.15.59:0/2121927390 shutdown_connections 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 -- 172.21.15.59:0/2121927390 wait complete. 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 -- start start 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.799+0000 7f96015c5b80 1 Processor -- start 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fdd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x5612245fe240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f96015c5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x56122467e130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f96015c5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561224616fb0 con 0x561224600c30 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f96015c5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5612246168e0 con 0x5612245fc480 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f96015c5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561224617680 con 0x5612245fcda0 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f96015c5b80 10 client.0 _ll_register_callbacks cb 0x5612244e68d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f52cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x56122467e130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f52cf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x56122467e130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47764/0 (socket says 172.21.15.59:47764) 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f52cf700 1 -- 172.21.15.59:0/4093487540 learned_addr learned my addr 172.21.15.59:0/4093487540 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f4ace700 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fdd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.805 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95effff700 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x5612245fe240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f52cf700 1 -- 172.21.15.59:0/4093487540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 msgr2=0x5612245fe240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f52cf700 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x5612245fe240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f52cf700 1 -- 172.21.15.59:0/4093487540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 msgr2=0x5612245fdd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f52cf700 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fdd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f52cf700 1 -- 172.21.15.59:0/4093487540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56122467e720 con 0x561224600c30 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f4ace700 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95f52cf700 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x56122467e130 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f95e8016010 tx=0x7f95e80075a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da545108c1da780b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95cffff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f95cffff700 1 -- 172.21.15.59:0/4093487540 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95e8004bb0 con 0x561224600c30 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.800+0000 7f96015c5b80 1 -- 172.21.15.59:0/4093487540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f95e8004d50 con 0x561224600c30 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f95cffff700 1 -- 172.21.15.59:0/4093487540 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f95e801d650 con 0x561224600c30 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f95cffff700 1 -- 172.21.15.59:0/4093487540 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f95e803b050 con 0x561224600c30 2024-05-08T05:29:15.806 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f96015c5b80 10 client.15201 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f96015c5b80 1 -- 172.21.15.59:0/4093487540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561224685aa0 con 0x561224600c30 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f95cffff700 1 -- 172.21.15.59:0/4093487540 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95e8004d50 con 0x561224600c30 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f95cffff700 1 -- 172.21.15.59:0/4093487540 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f95e80258f0 con 0x561224600c30 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f96015c5b80 10 client.15201 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f96015c5b80 10 client.15201 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f96015c5b80 20 client.15201 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f96015c5b80 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x561224615cc0 0x561224688b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f96015c5b80 4 client.15201 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f96015c5b80 1 -- 172.21.15.59:0/4093487540 --> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] -- command(tid 0: {"prefix": "status"}) -- 0x56122455c1f0 con 0x561224615cc0 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.801+0000 7f95f4ace700 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x561224615cc0 0x561224688b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f95f4ace700 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x561224615cc0 0x561224688b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2b8c40c55bb13e9 server_cookie=902b161d38c79ace in_seq=0 out_seq=0 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f95cffff700 10 client.15201 ms_handle_connect on v2:172.21.15.143:6836/555225813 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f95cffff700 1 -- 172.21.15.59:0/4093487540 <== mds.3 v2:172.21.15.143:6836/555225813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x56122455c1f0 con 0x561224615cc0 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f95cffff700 10 client.15201 handle_command_reply: tid=0 2024-05-08T05:29:15.807 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 client.15201 shutdown 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 -- 172.21.15.59:0/4093487540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 msgr2=0x56122467e130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x56122467e130 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f95e8016010 tx=0x7f95e80075a0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 -- 172.21.15.59:0/4093487540 shutdown_connections 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561224600c30 0x56122467e130 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5612245fc480 0x5612245fdd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x561224615cc0 0x561224688b30 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 --2- 172.21.15.59:0/4093487540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5612245fcda0 0x5612245fe240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f95cffff700 0 client.15201 ms_handle_reset on v2:172.21.15.143:6836/555225813 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 -- 172.21.15.59:0/4093487540 >> 172.21.15.59:0/4093487540 conn(0x5612245280b0 msgr2=0x561224603890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 -- 172.21.15.59:0/4093487540 shutdown_connections 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.802+0000 7f96015c5b80 1 -- 172.21.15.59:0/4093487540 wait complete. 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f96015c5b80 20 client.15201 trim_cache size 0 max 16384 2024-05-08T05:29:15.808 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 3, 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24421, 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 169.40206247500001, 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 259.91847301400003 2024-05-08T05:29:15.809 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:29:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 -- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f95e00b2880 msgr2=0x7f95e00b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f95e00b2880 0x7f95e00b4d40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f95f019aa20 tx=0x7f95e400a380 comp rx=0 tx=0).stop 2024-05-08T05:29:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 -- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0102450 msgr2=0x7f95f019fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0102450 0x7f95f019fc60 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f95dc000c00 tx=0x7f95dc0057c0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 -- 172.21.15.59:0/4243146206 shutdown_connections 2024-05-08T05:29:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f95e00b2880 0x7f95e00b4d40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f95f0109520 0x7f95f01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.810 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95f0101a90 0x7f95f019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 --2- 172.21.15.59:0/4243146206 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f95f0102450 0x7f95f019fc60 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:15.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.803+0000 7f95f6d32700 1 -- 172.21.15.59:0/4243146206 >> 172.21.15.59:0/4243146206 conn(0x7f95f0078a10 msgr2=0x7f95f0100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:15.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.804+0000 7f95f6d32700 1 -- 172.21.15.59:0/4243146206 shutdown_connections 2024-05-08T05:29:15.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:15.804+0000 7f95f6d32700 1 -- 172.21.15.59:0/4243146206 wait complete. 2024-05-08T05:29:16.563 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T05:29:16.821 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:17.271 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:17 smithi059 ceph-mon[30304]: pgmap v416: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:29:17.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:17 smithi143 ceph-mon[34337]: pgmap v416: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:29:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:17 smithi096 ceph-mon[34253]: pgmap v416: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:29:18.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.523+0000 7f3d408ea700 1 -- 172.21.15.59:0/201456714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 msgr2=0x7f3d3c103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:18.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.523+0000 7f3d408ea700 1 --2- 172.21.15.59:0/201456714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 0x7f3d3c103b60 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0099c0 tx=0x7f3d2c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.523+0000 7f3d408ea700 1 -- 172.21.15.59:0/201456714 shutdown_connections 2024-05-08T05:29:18.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.523+0000 7f3d408ea700 1 --2- 172.21.15.59:0/201456714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d3c108870 0x7f3d3c1103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.523+0000 7f3d408ea700 1 --2- 172.21.15.59:0/201456714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 0x7f3d3c103b60 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.523+0000 7f3d408ea700 1 --2- 172.21.15.59:0/201456714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d3c104130 0x7f3d3c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.523+0000 7f3d408ea700 1 -- 172.21.15.59:0/201456714 >> 172.21.15.59:0/201456714 conn(0x7f3d3c07a610 msgr2=0x7f3d3c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.523+0000 7f3d408ea700 1 -- 172.21.15.59:0/201456714 shutdown_connections 2024-05-08T05:29:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.523+0000 7f3d408ea700 1 -- 172.21.15.59:0/201456714 wait complete. 2024-05-08T05:29:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d408ea700 1 Processor -- start 2024-05-08T05:29:18.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d408ea700 1 -- start start 2024-05-08T05:29:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d408ea700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 0x7f3d3c19f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d408ea700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d3c104130 0x7f3d3c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d408ea700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d3c108870 0x7f3d3c1998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d408ea700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c113660 con 0x7f3d3c103780 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d408ea700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c1134e0 con 0x7f3d3c104130 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d408ea700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c1137e0 con 0x7f3d3c108870 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d3a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 0x7f3d3c19f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d3a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 0x7f3d3c19f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47782/0 (socket says 172.21.15.59:47782) 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d3a59c700 1 -- 172.21.15.59:0/3295780391 learned_addr learned my addr 172.21.15.59:0/3295780391 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d3ad9d700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d3c108870 0x7f3d3c1998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.524+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d3c104130 0x7f3d3c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d3a59c700 1 -- 172.21.15.59:0/3295780391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d3c108870 msgr2=0x7f3d3c1998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d3c108870 0x7f3d3c1998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d3a59c700 1 -- 172.21.15.59:0/3295780391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d3c104130 msgr2=0x7f3d3c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d3c104130 0x7f3d3c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d3a59c700 1 -- 172.21.15.59:0/3295780391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c19a000 con 0x7f3d3c103780 2024-05-08T05:29:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 0x7f3d3c19f6e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00b580 tx=0x7f3d2c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d2b7fe700 1 -- 172.21.15.59:0/3295780391 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c027070 con 0x7f3d3c103780 2024-05-08T05:29:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d2b7fe700 1 -- 172.21.15.59:0/3295780391 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d2c003f20 con 0x7f3d3c103780 2024-05-08T05:29:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d408ea700 1 -- 172.21.15.59:0/3295780391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c19a290 con 0x7f3d3c103780 2024-05-08T05:29:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.525+0000 7f3d2b7fe700 1 -- 172.21.15.59:0/3295780391 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c022d70 con 0x7f3d3c103780 2024-05-08T05:29:18.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.526+0000 7f3d408ea700 1 -- 172.21.15.59:0/3295780391 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c1a65b0 con 0x7f3d3c103780 2024-05-08T05:29:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.528+0000 7f3d2b7fe700 1 -- 172.21.15.59:0/3295780391 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f3d2c022500 con 0x7f3d3c103780 2024-05-08T05:29:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.528+0000 7f3d2b7fe700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3d240b2880 0x7f3d240b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.529+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3d240b2880 0x7f3d240b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.529+0000 7f3d2b7fe700 1 -- 172.21.15.59:0/3295780391 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3d2c0e3280 con 0x7f3d3c103780 2024-05-08T05:29:18.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.529+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3d240b2880 0x7f3d240b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3d30005ea0 tx=0x7f3d30005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:18.530 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.529+0000 7f3d4b17db80 1 Processor -- start 2024-05-08T05:29:18.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d4b17db80 1 -- start start 2024-05-08T05:29:18.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829b480 0x5562c829b860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829bda0 0x5562c829f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 0x5562c82a00b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5562c81ed810 con 0x5562c829b480 2024-05-08T05:29:18.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562c818c770 con 0x5562c829fc30 2024-05-08T05:29:18.531 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5562c81e97d0 con 0x5562c829bda0 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d3a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829b480 0x5562c829b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d3a59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829b480 0x5562c829b860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47790/0 (socket says 172.21.15.59:47790) 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d3a59c700 1 -- 172.21.15.59:0/2667561350 learned_addr learned my addr 172.21.15.59:0/2667561350 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d3ad9d700 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 0x5562c82a00b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829bda0 0x5562c829f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d3a59c700 1 -- 172.21.15.59:0/2667561350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829bda0 msgr2=0x5562c829f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829bda0 0x5562c829f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d3a59c700 1 -- 172.21.15.59:0/2667561350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 msgr2=0x5562c82a00b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 0x5562c82a00b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d3a59c700 1 -- 172.21.15.59:0/2667561350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562c813a880 con 0x5562c829b480 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.530+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829bda0 0x5562c829f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.531+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829b480 0x5562c829b860 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0a6620 tx=0x7f3d2c0228a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=364ba85569032435 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.531+0000 7f3d28ff9700 1 -- 172.21.15.59:0/2667561350 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0ae950 con 0x5562c829b480 2024-05-08T05:29:18.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.531+0000 7f3d28ff9700 1 -- 172.21.15.59:0/2667561350 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d2c0b3370 con 0x5562c829b480 2024-05-08T05:29:18.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.531+0000 7f3d28ff9700 1 -- 172.21.15.59:0/2667561350 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d2c0a9800 con 0x5562c829b480 2024-05-08T05:29:18.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2667561350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829b480 msgr2=0x5562c829b860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:18.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829b480 0x5562c829b860 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c0a6620 tx=0x7f3d2c0228a0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2667561350 shutdown_connections 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 0x5562c82a00b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829b480 0x5562c829b860 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/2667561350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829bda0 0x5562c829f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2667561350 >> 172.21.15.59:0/2667561350 conn(0x5562c81c70b0 msgr2=0x5562c8295c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2667561350 shutdown_connections 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2667561350 wait complete. 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 -- start start 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 Processor -- start 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 0x5562c82a70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c82a4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.532+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829fc30 0x5562c82a49a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5562c82a7c00 con 0x5562c829bda0 2024-05-08T05:29:18.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562c8153be0 con 0x5562c829b480 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5562c82a7d80 con 0x5562c829fc30 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d4b17db80 10 client.0 _ll_register_callbacks cb 0x5562c81858d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d39d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c82a4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d39d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c82a4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47792/0 (socket says 172.21.15.59:47792) 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d39d9b700 1 -- 172.21.15.59:0/16509783 learned_addr learned my addr 172.21.15.59:0/16509783 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 0x5562c82a70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d39d9b700 1 -- 172.21.15.59:0/16509783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829fc30 msgr2=0x5562c82a49a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829fc30 0x5562c82a49a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d39d9b700 1 -- 172.21.15.59:0/16509783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 msgr2=0x5562c82a70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 0x5562c82a70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d39d9b700 1 -- 172.21.15.59:0/16509783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562c8189640 con 0x5562c829bda0 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 0x5562c82a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c82a4460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3d30010830 tx=0x7f3d300047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fcf18a0cb7835e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:18.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d21ffb700 1 -- 172.21.15.59:0/16509783 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3000f950 con 0x5562c829bda0 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d21ffb700 1 -- 172.21.15.59:0/16509783 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3d3000faf0 con 0x5562c829bda0 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.533+0000 7f3d4b17db80 1 -- 172.21.15.59:0/16509783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d2c0b3370 con 0x5562c829bda0 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d21ffb700 1 -- 172.21.15.59:0/16509783 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3d3002f050 con 0x5562c829bda0 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d4b17db80 10 client.15213 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d4b17db80 1 -- 172.21.15.59:0/16509783 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5562c82b24a0 con 0x5562c829bda0 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d21ffb700 1 -- 172.21.15.59:0/16509783 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d2c0b3370 con 0x5562c829bda0 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d21ffb700 1 -- 172.21.15.59:0/16509783 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3d3001d3c0 con 0x5562c829bda0 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d4b17db80 10 client.15213 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d4b17db80 10 client.15213 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d4b17db80 20 client.15213 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5562c82b4b50 0x5562c82b5fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d4b17db80 4 client.15213 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.534+0000 7f3d4b17db80 1 -- 172.21.15.59:0/16509783 --> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5562c8267600 con 0x5562c82b4b50 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.535+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5562c82b4b50 0x5562c82b5fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.535+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5562c82b4b50 0x5562c82b5fc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ca17f709ae5aa508 server_cookie=7220cfe547e04627 in_seq=0 out_seq=0 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.535+0000 7f3d21ffb700 10 client.15213 ms_handle_connect on v2:172.21.15.143:6838/2166229809 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d21ffb700 1 -- 172.21.15.59:0/16509783 <== mds.2 v2:172.21.15.143:6838/2166229809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5562c8267600 con 0x5562c82b4b50 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d21ffb700 10 client.15213 handle_command_reply: tid=0 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 client.15213 shutdown 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 -- 172.21.15.59:0/16509783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 msgr2=0x5562c82a4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c82a4460 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f3d30010830 tx=0x7f3d300047c0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 -- 172.21.15.59:0/16509783 shutdown_connections 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829fc30 0x5562c82a49a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 0x5562c82a70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5562c82b4b50 0x5562c82b5fc0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/16509783 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c82a4460 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 -- 172.21.15.59:0/16509783 >> 172.21.15.59:0/16509783 conn(0x5562c81c70b0 msgr2=0x5562c82906d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:18.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d21ffb700 0 client.15213 ms_handle_reset on v2:172.21.15.143:6838/2166229809 2024-05-08T05:29:18.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 -- 172.21.15.59:0/16509783 shutdown_connections 2024-05-08T05:29:18.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 1 -- 172.21.15.59:0/16509783 wait complete. 2024-05-08T05:29:18.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.536+0000 7f3d4b17db80 20 client.15213 trim_cache size 0 max 16384 2024-05-08T05:29:18.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.557+0000 7f3d4b17db80 1 Processor -- start 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.557+0000 7f3d4b17db80 1 -- start start 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.557+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829b480 0x5562c82bf810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.557+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c828fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.557+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 0x5562c8290380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.557+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5562c82b4300 con 0x5562c829bda0 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.557+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562c82b49d0 con 0x5562c829fc30 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.557+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5562c82b50a0 con 0x5562c829b480 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d39d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c828fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d39d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c828fe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47798/0 (socket says 172.21.15.59:47798) 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d39d9b700 1 -- 172.21.15.59:0/2989027601 learned_addr learned my addr 172.21.15.59:0/2989027601 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/2989027601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829b480 0x5562c82bf810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d39d9b700 1 -- 172.21.15.59:0/2989027601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829b480 msgr2=0x5562c82bf810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/2989027601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829b480 0x5562c82bf810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d3ad9d700 1 --2- 172.21.15.59:0/2989027601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 0x5562c8290380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d39d9b700 1 -- 172.21.15.59:0/2989027601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 msgr2=0x5562c8290380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/2989027601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 0x5562c8290380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d39d9b700 1 -- 172.21.15.59:0/2989027601 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562c82974d0 con 0x5562c829bda0 2024-05-08T05:29:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/2989027601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c828fe40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3d300059d0 tx=0x7f3d3001dfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb090f59d37c9d1f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d23fff700 1 -- 172.21.15.59:0/2989027601 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d3001d9d0 con 0x5562c829bda0 2024-05-08T05:29:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d23fff700 1 -- 172.21.15.59:0/2989027601 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3003a070 con 0x5562c829bda0 2024-05-08T05:29:18.560 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.558+0000 7f3d23fff700 1 -- 172.21.15.59:0/2989027601 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d300354a0 con 0x5562c829bda0 2024-05-08T05:29:18.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2989027601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 msgr2=0x5562c828fe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:18.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/2989027601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c828fe40 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f3d300059d0 tx=0x7f3d3001dfb0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2989027601 shutdown_connections 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/2989027601 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829fc30 0x5562c8290380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/2989027601 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829b480 0x5562c82bf810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/2989027601 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c828fe40 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2989027601 >> 172.21.15.59:0/2989027601 conn(0x5562c81c70b0 msgr2=0x5562c82a2d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2989027601 shutdown_connections 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 -- 172.21.15.59:0/2989027601 wait complete. 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.559+0000 7f3d4b17db80 1 -- start start 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d4b17db80 1 Processor -- start 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 0x5562c829c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c829cb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d4b17db80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829fc30 0x5562c829d040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5562c82b5e40 con 0x5562c829bda0 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5562c82b5770 con 0x5562c829b480 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d4b17db80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5562c82b6510 con 0x5562c829fc30 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d4b17db80 10 client.0 _ll_register_callbacks cb 0x5562c81858d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d39d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c829cb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d39d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c829cb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47808/0 (socket says 172.21.15.59:47808) 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d39d9b700 1 -- 172.21.15.59:0/460023151 learned_addr learned my addr 172.21.15.59:0/460023151 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d3ad9d700 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829fc30 0x5562c829d040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 0x5562c829c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d39d9b700 1 -- 172.21.15.59:0/460023151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829fc30 msgr2=0x5562c829d040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829fc30 0x5562c829d040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d39d9b700 1 -- 172.21.15.59:0/460023151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 msgr2=0x5562c829c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:18.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 0x5562c829c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d39d9b700 1 -- 172.21.15.59:0/460023151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562c8323040 con 0x5562c829bda0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d39d9b700 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c829cb00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d30037040 tx=0x7f3d3001dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d655a4f3c8ea31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.560+0000 7f3d21ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d21ffb700 1 -- 172.21.15.59:0/460023151 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d30005610 con 0x5562c829bda0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d21ffb700 1 -- 172.21.15.59:0/460023151 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f3d30004710 con 0x5562c829bda0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d4b17db80 1 -- 172.21.15.59:0/460023151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d3003a070 con 0x5562c829bda0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d21ffb700 1 -- 172.21.15.59:0/460023151 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3d3004c050 con 0x5562c829bda0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d4b17db80 10 client.15219 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d4b17db80 1 -- 172.21.15.59:0/460023151 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5562c8324340 con 0x5562c829bda0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d21ffb700 1 -- 172.21.15.59:0/460023151 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d3003a070 con 0x5562c829bda0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d21ffb700 1 -- 172.21.15.59:0/460023151 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3d30032070 con 0x5562c829bda0 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d4b17db80 10 client.15219 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d4b17db80 10 client.15219 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d4b17db80 20 client.15219 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:18.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5562c8328a80 0x5562c8327790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d4b17db80 4 client.15219 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.561+0000 7f3d4b17db80 1 -- 172.21.15.59:0/460023151 --> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] -- command(tid 0: {"prefix": "status"}) -- 0x5562c81fb1f0 con 0x5562c8328a80 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5562c8328a80 0x5562c8327790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d3a59c700 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5562c8328a80 0x5562c8327790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a0db885cd652af server_cookie=117e2e71a03dd4e7 in_seq=0 out_seq=0 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d21ffb700 10 client.15219 ms_handle_connect on v2:172.21.15.143:6838/2166229809 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d21ffb700 1 -- 172.21.15.59:0/460023151 <== mds.2 v2:172.21.15.143:6838/2166229809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5562c81fb1f0 con 0x5562c8328a80 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d21ffb700 10 client.15219 handle_command_reply: tid=0 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d4b17db80 1 client.15219 shutdown 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d4b17db80 1 -- 172.21.15.59:0/460023151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 msgr2=0x5562c829cb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c829cb00 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f3d30037040 tx=0x7f3d3001dc30 comp rx=0 tx=0).stop 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d4b17db80 1 -- 172.21.15.59:0/460023151 shutdown_connections 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5562c829fc30 0x5562c829d040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5562c829b480 0x5562c829c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5562c8328a80 0x5562c8327790 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d4b17db80 1 --2- 172.21.15.59:0/460023151 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5562c829bda0 0x5562c829cb00 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d4b17db80 1 -- 172.21.15.59:0/460023151 >> 172.21.15.59:0/460023151 conn(0x5562c81c70b0 msgr2=0x5562c82906d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.562+0000 7f3d21ffb700 0 client.15219 ms_handle_reset on v2:172.21.15.143:6838/2166229809 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d4b17db80 1 -- 172.21.15.59:0/460023151 shutdown_connections 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d4b17db80 1 -- 172.21.15.59:0/460023151 wait complete. 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d4b17db80 20 client.15219 trim_cache size 0 max 16384 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 2, 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24433, 2024-05-08T05:29:18.569 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:29:18.570 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:29:18.570 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:29:18.570 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 173.169835879, 2024-05-08T05:29:18.570 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:29:18.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:29:18.570 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:29:18.570 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 257.03264459000002 2024-05-08T05:29:18.570 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 -- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3d240b2880 msgr2=0x7f3d240b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3d240b2880 0x7f3d240b4d40 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f3d30005ea0 tx=0x7f3d30005e30 comp rx=0 tx=0).stop 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 -- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 msgr2=0x7f3d3c19f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 0x7f3d3c19f6e0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c00b580 tx=0x7f3d2c0047b0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 -- 172.21.15.59:0/3295780391 shutdown_connections 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3d3c108870 0x7f3d3c1998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3d3c103780 0x7f3d3c19f6e0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f3d240b2880 0x7f3d240b4d40 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 --2- 172.21.15.59:0/3295780391 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3d3c104130 0x7f3d3c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 -- 172.21.15.59:0/3295780391 >> 172.21.15.59:0/3295780391 conn(0x7f3d3c07a610 msgr2=0x7f3d3c102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 -- 172.21.15.59:0/3295780391 shutdown_connections 2024-05-08T05:29:18.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:18.563+0000 7f3d408ea700 1 -- 172.21.15.59:0/3295780391 wait complete. 2024-05-08T05:29:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:19 smithi059 ceph-mon[30304]: pgmap v417: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:29:19.381 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-08T05:29:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:19 smithi143 ceph-mon[34337]: pgmap v417: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:29:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:19 smithi096 ceph-mon[34253]: pgmap v417: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 8 op/s 2024-05-08T05:29:19.638 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:21.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.332+0000 7f1947438700 1 -- 172.21.15.59:0/4207597359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940108850 msgr2=0x7f19401103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:21.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.332+0000 7f1947438700 1 --2- 172.21.15.59:0/4207597359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940108850 0x7f19401103d0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f193800b0d0 tx=0x7f193800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.332+0000 7f1947438700 1 -- 172.21.15.59:0/4207597359 shutdown_connections 2024-05-08T05:29:21.333 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.332+0000 7f1947438700 1 --2- 172.21.15.59:0/4207597359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940108850 0x7f19401103d0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.332+0000 7f1947438700 1 --2- 172.21.15.59:0/4207597359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1940103750 0x7f1940103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.332+0000 7f1947438700 1 --2- 172.21.15.59:0/4207597359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1940104100 0x7f1940108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.332+0000 7f1947438700 1 -- 172.21.15.59:0/4207597359 >> 172.21.15.59:0/4207597359 conn(0x7f194007a600 msgr2=0x7f194007aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:21.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.332+0000 7f1947438700 1 -- 172.21.15.59:0/4207597359 shutdown_connections 2024-05-08T05:29:21.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.332+0000 7f1947438700 1 -- 172.21.15.59:0/4207597359 wait complete. 2024-05-08T05:29:21.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:21 smithi059 ceph-mon[30304]: pgmap v418: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f1947438700 1 Processor -- start 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f1947438700 1 -- start start 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f1947438700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940103750 0x7f19400757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f1947438700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1940104100 0x7f1940075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f1947438700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1940108850 0x7f1940076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f1947438700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1940113670 con 0x7f1940103750 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f1947438700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f19401134f0 con 0x7f1940104100 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f1947438700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f19401137f0 con 0x7f1940108850 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f19451d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940103750 0x7f19400757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f19451d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940103750 0x7f19400757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56524/0 (socket says 172.21.15.59:56524) 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f19451d4700 1 -- 172.21.15.59:0/3822705699 learned_addr learned my addr 172.21.15.59:0/3822705699 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f19449d3700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1940104100 0x7f1940075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.333+0000 7f19459d5700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1940108850 0x7f1940076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19451d4700 1 -- 172.21.15.59:0/3822705699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1940108850 msgr2=0x7f1940076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19451d4700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1940108850 0x7f1940076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19451d4700 1 -- 172.21.15.59:0/3822705699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1940104100 msgr2=0x7f1940075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19451d4700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1940104100 0x7f1940075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19451d4700 1 -- 172.21.15.59:0/3822705699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19401a8010 con 0x7f1940103750 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19459d5700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1940108850 0x7f1940076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19451d4700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940103750 0x7f19400757a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f193c00e9a0 tx=0x7f193c00ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19367fc700 1 -- 172.21.15.59:0/3822705699 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c00cbd0 con 0x7f1940103750 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19367fc700 1 -- 172.21.15.59:0/3822705699 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f193c00cd70 con 0x7f1940103750 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f19367fc700 1 -- 172.21.15.59:0/3822705699 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c018680 con 0x7f1940103750 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f1947438700 1 -- 172.21.15.59:0/3822705699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19401a8300 con 0x7f1940103750 2024-05-08T05:29:21.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.334+0000 7f1947438700 1 -- 172.21.15.59:0/3822705699 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19401a87f0 con 0x7f1940103750 2024-05-08T05:29:21.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.337+0000 7f19367fc700 1 -- 172.21.15.59:0/3822705699 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f193c010430 con 0x7f1940103750 2024-05-08T05:29:21.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.337+0000 7f19367fc700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f192c0b2880 0x7f192c0b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.338+0000 7f19449d3700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f192c0b2880 0x7f192c0b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.338+0000 7f19367fc700 1 -- 172.21.15.59:0/3822705699 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f193c0e2840 con 0x7f1940103750 2024-05-08T05:29:21.339 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.338+0000 7f19449d3700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f192c0b2880 0x7f192c0b4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19401030c0 tx=0x7f19300073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:21.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f1951ccbb80 1 Processor -- start 2024-05-08T05:29:21.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f1951ccbb80 1 -- start start 2024-05-08T05:29:21.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x55895909f860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.340 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 0x5589590a36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589590a3c30 0x5589590a40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558958ff1810 con 0x55895909f480 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558958f90770 con 0x55895909fda0 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558958fed7d0 con 0x5589590a3c30 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19451d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x55895909f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19451d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x55895909f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56528/0 (socket says 172.21.15.59:56528) 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19451d4700 1 -- 172.21.15.59:0/3010341315 learned_addr learned my addr 172.21.15.59:0/3010341315 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19459d5700 1 --2- 172.21.15.59:0/3010341315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589590a3c30 0x5589590a40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19449d3700 1 --2- 172.21.15.59:0/3010341315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 0x5589590a36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19451d4700 1 -- 172.21.15.59:0/3010341315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589590a3c30 msgr2=0x5589590a40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19451d4700 1 --2- 172.21.15.59:0/3010341315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589590a3c30 0x5589590a40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19451d4700 1 -- 172.21.15.59:0/3010341315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 msgr2=0x5589590a36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19451d4700 1 --2- 172.21.15.59:0/3010341315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 0x5589590a36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.341 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19451d4700 1 -- 172.21.15.59:0/3010341315 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558958f3e880 con 0x55895909f480 2024-05-08T05:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.339+0000 7f19451d4700 1 --2- 172.21.15.59:0/3010341315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x55895909f860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f193c0a5b50 tx=0x7f193c0a5ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f647891b4bda8335 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.340+0000 7f192b7fe700 1 -- 172.21.15.59:0/3010341315 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0b2280 con 0x55895909f480 2024-05-08T05:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.340+0000 7f192b7fe700 1 -- 172.21.15.59:0/3010341315 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f193c0a95e0 con 0x55895909f480 2024-05-08T05:29:21.342 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.340+0000 7f192b7fe700 1 -- 172.21.15.59:0/3010341315 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0b2700 con 0x55895909f480 2024-05-08T05:29:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.340+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3010341315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 msgr2=0x55895909f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.340+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3010341315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x55895909f860 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f193c0a5b50 tx=0x7f193c0a5ac0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3010341315 shutdown_connections 2024-05-08T05:29:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3010341315 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5589590a3c30 0x5589590a40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3010341315 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x55895909f860 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.345 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3010341315 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 0x5589590a36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3010341315 >> 172.21.15.59:0/3010341315 conn(0x558958fcb0b0 msgr2=0x558959099c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3010341315 shutdown_connections 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3010341315 wait complete. 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 -- start start 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 Processor -- start 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x5589590ab0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 0x5589590a8460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.341+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589590a3c30 0x5589590a89a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5589590abd80 con 0x55895909f480 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558958f57be0 con 0x5589590a3c30 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5589590abf00 con 0x55895909fda0 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1951ccbb80 10 client.0 _ll_register_callbacks cb 0x558958f898d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19451d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x5589590ab0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.346 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19451d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x5589590ab0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56532/0 (socket says 172.21.15.59:56532) 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19451d4700 1 -- 172.21.15.59:0/3611083510 learned_addr learned my addr 172.21.15.59:0/3611083510 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19459d5700 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589590a3c30 0x5589590a89a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19451d4700 1 -- 172.21.15.59:0/3611083510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 msgr2=0x5589590a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19449d3700 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 0x5589590a8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19451d4700 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 0x5589590a8460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19451d4700 1 -- 172.21.15.59:0/3611083510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589590a3c30 msgr2=0x5589590a89a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19451d4700 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589590a3c30 0x5589590a89a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19451d4700 1 -- 172.21.15.59:0/3611083510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558958f8d640 con 0x55895909f480 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f19451d4700 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x5589590ab0c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f193c005830 tx=0x7f193c010950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7cc30059d9c8267 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1928ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1928ff9700 1 -- 172.21.15.59:0/3611083510 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193c0c2030 con 0x55895909f480 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1928ff9700 1 -- 172.21.15.59:0/3611083510 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f193c0aae00 con 0x55895909f480 2024-05-08T05:29:21.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3611083510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f193c0a95e0 con 0x55895909f480 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1928ff9700 1 -- 172.21.15.59:0/3611083510 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f193c0a9ad0 con 0x55895909f480 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.342+0000 7f1951ccbb80 10 client.15231 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3611083510 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589590b6630 con 0x55895909f480 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f1928ff9700 1 -- 172.21.15.59:0/3611083510 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f193c0a95e0 con 0x55895909f480 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f1928ff9700 1 -- 172.21.15.59:0/3611083510 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f193c06f260 con 0x55895909f480 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f1951ccbb80 10 client.15231 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f1951ccbb80 10 client.15231 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f1951ccbb80 20 client.15231 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5589590b8820 0x5589590b9c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f1951ccbb80 4 client.15231 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3611083510 --> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55895906b600 con 0x5589590b8820 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.343+0000 7f19449d3700 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5589590b8820 0x5589590b9c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.344+0000 7f19449d3700 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5589590b8820 0x5589590b9c90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=17f064e4ef1eb367 server_cookie=23d78562a58e9e09 in_seq=0 out_seq=0 2024-05-08T05:29:21.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.344+0000 7f1928ff9700 10 client.15231 ms_handle_connect on v2:172.21.15.96:6836/780978870 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1928ff9700 1 -- 172.21.15.59:0/3611083510 <== mds.1 v2:172.21.15.96:6836/780978870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55895906b600 con 0x5589590b8820 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1928ff9700 10 client.15231 handle_command_reply: tid=0 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 client.15231 shutdown 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3611083510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 msgr2=0x5589590ab0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x5589590ab0c0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f193c005830 tx=0x7f193c010950 comp rx=0 tx=0).stop 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3611083510 shutdown_connections 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x5589590b8820 0x5589590b9c90 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5589590a3c30 0x5589590a89a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55895909f480 0x5589590ab0c0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1928ff9700 0 client.15231 ms_handle_reset on v2:172.21.15.96:6836/780978870 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3611083510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 0x5589590a8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3611083510 >> 172.21.15.59:0/3611083510 conn(0x558958fcb0b0 msgr2=0x5589590946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:21.349 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3611083510 shutdown_connections 2024-05-08T05:29:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3611083510 wait complete. 2024-05-08T05:29:21.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.345+0000 7f1951ccbb80 20 client.15231 trim_cache size 0 max 16384 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.366+0000 7f1951ccbb80 1 Processor -- start 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.366+0000 7f1951ccbb80 1 -- start start 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.366+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909f480 0x558959099b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.366+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 0x5589590a14d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.366+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.366+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5589590b7fd0 con 0x5589590a3c30 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.366+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589590b86a0 con 0x55895909fda0 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.366+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5589590b8d70 con 0x55895909f480 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19459d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19459d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56546/0 (socket says 172.21.15.59:56546) 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19459d5700 1 -- 172.21.15.59:0/1530386321 learned_addr learned my addr 172.21.15.59:0/1530386321 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19449d3700 1 --2- 172.21.15.59:0/1530386321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 0x5589590a14d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19451d4700 1 --2- 172.21.15.59:0/1530386321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909f480 0x558959099b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19459d5700 1 -- 172.21.15.59:0/1530386321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909f480 msgr2=0x558959099b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19459d5700 1 --2- 172.21.15.59:0/1530386321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909f480 0x558959099b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19459d5700 1 -- 172.21.15.59:0/1530386321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 msgr2=0x5589590a14d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19459d5700 1 --2- 172.21.15.59:0/1530386321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 0x5589590a14d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19459d5700 1 -- 172.21.15.59:0/1530386321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589590a3a40 con 0x5589590a3c30 2024-05-08T05:29:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f19459d5700 1 --2- 172.21.15.59:0/1530386321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590a1a10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1938009520 tx=0x7f1938009550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2be3da044c306e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f192affd700 1 -- 172.21.15.59:0/1530386321 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f193800be00 con 0x5589590a3c30 2024-05-08T05:29:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f192affd700 1 -- 172.21.15.59:0/1530386321 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19380074e0 con 0x5589590a3c30 2024-05-08T05:29:21.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.367+0000 7f192affd700 1 -- 172.21.15.59:0/1530386321 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938005270 con 0x5589590a3c30 2024-05-08T05:29:21.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 -- 172.21.15.59:0/1530386321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 msgr2=0x5589590a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:21.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/1530386321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590a1a10 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f1938009520 tx=0x7f1938009550 comp rx=0 tx=0).stop 2024-05-08T05:29:21.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 -- 172.21.15.59:0/1530386321 shutdown_connections 2024-05-08T05:29:21.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/1530386321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590a1a10 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/1530386321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909f480 0x558959099b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/1530386321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909fda0 0x5589590a14d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.373 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 -- 172.21.15.59:0/1530386321 >> 172.21.15.59:0/1530386321 conn(0x558958fcb0b0 msgr2=0x5589590946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 -- 172.21.15.59:0/1530386321 shutdown_connections 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 -- 172.21.15.59:0/1530386321 wait complete. 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 -- start start 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.368+0000 7f1951ccbb80 1 Processor -- start 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909f480 0x5589590c3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 0x5589590c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f1951ccbb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590bd510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5589590b9b10 con 0x5589590a3c30 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5589590b9440 con 0x55895909f480 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f1951ccbb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5589590ba1e0 con 0x55895909fda0 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f1951ccbb80 10 client.0 _ll_register_callbacks cb 0x558958f898d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19459d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590bd510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.374 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19459d5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590bd510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56560/0 (socket says 172.21.15.59:56560) 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19459d5700 1 -- 172.21.15.59:0/3925593058 learned_addr learned my addr 172.21.15.59:0/3925593058 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19451d4700 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909f480 0x5589590c3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19449d3700 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 0x5589590c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19459d5700 1 -- 172.21.15.59:0/3925593058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 msgr2=0x5589590c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19459d5700 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 0x5589590c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19459d5700 1 -- 172.21.15.59:0/3925593058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909f480 msgr2=0x5589590c3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19459d5700 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909f480 0x5589590c3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.369+0000 7f19459d5700 1 -- 172.21.15.59:0/3925593058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589590bdd50 con 0x5589590a3c30 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f19459d5700 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590bd510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f193800bb50 tx=0x7f1938005a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2781a6e443e195ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1928ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1928ff9700 1 -- 172.21.15.59:0/3925593058 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1938027680 con 0x5589590a3c30 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1928ff9700 1 -- 172.21.15.59:0/3925593058 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f1938027820 con 0x5589590a3c30 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3925593058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f19380074e0 con 0x5589590a3c30 2024-05-08T05:29:21.375 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1928ff9700 1 -- 172.21.15.59:0/3925593058 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1938038050 con 0x5589590a3c30 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1951ccbb80 10 client.15237 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3925593058 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558959124810 con 0x5589590a3c30 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1928ff9700 1 -- 172.21.15.59:0/3925593058 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f19380074e0 con 0x5589590a3c30 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1928ff9700 1 -- 172.21.15.59:0/3925593058 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1938011070 con 0x5589590a3c30 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1951ccbb80 10 client.15237 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1951ccbb80 10 client.15237 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1951ccbb80 20 client.15237 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.370+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x558959128e80 0x558959127b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.371+0000 7f1951ccbb80 4 client.15237 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.371+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3925593058 --> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] -- command(tid 0: {"prefix": "status"}) -- 0x558958fff1f0 con 0x558959128e80 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.371+0000 7f19451d4700 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x558959128e80 0x558959127b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:21.376 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.371+0000 7f19451d4700 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x558959128e80 0x558959127b90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c2f2d6319dd09744 server_cookie=74847de5fabad63f in_seq=0 out_seq=0 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.371+0000 7f1928ff9700 10 client.15237 ms_handle_connect on v2:172.21.15.96:6836/780978870 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.371+0000 7f1928ff9700 1 -- 172.21.15.59:0/3925593058 <== mds.1 v2:172.21.15.96:6836/780978870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x558958fff1f0 con 0x558959128e80 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.371+0000 7f1928ff9700 10 client.15237 handle_command_reply: tid=0 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.371+0000 7f1951ccbb80 1 client.15237 shutdown 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3925593058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 msgr2=0x5589590bd510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590bd510 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f193800bb50 tx=0x7f1938005a60 comp rx=0 tx=0).stop 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3925593058 shutdown_connections 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x558959128e80 0x558959127b90 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5589590a3c30 0x5589590bd510 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55895909f480 0x5589590c3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 --2- 172.21.15.59:0/3925593058 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55895909fda0 0x5589590c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3925593058 >> 172.21.15.59:0/3925593058 conn(0x558958fcb0b0 msgr2=0x5589590946d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1928ff9700 0 client.15237 ms_handle_reset on v2:172.21.15.96:6836/780978870 2024-05-08T05:29:21.377 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3925593058 shutdown_connections 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 1 -- 172.21.15.59:0/3925593058 wait complete. 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1951ccbb80 20 client.15237 trim_cache size 0 max 16384 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 1, 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24443, 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:29:21.378 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 176.983399848, 2024-05-08T05:29:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:29:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:29:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:29:21.379 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 262.570259642 2024-05-08T05:29:21.379 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 -- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f192c0b2880 msgr2=0x7f192c0b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f192c0b2880 0x7f192c0b4d40 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f19401030c0 tx=0x7f19300073d0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 -- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940103750 msgr2=0x7f19400757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940103750 0x7f19400757a0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f193c00e9a0 tx=0x7f193c00ecb0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 -- 172.21.15.59:0/3822705699 shutdown_connections 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1940108850 0x7f1940076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1940103750 0x7f19400757a0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f192c0b2880 0x7f192c0b4d40 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 --2- 172.21.15.59:0/3822705699 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1940104100 0x7f1940075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.372+0000 7f1947438700 1 -- 172.21.15.59:0/3822705699 >> 172.21.15.59:0/3822705699 conn(0x7f194007a600 msgr2=0x7f1940101740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.373+0000 7f1947438700 1 -- 172.21.15.59:0/3822705699 shutdown_connections 2024-05-08T05:29:21.380 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:21.373+0000 7f1947438700 1 -- 172.21.15.59:0/3822705699 wait complete. 2024-05-08T05:29:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:21 smithi143 ceph-mon[34337]: pgmap v418: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:21 smithi096 ceph-mon[34253]: pgmap v418: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:22.055 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-08T05:29:22.311 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:23 smithi143 ceph-mon[34337]: pgmap v419: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:29:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:23 smithi096 ceph-mon[34253]: pgmap v419: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:29:23.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:23 smithi059 ceph-mon[30304]: pgmap v419: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 10 op/s 2024-05-08T05:29:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.202+0000 7f84e4eeb700 1 -- 172.21.15.59:0/4250182436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 msgr2=0x7f84e01099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.202+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/4250182436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 0x7f84e01099e0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009990 tx=0x7f84d0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.202+0000 7f84e4eeb700 1 -- 172.21.15.59:0/4250182436 shutdown_connections 2024-05-08T05:29:24.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.202+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/4250182436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84e0103eb0 0x7f84e0110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.202+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/4250182436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 0x7f84e01099e0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.202+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/4250182436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84e01034f0 0x7f84e0103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.202+0000 7f84e4eeb700 1 -- 172.21.15.59:0/4250182436 >> 172.21.15.59:0/4250182436 conn(0x7f84e007a650 msgr2=0x7f84e007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:24.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.202+0000 7f84e4eeb700 1 -- 172.21.15.59:0/4250182436 shutdown_connections 2024-05-08T05:29:24.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.202+0000 7f84e4eeb700 1 -- 172.21.15.59:0/4250182436 wait complete. 2024-05-08T05:29:24.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.203+0000 7f84e4eeb700 1 Processor -- start 2024-05-08T05:29:24.204 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.203+0000 7f84e4eeb700 1 -- start start 2024-05-08T05:29:24.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.203+0000 7f84e4eeb700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84e01034f0 0x7f84e00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.205 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.203+0000 7f84e4eeb700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84e0103eb0 0x7f84e0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.203+0000 7f84e4eeb700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 0x7f84e0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.203+0000 7f84e4eeb700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f84e0113750 con 0x7f84e0109600 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.203+0000 7f84e4eeb700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01135d0 con 0x7f84e0103eb0 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.203+0000 7f84e4eeb700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f84e01138d0 con 0x7f84e01034f0 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ded9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 0x7f84e0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ded9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 0x7f84e0076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56564/0 (socket says 172.21.15.59:56564) 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ded9d700 1 -- 172.21.15.59:0/1400618770 learned_addr learned my addr 172.21.15.59:0/1400618770 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84de59c700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84e01034f0 0x7f84e00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84e0103eb0 0x7f84e0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ded9d700 1 -- 172.21.15.59:0/1400618770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84e01034f0 msgr2=0x7f84e00757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ded9d700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84e01034f0 0x7f84e00757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ded9d700 1 -- 172.21.15.59:0/1400618770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84e0103eb0 msgr2=0x7f84e0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:24.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ded9d700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84e0103eb0 0x7f84e0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ded9d700 1 -- 172.21.15.59:0/1400618770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84e01a81d0 con 0x7f84e0109600 2024-05-08T05:29:24.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84ded9d700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 0x7f84e0076220 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f84d800b9e0 tx=0x7f84d800bcf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:24.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84cf7fe700 1 -- 172.21.15.59:0/1400618770 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d800d820 con 0x7f84e0109600 2024-05-08T05:29:24.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84e4eeb700 1 -- 172.21.15.59:0/1400618770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84e01a84c0 con 0x7f84e0109600 2024-05-08T05:29:24.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.204+0000 7f84cf7fe700 1 -- 172.21.15.59:0/1400618770 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d8015420 con 0x7f84e0109600 2024-05-08T05:29:24.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.205+0000 7f84cf7fe700 1 -- 172.21.15.59:0/1400618770 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d8014610 con 0x7f84e0109600 2024-05-08T05:29:24.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.205+0000 7f84e4eeb700 1 -- 172.21.15.59:0/1400618770 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84e01a8a00 con 0x7f84e0109600 2024-05-08T05:29:24.207 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.207+0000 7f84cf7fe700 1 -- 172.21.15.59:0/1400618770 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7f84d80147b0 con 0x7f84e0109600 2024-05-08T05:29:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.208+0000 7f84cf7fe700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84c80b2880 0x7f84c80b4d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.208+0000 7f84de59c700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84c80b2880 0x7f84c80b4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.208+0000 7f84cf7fe700 1 -- 172.21.15.59:0/1400618770 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f84d80e1e90 con 0x7f84e0109600 2024-05-08T05:29:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.208+0000 7f84ef77eb80 1 Processor -- start 2024-05-08T05:29:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.208+0000 7f84ef77eb80 1 -- start start 2024-05-08T05:29:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84de59c700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84c80b2880 0x7f84c80b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f84d00098e0 tx=0x7f84d0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1480 0x564c504f1860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f56a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 0x564c504f60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564c50443810 con 0x564c504f1da0 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c503e2770 con 0x564c504f1480 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564c5043f7d0 con 0x564c504f5c30 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ddd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f56a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ddd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f56a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56574/0 (socket says 172.21.15.59:56574) 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ddd9b700 1 -- 172.21.15.59:0/1749845630 learned_addr learned my addr 172.21.15.59:0/1749845630 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84de59c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1480 0x564c504f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ded9d700 1 --2- 172.21.15.59:0/1749845630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 0x564c504f60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ddd9b700 1 -- 172.21.15.59:0/1749845630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 msgr2=0x564c504f60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/1749845630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 0x564c504f60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ddd9b700 1 -- 172.21.15.59:0/1749845630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1480 msgr2=0x564c504f1860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/1749845630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1480 0x564c504f1860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84ddd9b700 1 -- 172.21.15.59:0/1749845630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c50390880 con 0x564c504f1da0 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.209+0000 7f84de59c700 1 --2- 172.21.15.59:0/1749845630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1480 0x564c504f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.210+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/1749845630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f56a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009a80 tx=0x7f84d4009d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8289650e0a6fb17d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.210+0000 7f84ccff9700 1 -- 172.21.15.59:0/1749845630 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4017070 con 0x564c504f1da0 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.210+0000 7f84ccff9700 1 -- 172.21.15.59:0/1749845630 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d400ed80 con 0x564c504f1da0 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.210+0000 7f84ccff9700 1 -- 172.21.15.59:0/1749845630 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4013d70 con 0x564c504f1da0 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.210+0000 7f84ef77eb80 1 -- 172.21.15.59:0/1749845630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 msgr2=0x564c504f56a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.210+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/1749845630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f56a0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009a80 tx=0x7f84d4009d90 comp rx=0 tx=0).stop 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 -- 172.21.15.59:0/1749845630 shutdown_connections 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/1749845630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 0x564c504f60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/1749845630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1480 0x564c504f1860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/1749845630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f56a0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 -- 172.21.15.59:0/1749845630 >> 172.21.15.59:0/1749845630 conn(0x564c5041d0b0 msgr2=0x564c504ebc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:24.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 -- 172.21.15.59:0/1749845630 shutdown_connections 2024-05-08T05:29:24.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 -- 172.21.15.59:0/1749845630 wait complete. 2024-05-08T05:29:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 -- start start 2024-05-08T05:29:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 Processor -- start 2024-05-08T05:29:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1480 0x564c504fcf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1da0 0x564c504fa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 0x564c504fa7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.217 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564c504fde40 con 0x564c504f1480 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c503a9be0 con 0x564c504f1da0 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.211+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564c504fdfc0 con 0x564c504f5c30 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84ef77eb80 10 client.0 _ll_register_callbacks cb 0x564c503db8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84de59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1480 0x564c504fcf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84de59c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1480 0x564c504fcf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56582/0 (socket says 172.21.15.59:56582) 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84de59c700 1 -- 172.21.15.59:0/3776314883 learned_addr learned my addr 172.21.15.59:0/3776314883 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84ded9d700 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 0x564c504fa7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1da0 0x564c504fa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84de59c700 1 -- 172.21.15.59:0/3776314883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 msgr2=0x564c504fa7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84de59c700 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 0x564c504fa7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84de59c700 1 -- 172.21.15.59:0/3776314883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1da0 msgr2=0x564c504fa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84de59c700 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1da0 0x564c504fa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.218 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84de59c700 1 -- 172.21.15.59:0/3776314883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c503df640 con 0x564c504f1480 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84de59c700 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1480 0x564c504fcf10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009990 tx=0x7f84d0009310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99a531b37231a188 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3776314883 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d0030660 con 0x564c504f1480 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3776314883 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f84d0030800 con 0x564c504f1480 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.212+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3776314883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d400ed80 con 0x564c504f1480 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3776314883 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84d003e050 con 0x564c504f1480 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84ef77eb80 10 client.15249 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3776314883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c50508540 con 0x564c504f1480 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3776314883 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d400ed80 con 0x564c504f1480 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3776314883 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f84d000fab0 con 0x564c504f1480 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84ef77eb80 10 client.15249 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84ef77eb80 10 client.15249 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84ef77eb80 20 client.15249 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:24.219 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564c5050ac40 0x564c5050c0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84ef77eb80 4 client.15249 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3776314883 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564c504bd600 con 0x564c5050ac40 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.213+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564c5050ac40 0x564c5050c0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.214+0000 7f84c5ffb700 10 client.15249 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.214+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564c5050ac40 0x564c5050c0b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8834a5064fe6d68 server_cookie=2025afb613cf90e2 in_seq=0 out_seq=0 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3776314883 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564c504bd600 con 0x564c5050ac40 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84c5ffb700 10 client.15249 handle_command_reply: tid=0 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 client.15249 shutdown 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3776314883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1480 msgr2=0x564c504fcf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1480 0x564c504fcf10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f84d0009990 tx=0x7f84d0009310 comp rx=0 tx=0).stop 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3776314883 shutdown_connections 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564c5050ac40 0x564c5050c0b0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f5c30 0x564c504fa7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.220 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1480 0x564c504fcf10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3776314883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f1da0 0x564c504fa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84c5ffb700 0 client.15249 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:29:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3776314883 >> 172.21.15.59:0/3776314883 conn(0x564c5041d0b0 msgr2=0x564c504e66d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3776314883 shutdown_connections 2024-05-08T05:29:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3776314883 wait complete. 2024-05-08T05:29:24.221 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.215+0000 7f84ef77eb80 20 client.15249 trim_cache size 0 max 16384 2024-05-08T05:29:24.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ef77eb80 1 Processor -- start 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ef77eb80 1 -- start start 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f1480 0x564c504f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f34d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 0x564c504f3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564c5050a3f0 con 0x564c504f1da0 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c5050aac0 con 0x564c504f5c30 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564c5050b190 con 0x564c504f1480 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ddd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f34d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ddd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f34d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56598/0 (socket says 172.21.15.59:56598) 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.236+0000 7f84ddd9b700 1 -- 172.21.15.59:0/3890918983 learned_addr learned my addr 172.21.15.59:0/3890918983 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:24.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84ded9d700 1 --2- 172.21.15.59:0/3890918983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 0x564c504f3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84ddd9b700 1 -- 172.21.15.59:0/3890918983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f1480 msgr2=0x564c504f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/3890918983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f1480 0x564c504f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84ddd9b700 1 -- 172.21.15.59:0/3890918983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 msgr2=0x564c504f3a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/3890918983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 0x564c504f3a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84ddd9b700 1 -- 172.21.15.59:0/3890918983 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564c504e87c0 con 0x564c504f1da0 2024-05-08T05:29:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/3890918983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f34d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009a80 tx=0x7f84d4009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a70a4fca903ddc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84c7fff700 1 -- 172.21.15.59:0/3890918983 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4029070 con 0x564c504f1da0 2024-05-08T05:29:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84c7fff700 1 -- 172.21.15.59:0/3890918983 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d400ed80 con 0x564c504f1da0 2024-05-08T05:29:24.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.237+0000 7f84c7fff700 1 -- 172.21.15.59:0/3890918983 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4017040 con 0x564c504f1da0 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3890918983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 msgr2=0x564c504f34d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3890918983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f34d0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009a80 tx=0x7f84d4009190 comp rx=0 tx=0).stop 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3890918983 shutdown_connections 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3890918983 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 0x564c504f3a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3890918983 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f1480 0x564c504f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3890918983 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c504f34d0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3890918983 >> 172.21.15.59:0/3890918983 conn(0x564c5041d0b0 msgr2=0x564c504f8ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3890918983 shutdown_connections 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3890918983 wait complete. 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 -- start start 2024-05-08T05:29:24.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 Processor -- start 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f1480 0x564c50513e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c50514350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 0x564c5050f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564c5050bf30 con 0x564c504f1da0 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564c5050b860 con 0x564c504f5c30 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564c5050c600 con 0x564c504f1480 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.238+0000 7f84ef77eb80 10 client.0 _ll_register_callbacks cb 0x564c503db8d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ddd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c50514350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ddd9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c50514350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56602/0 (socket says 172.21.15.59:56602) 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ddd9b700 1 -- 172.21.15.59:0/3162872918 learned_addr learned my addr 172.21.15.59:0/3162872918 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84de59c700 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f1480 0x564c50513e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ded9d700 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 0x564c5050f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ddd9b700 1 -- 172.21.15.59:0/3162872918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f1480 msgr2=0x564c50513e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:24.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f1480 0x564c50513e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ddd9b700 1 -- 172.21.15.59:0/3162872918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 msgr2=0x564c5050f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 0x564c5050f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ddd9b700 1 -- 172.21.15.59:0/3162872918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564c5050fce0 con 0x564c504f1da0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ded9d700 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 0x564c5050f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ddd9b700 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c50514350 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f84d40137d0 tx=0x7f84d4024400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d10a6fd4c4a747 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84c5ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3162872918 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4029070 con 0x564c504f1da0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3162872918 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f84d40138e0 con 0x564c504f1da0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3162872918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d400ed80 con 0x564c504f1da0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3162872918 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84d403b050 con 0x564c504f1da0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ef77eb80 10 client.15255 fetch_fsmap learned FSMap version 29 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.239+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3162872918 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564c505767b0 con 0x564c504f1da0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.240+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3162872918 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d400ed80 con 0x564c504f1da0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.240+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3162872918 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f84d402d470 con 0x564c504f1da0 2024-05-08T05:29:24.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.240+0000 7f84ef77eb80 10 client.15255 fetch_fsmap finished waiting for FSMap version 29 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.240+0000 7f84ef77eb80 10 client.15255 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.240+0000 7f84ef77eb80 20 client.15255 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.240+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564c5057ab00 0x564c50579810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.240+0000 7f84ef77eb80 4 client.15255 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.240+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3162872918 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "status"}) -- 0x564c504511f0 con 0x564c5057ab00 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.240+0000 7f84de59c700 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564c5057ab00 0x564c50579810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84de59c700 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564c5057ab00 0x564c50579810 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45530a3188399484 server_cookie=a61c4b2d7ea8bb3b in_seq=0 out_seq=0 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84c5ffb700 10 client.15255 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84c5ffb700 1 -- 172.21.15.59:0/3162872918 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564c504511f0 con 0x564c5057ab00 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84c5ffb700 10 client.15255 handle_command_reply: tid=0 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 client.15255 shutdown 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3162872918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 msgr2=0x564c50514350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c50514350 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f84d40137d0 tx=0x7f84d4024400 comp rx=0 tx=0).stop 2024-05-08T05:29:24.246 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3162872918 shutdown_connections 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564c5057ab00 0x564c50579810 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564c504f5c30 0x564c5050f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564c504f1480 0x564c50513e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 --2- 172.21.15.59:0/3162872918 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564c504f1da0 0x564c50514350 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3162872918 >> 172.21.15.59:0/3162872918 conn(0x564c5041d0b0 msgr2=0x564c504f44d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84c5ffb700 0 client.15255 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3162872918 shutdown_connections 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 1 -- 172.21.15.59:0/3162872918 wait complete. 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.241+0000 7f84ef77eb80 20 client.15255 trim_cache size 0 max 16384 2024-05-08T05:29:24.247 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24473, 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 236.57835184300001, 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 29, 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 90, 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 88, 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 252.49735509800001 2024-05-08T05:29:24.248 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:29:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 -- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84c80b2880 msgr2=0x7f84c80b4d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84c80b2880 0x7f84c80b4d40 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f84d00098e0 tx=0x7f84d0005960 comp rx=0 tx=0).stop 2024-05-08T05:29:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 -- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 msgr2=0x7f84e0076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 0x7f84e0076220 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f84d800b9e0 tx=0x7f84d800bcf0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 -- 172.21.15.59:0/1400618770 shutdown_connections 2024-05-08T05:29:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f84e0109600 0x7f84e0076220 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f84e01034f0 0x7f84e00757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f84c80b2880 0x7f84c80b4d40 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 --2- 172.21.15.59:0/1400618770 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f84e0103eb0 0x7f84e0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 -- 172.21.15.59:0/1400618770 >> 172.21.15.59:0/1400618770 conn(0x7f84e007a650 msgr2=0x7f84e0102200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 -- 172.21.15.59:0/1400618770 shutdown_connections 2024-05-08T05:29:24.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:24.242+0000 7f84e4eeb700 1 -- 172.21.15.59:0/1400618770 wait complete. 2024-05-08T05:29:25.001 INFO:teuthology.run_tasks:Running task sequential... 2024-05-08T05:29:25.012 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-05-08T05:29:25.012 INFO:teuthology.task.sequential:In sequential, running task print... 2024-05-08T05:29:25.012 INFO:teuthology.task.print:Enabling mgr modules 2024-05-08T05:29:25.012 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-05-08T05:29:25.013 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T05:29:25.013 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:29:25.014 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-05-08T05:29:25.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 -- 172.21.15.59:0/1132984593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424101910 msgr2=0x7fc424101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:25.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 --2- 172.21.15.59:0/1132984593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424101910 0x7fc424101d30 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fc414009ca0 tx=0x7fc41402a8c0 comp rx=0 tx=0).stop 2024-05-08T05:29:25.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 -- 172.21.15.59:0/1132984593 shutdown_connections 2024-05-08T05:29:25.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 --2- 172.21.15.59:0/1132984593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4241034c0 0x7fc424109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:25.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 --2- 172.21.15.59:0/1132984593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc424102b00 0x7fc424102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:25.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 --2- 172.21.15.59:0/1132984593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424101910 0x7fc424101d30 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:25.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 -- 172.21.15.59:0/1132984593 >> 172.21.15.59:0/1132984593 conn(0x7fc4240fce80 msgr2=0x7fc4240ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:25.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 -- 172.21.15.59:0/1132984593 shutdown_connections 2024-05-08T05:29:25.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 -- 172.21.15.59:0/1132984593 wait complete. 2024-05-08T05:29:25.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.130+0000 7fc429d11640 1 Processor -- start 2024-05-08T05:29:25.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc429d11640 1 -- start start 2024-05-08T05:29:25.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc429d11640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc424101910 0x7fc424198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:25.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc429d11640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424102b00 0x7fc4241990d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:25.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc429d11640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4241034c0 0x7fc4241a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc429d11640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc42410e560 con 0x7fc424102b00 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc429d11640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc42410e3e0 con 0x7fc424101910 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc429d11640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc42410e6e0 con 0x7fc4241034c0 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc422ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424102b00 0x7fc4241990d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc422ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424102b00 0x7fc4241990d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56628/0 (socket says 172.21.15.59:56628) 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc422ffd640 1 -- 172.21.15.59:0/1889185106 learned_addr learned my addr 172.21.15.59:0/1889185106 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc423fff640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4241034c0 0x7fc4241a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc4237fe640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc424101910 0x7fc424198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc422ffd640 1 -- 172.21.15.59:0/1889185106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4241034c0 msgr2=0x7fc4241a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc422ffd640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4241034c0 0x7fc4241a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc422ffd640 1 -- 172.21.15.59:0/1889185106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc424101910 msgr2=0x7fc424198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc422ffd640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc424101910 0x7fc424198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc422ffd640 1 -- 172.21.15.59:0/1889185106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4241a07b0 con 0x7fc424102b00 2024-05-08T05:29:25.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.131+0000 7fc422ffd640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424102b00 0x7fc4241990d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c920 tx=0x7fc40c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:25.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.132+0000 7fc420ff9640 1 -- 172.21.15.59:0/1889185106 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c007c70 con 0x7fc424102b00 2024-05-08T05:29:25.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.132+0000 7fc420ff9640 1 -- 172.21.15.59:0/1889185106 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc40c007e10 con 0x7fc424102b00 2024-05-08T05:29:25.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.132+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4241a0aa0 con 0x7fc424102b00 2024-05-08T05:29:25.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.132+0000 7fc420ff9640 1 -- 172.21.15.59:0/1889185106 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc40c018820 con 0x7fc424102b00 2024-05-08T05:29:25.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.132+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4241a1040 con 0x7fc424102b00 2024-05-08T05:29:25.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.133+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc424108c30 con 0x7fc424102b00 2024-05-08T05:29:25.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.134+0000 7fc420ff9640 1 -- 172.21.15.59:0/1889185106 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 18) ==== 149652+0+0 (secure 0 0 0) 0x7fc40c00f450 con 0x7fc424102b00 2024-05-08T05:29:25.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.135+0000 7fc420ff9640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc4100b2920 0x7fc4100b4de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:25.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.135+0000 7fc4237fe640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc4100b2920 0x7fc4100b4de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:25.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.135+0000 7fc420ff9640 1 -- 172.21.15.59:0/1889185106 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fc40c0e2730 con 0x7fc424102b00 2024-05-08T05:29:25.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.136+0000 7fc4237fe640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc4100b2920 0x7fc4100b4de0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc414000f80 tx=0x7fc4140093f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:25.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.140+0000 7fc420ff9640 1 -- 172.21.15.59:0/1889185106 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fc40c014030 con 0x7fc424102b00 2024-05-08T05:29:25.386 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:25.386+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7fc4240635b0 con 0x7fc424102b00 2024-05-08T05:29:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:25 smithi143 ceph-mon[34337]: pgmap v420: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:25.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:25 smithi096 ceph-mon[34253]: pgmap v420: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:25.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:25 smithi059 ceph-mon[30304]: pgmap v420: 129 pgs: 129 active+clean; 648 KiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-05-08T05:29:26.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.123+0000 7fc420ff9640 1 -- 172.21.15.59:0/1889185106 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7fc40c0aa0c0 con 0x7fc424102b00 2024-05-08T05:29:26.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc4100b2920 msgr2=0x7fc4100b4de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc4100b2920 0x7fc4100b4de0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fc414000f80 tx=0x7fc4140093f0 comp rx=0 tx=0).stop 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424102b00 msgr2=0x7fc4241990d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424102b00 0x7fc4241990d0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7fc40c00c920 tx=0x7fc40c00cde0 comp rx=0 tx=0).stop 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 shutdown_connections 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fc4100b2920 0x7fc4100b4de0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4241034c0 0x7fc4241a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc424102b00 0x7fc4241990d0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 --2- 172.21.15.59:0/1889185106 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc424101910 0x7fc424198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 >> 172.21.15.59:0/1889185106 conn(0x7fc4240fce80 msgr2=0x7fc424105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 shutdown_connections 2024-05-08T05:29:26.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:26.126+0000 7fc429d11640 1 -- 172.21.15.59:0/1889185106 wait complete. 2024-05-08T05:29:26.147 INFO:teuthology.run_tasks:Running task kclient... 2024-05-08T05:29:26.160 INFO:tasks.kclient:Mounting kernel clients... 2024-05-08T05:29:26.160 INFO:tasks.kclient:config is {'client.0': {}} 2024-05-08T05:29:26.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:26.161 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-05-08T05:29:26.181 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-05-08T05:29:26.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:26.181 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-05-08T05:29:26.251 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:26.251 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-05-08T05:29:26.348 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=legacy', 'nowsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-05-08T05:29:26.348 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-05-08T05:29:26.348 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:29:26.348 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs ls 2024-05-08T05:29:26.356 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:26 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1889185106' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T05:29:26.357 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: ignoring --setuser ceph since I am not root 2024-05-08T05:29:26.357 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: ignoring --setgroup ceph since I am not root 2024-05-08T05:29:26.357 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:26.340+0000 7fe8f6835200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:29:26.489 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:26 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1889185106' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T05:29:26.490 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: ignoring --setuser ceph since I am not root 2024-05-08T05:29:26.490 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: ignoring --setgroup ceph since I am not root 2024-05-08T05:29:26.490 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:26 smithi143 ceph-mgr[36197]: -- 172.21.15.143:0/2617037277 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55f0cf3205a0 con 0x55f0cf2f4800 2024-05-08T05:29:26.490 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:26.303+0000 7f15286a5200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:29:26.490 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:26.489+0000 7f15286a5200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:29:26.491 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:26 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1889185106' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-05-08T05:29:26.491 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: ignoring --setuser ceph since I am not root 2024-05-08T05:29:26.491 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: ignoring --setgroup ceph since I am not root 2024-05-08T05:29:26.491 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:26 smithi096 ceph-mgr[35654]: -- 172.21.15.96:0/3913682443 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55da3e02c5a0 con 0x55da3e001000 2024-05-08T05:29:26.491 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:26.307+0000 7f5a40678200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-05-08T05:29:26.491 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:26.490+0000 7f5a40678200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:29:26.618 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:26.552+0000 7fe8f6835200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:29:26.631 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:26.743 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:26.591+0000 7f15286a5200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:29:26.779 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:26.591+0000 7f5a40678200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:29:26.888 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:26.681+0000 7fe8f6835200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-05-08T05:29:27.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:27 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1889185106' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T05:29:27.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:27 smithi143 ceph-mon[34337]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-08T05:29:27.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:26.900+0000 7f15286a5200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:29:27.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:26.986+0000 7f15286a5200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:29:27.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:27 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1889185106' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T05:29:27.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:27 smithi096 ceph-mon[34253]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-08T05:29:27.280 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:26.895+0000 7f5a40678200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:29:27.280 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:26.979+0000 7f5a40678200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:29:27.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:27 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1889185106' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-05-08T05:29:27.407 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:27 smithi059 ceph-mon[30304]: mgrmap e19: x(active, since 11m), standbys: z, y 2024-05-08T05:29:27.407 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:27 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:27.020+0000 7fe8f6835200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-05-08T05:29:27.407 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:27 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:27.120+0000 7fe8f6835200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-05-08T05:29:28.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.397+0000 7f27a64c4700 1 -- 172.21.15.59:0/831958381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a01034f0 msgr2=0x7f27a0103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:28.398 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.397+0000 7f27a64c4700 1 --2- 172.21.15.59:0/831958381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a01034f0 0x7f27a0103970 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2788009990 tx=0x7f2788009ca0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.397+0000 7f27a64c4700 1 -- 172.21.15.59:0/831958381 shutdown_connections 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.397+0000 7f27a64c4700 1 --2- 172.21.15.59:0/831958381 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a0103eb0 0x7f27a0110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.397+0000 7f27a64c4700 1 --2- 172.21.15.59:0/831958381 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a0109600 0x7f27a01099e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.397+0000 7f27a64c4700 1 --2- 172.21.15.59:0/831958381 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a01034f0 0x7f27a0103970 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.397+0000 7f27a64c4700 1 -- 172.21.15.59:0/831958381 >> 172.21.15.59:0/831958381 conn(0x7f27a007a650 msgr2=0x7f27a007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.397+0000 7f27a64c4700 1 -- 172.21.15.59:0/831958381 shutdown_connections 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.397+0000 7f27a64c4700 1 -- 172.21.15.59:0/831958381 wait complete. 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f27a64c4700 1 Processor -- start 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f27a64c4700 1 -- start start 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f27a64c4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a01034f0 0x7f27a00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f27a64c4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a0103eb0 0x7f27a0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f27a64c4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a0109600 0x7f27a0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f27a64c4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f27a0113670 con 0x7f27a0103eb0 2024-05-08T05:29:28.399 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f27a64c4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01134f0 con 0x7f27a01034f0 2024-05-08T05:29:28.400 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f27a64c4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f27a01137f0 con 0x7f27a0109600 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a0103eb0 0x7f27a0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f279f7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a0103eb0 0x7f27a0075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56650/0 (socket says 172.21.15.59:56650) 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f279f7fe700 1 -- 172.21.15.59:0/1563186240 learned_addr learned my addr 172.21.15.59:0/1563186240 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f279ffff700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a01034f0 0x7f27a00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.398+0000 7f27a4a61700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a0109600 0x7f27a0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279ffff700 1 -- 172.21.15.59:0/1563186240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a0109600 msgr2=0x7f27a0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279ffff700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a0109600 0x7f27a0076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279ffff700 1 -- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a0103eb0 msgr2=0x7f27a0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279ffff700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a0103eb0 0x7f27a0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279ffff700 1 -- 172.21.15.59:0/1563186240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27a01a81b0 con 0x7f27a01034f0 2024-05-08T05:29:28.401 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279f7fe700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a0103eb0 0x7f27a0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279ffff700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a01034f0 0x7f27a00757a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f279000e9f0 tx=0x7f279000ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279d7fa700 1 -- 172.21.15.59:0/1563186240 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790004220 con 0x7f27a01034f0 2024-05-08T05:29:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27a01a84a0 con 0x7f27a01034f0 2024-05-08T05:29:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279d7fa700 1 -- 172.21.15.59:0/1563186240 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27900043c0 con 0x7f27a01034f0 2024-05-08T05:29:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f279d7fa700 1 -- 172.21.15.59:0/1563186240 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790005020 con 0x7f27a01034f0 2024-05-08T05:29:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.399+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27a01a89e0 con 0x7f27a01034f0 2024-05-08T05:29:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.400+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27a006a190 con 0x7f27a01034f0 2024-05-08T05:29:28.402 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.402+0000 7f279d7fa700 1 -- 172.21.15.59:0/1563186240 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f2790004750 con 0x7f27a01034f0 2024-05-08T05:29:28.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.403+0000 7f279d7fa700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f278c0b2950 0x7f278c0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:28.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.403+0000 7f279f7fe700 1 -- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f278c0b2950 msgr2=0x7f278c0b4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/546809822 2024-05-08T05:29:28.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.403+0000 7f279d7fa700 1 -- 172.21.15.59:0/1563186240 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f27900e2510 con 0x7f27a01034f0 2024-05-08T05:29:28.403 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.403+0000 7f279f7fe700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f278c0b2950 0x7f278c0b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:29:28.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.406+0000 7f279d7fa700 1 -- 172.21.15.59:0/1563186240 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f27900a9f30 con 0x7f27a01034f0 2024-05-08T05:29:28.485 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:28.148+0000 7f5a40678200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:29:28.485 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:28.315+0000 7f5a40678200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:29:28.485 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:28.484+0000 7f5a40678200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:29:28.493 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:28.172+0000 7f15286a5200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:29:28.493 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:28.341+0000 7f15286a5200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:29:28.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.604+0000 7f279f7fe700 1 -- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f278c0b2950 msgr2=0x7f278c0b4e10 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/546809822 2024-05-08T05:29:28.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.604+0000 7f279f7fe700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f278c0b2950 0x7f278c0b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:29:28.669 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.668+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f27a0076cd0 con 0x7f27a01034f0 2024-05-08T05:29:28.669 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:28.415+0000 7fe8f6835200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-05-08T05:29:28.669 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:28.598+0000 7fe8f6835200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-05-08T05:29:28.670 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.669+0000 7f279d7fa700 1 -- 172.21.15.59:0/1563186240 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v29) ==== 53+0+88 (secure 0 0 0) 0x7f27900a9830 con 0x7f27a01034f0 2024-05-08T05:29:28.670 INFO:teuthology.orchestra.run.smithi059.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-05-08T05:29:28.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f278c0b2950 msgr2=0x7f278c0b4e10 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:29:28.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f278c0b2950 0x7f278c0b4e10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.672 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a01034f0 msgr2=0x7f27a00757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a01034f0 0x7f27a00757a0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f279000e9f0 tx=0x7f279000ed00 comp rx=0 tx=0).stop 2024-05-08T05:29:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 shutdown_connections 2024-05-08T05:29:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f27a0109600 0x7f27a0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f278c0b2950 0x7f278c0b4e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f27a01034f0 0x7f27a00757a0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 --2- 172.21.15.59:0/1563186240 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f27a0103eb0 0x7f27a0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 >> 172.21.15.59:0/1563186240 conn(0x7f27a007a650 msgr2=0x7f27a0102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 shutdown_connections 2024-05-08T05:29:28.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:28.672+0000 7f27a64c4700 1 -- 172.21.15.59:0/1563186240 wait complete. 2024-05-08T05:29:28.771 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:28 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1563186240' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:29:28.802 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:28 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1563186240' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:29:28.803 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:28.513+0000 7f15286a5200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:29:29.029 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:28.770+0000 7f5a40678200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:29:29.030 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:28.861+0000 7f5a40678200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:29:29.044 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:28 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1563186240' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-05-08T05:29:29.044 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:28.782+0000 7fe8f6835200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-05-08T05:29:29.101 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:28.802+0000 7f15286a5200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:29:29.101 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:28.894+0000 7f15286a5200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:29:29.101 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:29.100+0000 7f15286a5200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:29:29.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:29.091+0000 7fe8f6835200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-05-08T05:29:29.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:29.189+0000 7fe8f6835200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-05-08T05:29:29.449 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-05-08T05:29:29.449 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-05-08T05:29:29.449 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi059.front.sepia.ceph.com 2024-05-08T05:29:29.449 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-05-08T05:29:29.449 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-05-08T05:29:29.450 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-05-08T05:29:29.450 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-05-08T05:29:29.450 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-05-08T05:29:29.450 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:29:29.450 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:29.450 DEBUG:teuthology.orchestra.run.smithi059:> ip addr 2024-05-08T05:29:29.493 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:29.292+0000 7f15286a5200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:29:29.507 INFO:teuthology.orchestra.run.smithi059.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-05-08T05:29:29.507 INFO:teuthology.orchestra.run.smithi059.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-05-08T05:29:29.507 INFO:teuthology.orchestra.run.smithi059.stdout: inet 127.0.0.1/8 scope host lo 2024-05-08T05:29:29.507 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:29:29.507 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 ::1/128 scope host 2024-05-08T05:29:29.507 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:29:29.507 INFO:teuthology.orchestra.run.smithi059.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:29:29.507 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:ca brd ff:ff:ff:ff:ff:ff 2024-05-08T05:29:29.507 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f0 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:6b:fe:cb brd ff:ff:ff:ff:ff:ff 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp7s0f1 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2a brd ff:ff:ff:ff:ff:ff 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f0 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: inet 172.21.15.59/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft 41925sec preferred_lft 41925sec 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: inet6 fe80::ec4:7aff:febd:192a/64 scope link noprefixroute 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: valid_lft forever preferred_lft forever 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-05-08T05:29:29.508 INFO:teuthology.orchestra.run.smithi059.stdout: link/ether 0c:c4:7a:bd:19:2b brd ff:ff:ff:ff:ff:ff 2024-05-08T05:29:29.509 INFO:teuthology.orchestra.run.smithi059.stdout: altname enp1s0f1 2024-05-08T05:29:29.509 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-05-08T05:29:29.509 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:29:29.509 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:29:29.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add name ceph-brx type bridge 2024-05-08T05:29:29.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr flush dev ceph-brx 2024-05-08T05:29:29.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set ceph-brx up 2024-05-08T05:29:29.509 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-05-08T05:29:29.509 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:29:29.529 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:29 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:29.061+0000 7f5a40678200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:29:29.529 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:29 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:29.254+0000 7f5a40678200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:29:29.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:29:29.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:29:29.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:29.656 DEBUG:teuthology.orchestra.run.smithi059:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-05-08T05:29:29.679 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:29.403+0000 7fe8f6835200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-05-08T05:29:29.679 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:29.614+0000 7fe8f6835200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-05-08T05:29:29.682 INFO:teuthology.orchestra.run.smithi059.stdout:1 2024-05-08T05:29:29.684 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:29.684 DEBUG:teuthology.orchestra.run.smithi059:> route 2024-05-08T05:29:29.741 INFO:teuthology.orchestra.run.smithi059.stdout:Kernel IP routing table 2024-05-08T05:29:29.741 INFO:teuthology.orchestra.run.smithi059.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-05-08T05:29:29.741 INFO:teuthology.orchestra.run.smithi059.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-05-08T05:29:29.741 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-05-08T05:29:29.741 INFO:teuthology.orchestra.run.smithi059.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-05-08T05:29:29.742 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:29:29.742 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:29:29.742 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-05-08T05:29:29.742 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-05-08T05:29:29.742 DEBUG:teuthology.orchestra.run.smithi059:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-05-08T05:29:29.742 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:29:29.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:29 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:29:29.866 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:29 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:29:29.873 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:29.873 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-05-08T05:29:29.889 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:29.889 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list-id 2024-05-08T05:29:29.945 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:29:29.945 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:29:29.946 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:29:29.946 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-05-08T05:29:29.946 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:29:30.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:29:30.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:29:30.069 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-05-08T05:29:30.069 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:29:30.069 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:29:30.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-05-08T05:29:30.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-05-08T05:29:30.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-05-08T05:29:30.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-05-08T05:29:30.070 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-05-08T05:29:30.070 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:29:30.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:29:30.189 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:29:30.195 DEBUG:teuthology.orchestra.run.smithi059:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-05-08T05:29:30.195 DEBUG:teuthology.orchestra.run.smithi059:> set -e 2024-05-08T05:29:30.195 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set brx.0 up 2024-05-08T05:29:30.196 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link set dev brx.0 master ceph-brx 2024-05-08T05:29:30.196 DEBUG:teuthology.orchestra.run.smithi059:> ') 2024-05-08T05:29:30.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:29.895+0000 7f15286a5200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:29:30.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:30 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:30.116+0000 7f15286a5200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:29:30.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:29:30.271 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:29:30.278 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:29:30.278 DEBUG:teuthology.orchestra.run.smithi059:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-05-08T05:29:30.279 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:29 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:29.852+0000 7f5a40678200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:29:30.279 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:30.071+0000 7f5a40678200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:29:30.294 INFO:teuthology.orchestra.run.smithi059.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-05-08T05:29:30.294 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-05-08T05:29:30.294 DEBUG:teuthology.orchestra.run.smithi059:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:29:30.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:30.280+0000 7fe8f6835200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-05-08T05:29:30.350 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-05-08T05:29:30.350 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-05-08T05:29:30.350 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/5ff8d981-f338-4914-9e21-f8b83e78f038 2024-05-08T05:29:30.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:30.350 DEBUG:teuthology.orchestra.run.smithi059:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/5ff8d981-f338-4914-9e21-f8b83e78f038 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-08T05:29:30.435 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-08T05:29:30.435 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-05-08T05:29:30.435 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-05-08T05:29:30.546 INFO:teuthology.orchestra.run.smithi059.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync 2024-05-08T05:29:30.547 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync". 2024-05-08T05:29:30.547 INFO:teuthology.orchestra.run.smithi059.stdout:invalid new device string format 2024-05-08T05:29:30.547 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: resolved to: "172.21.15.59:6789,172.21.15.96:6789,172.21.15.143:6789" 2024-05-08T05:29:30.547 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.59:6789,172.21.15.96:6789,172.21.15.143:6789:/volumes/qa/sv_1/5ff8d981-f338-4914-9e21-f8b83e78f038 2024-05-08T05:29:30.547 INFO:teuthology.orchestra.run.smithi059.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=legacy,nowsync,key=0,fsid=0bf01a10-0cfa-11ef-bc97-c7b262605968" will pass to kernel 2024-05-08T05:29:30.548 INFO:teuthology.orchestra.run.smithi059.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-05-08T05:29:30.548 INFO:teuthology.orchestra.run.smithi059.stdout: You just mounted a file system that supports labels which does not 2024-05-08T05:29:30.548 INFO:teuthology.orchestra.run.smithi059.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-05-08T05:29:30.548 INFO:teuthology.orchestra.run.smithi059.stdout: applications will generate AVC messages and not be allowed access to 2024-05-08T05:29:30.548 INFO:teuthology.orchestra.run.smithi059.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-05-08T05:29:30.550 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-05-08T05:29:30.550 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:29:30.550 DEBUG:teuthology.orchestra.run.smithi059:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-05-08T05:29:30.580 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:29:30.673 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:29:30.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:29:30.681 INFO:teuthology.orchestra.run.smithi059.stdout:client24964 2024-05-08T05:29:30.681 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:29:30.681 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:29:30.699 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:29:30.779 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:30.493+0000 7f5a40678200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:29:30.779 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:30.591+0000 7f5a40678200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:29:30.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:29:30.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:29:30.799 INFO:teuthology.orchestra.run.smithi059.stdout:client24964 2024-05-08T05:29:30.799 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-05-08T05:29:30.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:30.522+0000 7fe8f6835200 -1 mgr[py] Module smb has missing NOTIFY_TYPES member 2024-05-08T05:29:30.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-05-08T05:29:30.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:29:30.858 INFO:teuthology.orchestra.run.smithi059.stdout:client24964 2024-05-08T05:29:30.858 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968.client24964/status') 2024-05-08T05:29:30.950 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-05-08T05:29:30.960 INFO:teuthology.orchestra.run.smithi059.stdout:instance: client.24964 (3)172.21.15.59:0/327388324 2024-05-08T05:29:30.960 INFO:teuthology.orchestra.run.smithi059.stdout:blocklisted: false 2024-05-08T05:29:30.960 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records in 2024-05-08T05:29:30.960 INFO:teuthology.orchestra.run.smithi059.stderr:0+1 records out 2024-05-08T05:29:30.960 INFO:teuthology.orchestra.run.smithi059.stderr:70 bytes copied, 2.5929e-05 s, 2.7 MB/s 2024-05-08T05:29:30.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-05-08T05:29:30.968 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T05:29:30.981 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T05:29:30.981 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:29:30.981 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate false' 2024-05-08T05:29:30.993 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:30 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:30.539+0000 7f15286a5200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:29:30.993 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:30 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:30.637+0000 7f15286a5200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.088+0000 7fd84a647640 1 -- 172.21.15.59:0/4114523567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844102b00 msgr2=0x7fd844102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.088+0000 7fd84a647640 1 --2- 172.21.15.59:0/4114523567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844102b00 0x7fd844102f80 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fd82c009900 tx=0x7fd82c02a410 comp rx=0 tx=0).stop 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.088+0000 7fd84a647640 1 -- 172.21.15.59:0/4114523567 shutdown_connections 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.088+0000 7fd84a647640 1 --2- 172.21.15.59:0/4114523567 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8441034c0 0x7fd844109d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.088+0000 7fd84a647640 1 --2- 172.21.15.59:0/4114523567 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844102b00 0x7fd844102f80 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.088+0000 7fd84a647640 1 --2- 172.21.15.59:0/4114523567 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd844101910 0x7fd844101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.088+0000 7fd84a647640 1 -- 172.21.15.59:0/4114523567 >> 172.21.15.59:0/4114523567 conn(0x7fd8440fce80 msgr2=0x7fd8440ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.088+0000 7fd84a647640 1 -- 172.21.15.59:0/4114523567 shutdown_connections 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.088+0000 7fd84a647640 1 -- 172.21.15.59:0/4114523567 wait complete. 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd84a647640 1 Processor -- start 2024-05-08T05:29:31.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd84a647640 1 -- start start 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd84a647640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844101910 0x7fd84419adb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd84a647640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd844102b00 0x7fd84419b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd84a647640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8441034c0 0x7fd8441a2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd84a647640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd84410d420 con 0x7fd844101910 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd84a647640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd84410d2a0 con 0x7fd8441034c0 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd84a647640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd84410d5a0 con 0x7fd844102b00 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd843fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844101910 0x7fd84419adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd843fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844101910 0x7fd84419adb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49292/0 (socket says 172.21.15.59:49292) 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd843fff640 1 -- 172.21.15.59:0/2647536235 learned_addr learned my addr 172.21.15.59:0/2647536235 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd83bfff640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd844102b00 0x7fd84419b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd848be4640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8441034c0 0x7fd8441a2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd843fff640 1 -- 172.21.15.59:0/2647536235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd844102b00 msgr2=0x7fd84419b2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:31.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd843fff640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd844102b00 0x7fd84419b2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd843fff640 1 -- 172.21.15.59:0/2647536235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8441034c0 msgr2=0x7fd8441a2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd843fff640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8441034c0 0x7fd8441a2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.089+0000 7fd843fff640 1 -- 172.21.15.59:0/2647536235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd8441a29c0 con 0x7fd844101910 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.090+0000 7fd843fff640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844101910 0x7fd84419adb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd83400e9c0 tx=0x7fd83400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.090+0000 7fd841ffb640 1 -- 172.21.15.59:0/2647536235 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd83400cd80 con 0x7fd844101910 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.090+0000 7fd841ffb640 1 -- 172.21.15.59:0/2647536235 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8340035f0 con 0x7fd844101910 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.090+0000 7fd841ffb640 1 -- 172.21.15.59:0/2647536235 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd834010690 con 0x7fd844101910 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.090+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd8441a2cb0 con 0x7fd844101910 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.090+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd8441a31a0 con 0x7fd844101910 2024-05-08T05:29:31.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.091+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd8441a3430 con 0x7fd844101910 2024-05-08T05:29:31.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.093+0000 7fd841ffb640 1 -- 172.21.15.59:0/2647536235 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7fd8340037a0 con 0x7fd844101910 2024-05-08T05:29:31.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.094+0000 7fd841ffb640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd8300b2970 0x7fd8300b4e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:31.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.094+0000 7fd83bfff640 1 -- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd8300b2970 msgr2=0x7fd8300b4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/546809822 2024-05-08T05:29:31.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.094+0000 7fd841ffb640 1 -- 172.21.15.59:0/2647536235 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7fd8340e29e0 con 0x7fd844101910 2024-05-08T05:29:31.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.095+0000 7fd83bfff640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd8300b2970 0x7fd8300b4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:29:31.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.099+0000 7fd841ffb640 1 -- 172.21.15.59:0/2647536235 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd8340aa310 con 0x7fd844101910 2024-05-08T05:29:31.279 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:30.962+0000 7f5a40678200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:29:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.296+0000 7fd83bfff640 1 -- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd8300b2970 msgr2=0x7fd8300b4e30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/546809822 2024-05-08T05:29:31.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.296+0000 7fd83bfff640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd8300b2970 0x7fd8300b4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:29:31.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.332+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"} v 0) -- 0x7fd8441a3d70 con 0x7fd844101910 2024-05-08T05:29:31.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:30.989+0000 7fe8f6835200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-05-08T05:29:31.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:31.097+0000 7fe8f6835200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-05-08T05:29:31.367 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:31.013+0000 7f15286a5200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:29:31.384 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.384+0000 7fd841ffb640 1 -- 172.21.15.59:0/2647536235 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]=0 v30) ==== 118+0+0 (secure 0 0 0) 0x7fd8340a9c10 con 0x7fd844101910 2024-05-08T05:29:31.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd8300b2970 msgr2=0x7fd8300b4e30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:29:31.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd8300b2970 0x7fd8300b4e30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844101910 msgr2=0x7fd84419adb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:31.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844101910 0x7fd84419adb0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fd83400e9c0 tx=0x7fd83400ee80 comp rx=0 tx=0).stop 2024-05-08T05:29:31.387 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 shutdown_connections 2024-05-08T05:29:31.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7fd8300b2970 0x7fd8300b4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd8441034c0 0x7fd8441a2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd844102b00 0x7fd84419b2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 --2- 172.21.15.59:0/2647536235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd844101910 0x7fd84419adb0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 >> 172.21.15.59:0/2647536235 conn(0x7fd8440fce80 msgr2=0x7fd8440fe930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:31.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 shutdown_connections 2024-05-08T05:29:31.388 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.386+0000 7fd84a647640 1 -- 172.21.15.59:0/2647536235 wait complete. 2024-05-08T05:29:31.399 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs subvolumegroup pin cephfs qa random 0.10' 2024-05-08T05:29:31.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.508+0000 7f041d050640 1 -- 172.21.15.59:0/464079843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0418103490 msgr2=0x7f0418105880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:31.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.508+0000 7f041d050640 1 --2- 172.21.15.59:0/464079843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0418103490 0x7f0418105880 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f0400009900 tx=0x7f040002a410 comp rx=0 tx=0).stop 2024-05-08T05:29:31.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.508+0000 7f041d050640 1 -- 172.21.15.59:0/464079843 shutdown_connections 2024-05-08T05:29:31.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.508+0000 7f041d050640 1 --2- 172.21.15.59:0/464079843 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f04181086f0 0x7f041810ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.508+0000 7f041d050640 1 --2- 172.21.15.59:0/464079843 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0418105dc0 0x7f04181081b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.508+0000 7f041d050640 1 --2- 172.21.15.59:0/464079843 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0418103490 0x7f0418105880 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.508+0000 7f041d050640 1 -- 172.21.15.59:0/464079843 >> 172.21.15.59:0/464079843 conn(0x7f04180fce80 msgr2=0x7f04180ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:31.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.508+0000 7f041d050640 1 -- 172.21.15.59:0/464079843 shutdown_connections 2024-05-08T05:29:31.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.508+0000 7f041d050640 1 -- 172.21.15.59:0/464079843 wait complete. 2024-05-08T05:29:31.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041d050640 1 Processor -- start 2024-05-08T05:29:31.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041d050640 1 -- start start 2024-05-08T05:29:31.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041d050640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0418103490 0x7f0418198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041d050640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0418105dc0 0x7f04181991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041d050640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04181086f0 0x7f04181a01c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041d050640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f041810e570 con 0x7f04181086f0 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041d050640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f041810e3f0 con 0x7f0418103490 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041d050640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f041810e6f0 con 0x7f0418105dc0 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041759e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04181086f0 0x7f04181a01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0418105dc0 0x7f04181991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f0416d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0418103490 0x7f0418198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0418105dc0 0x7f04181991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:38640/0 (socket says 172.21.15.59:38640) 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.509+0000 7f041659c640 1 -- 172.21.15.59:0/1585234235 learned_addr learned my addr 172.21.15.59:0/1585234235 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f041659c640 1 -- 172.21.15.59:0/1585234235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0418103490 msgr2=0x7f0418198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f041659c640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0418103490 0x7f0418198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f041659c640 1 -- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04181086f0 msgr2=0x7f04181a01c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f041659c640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f04181086f0 0x7f04181a01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:31.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f041659c640 1 -- 172.21.15.59:0/1585234235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181a0860 con 0x7f0418105dc0 2024-05-08T05:29:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f041659c640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0418105dc0 0x7f04181991e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f040800e9f0 tx=0x7f040800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040800ccc0 con 0x7f0418105dc0 2024-05-08T05:29:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f04080054d0 con 0x7f0418105dc0 2024-05-08T05:29:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181a0b50 con 0x7f0418105dc0 2024-05-08T05:29:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0408010430 con 0x7f0418105dc0 2024-05-08T05:29:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.510+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181a1050 con 0x7f0418105dc0 2024-05-08T05:29:31.513 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.511+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f04180fe970 con 0x7f0418105dc0 2024-05-08T05:29:31.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.512+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 19) ==== 149669+0+0 (secure 0 0 0) 0x7f04080046c0 con 0x7f0418105dc0 2024-05-08T05:29:31.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.513+0000 7f040ffff640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 0x7f04040b4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:31.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.513+0000 7f0416d9d640 1 -- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 msgr2=0x7f04040b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/546809822 2024-05-08T05:29:31.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.514+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9150+0+0 (secure 0 0 0) 0x7f04080e2b20 con 0x7f0418105dc0 2024-05-08T05:29:31.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.514+0000 7f0416d9d640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 0x7f04040b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-05-08T05:29:31.517 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.517+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f04080aa450 con 0x7f0418105dc0 2024-05-08T05:29:31.656 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:31 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2647536235' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T05:29:31.656 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:31.319+0000 7f5a40678200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:29:31.656 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:31.411+0000 7f5a40678200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:29:31.709 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:31 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2647536235' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T05:29:31.709 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:31.367+0000 7f15286a5200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:29:31.709 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:31.462+0000 7f15286a5200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:29:31.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.715+0000 7f0416d9d640 1 -- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 msgr2=0x7f04040b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/546809822 2024-05-08T05:29:31.715 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.715+0000 7f0416d9d640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 0x7f04040b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-05-08T05:29:31.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:31.743+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 --> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f0418062e90 con 0x7f04040b28f0 2024-05-08T05:29:31.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:31 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2647536235' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]: dispatch 2024-05-08T05:29:31.836 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:31.495+0000 7fe8f6835200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-05-08T05:29:31.993 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:31.708+0000 7f15286a5200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:29:31.993 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:31.843+0000 7f15286a5200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:29:31.993 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:31.939+0000 7f15286a5200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:29:32.029 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:31.655+0000 7f5a40678200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:29:32.029 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:31.792+0000 7f5a40678200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:29:32.029 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:31.888+0000 7f5a40678200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:29:32.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:32.116+0000 7f0416d9d640 1 -- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 msgr2=0x7f04040b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/546809822 2024-05-08T05:29:32.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:32.116+0000 7f0416d9d640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 0x7f04040b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-05-08T05:29:32.242 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:31.881+0000 7fe8f6835200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-05-08T05:29:32.242 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:31.980+0000 7fe8f6835200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-05-08T05:29:32.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:32 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2647536235' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T05:29:32.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:32 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:29:32.530 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:32.214+0000 7f5a40678200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:29:32.530 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:32.306+0000 7f5a40678200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:29:32.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:32 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2647536235' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T05:29:32.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:32 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:29:32.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:32.242+0000 7fe8f6835200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-05-08T05:29:32.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:32.385+0000 7fe8f6835200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-05-08T05:29:32.586 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:32.489+0000 7fe8f6835200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-05-08T05:29:32.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:32 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2647536235' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "false"}]': finished 2024-05-08T05:29:32.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:32 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:29:32.743 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:32 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:32.270+0000 7f15286a5200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:29:32.743 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:32 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:32.362+0000 7f15286a5200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:29:32.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:32.917+0000 7f0416d9d640 1 -- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 msgr2=0x7f04040b4db0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.59:6800/546809822 2024-05-08T05:29:32.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:32.917+0000 7f0416d9d640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 0x7f04040b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-05-08T05:29:33.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:32.831+0000 7fe8f6835200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-05-08T05:29:33.086 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:32.930+0000 7fe8f6835200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-05-08T05:29:33.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:32 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:32.967+0000 7f15286a5200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:29:33.243 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:29:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z[36190]: 2024-05-08T05:29:33.185+0000 7f15286a5200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:29:33.280 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:32.908+0000 7f5a40678200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:29:33.280 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:29:33 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y[35650]: 2024-05-08T05:29:33.124+0000 7f5a40678200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:29:33.404 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:33.403+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mgrmap(e 20) ==== 149669+0+0 (secure 0 0 0) 0x7f04080a5f50 con 0x7f0418105dc0 2024-05-08T05:29:33.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: Standby manager daemon y restarted 2024-05-08T05:29:33.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: Standby manager daemon y started 2024-05-08T05:29:33.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:29:33.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:29:33.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:29:33.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:29:33.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: Standby manager daemon z restarted 2024-05-08T05:29:33.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: Standby manager daemon z started 2024-05-08T05:29:33.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:29:33.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:29:33.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:29:33.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:33 smithi143 ceph-mon[34337]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:29:33.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: Standby manager daemon y restarted 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: Standby manager daemon y started 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: Standby manager daemon z restarted 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: Standby manager daemon z started 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:29:33.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:33 smithi096 ceph-mon[34253]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:29:33.816 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:33.577+0000 7fe8f6835200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-05-08T05:29:33.816 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:33.815+0000 7fe8f6835200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-05-08T05:29:33.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: Standby manager daemon y restarted 2024-05-08T05:29:33.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: Standby manager daemon y started 2024-05-08T05:29:33.816 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-05-08T05:29:33.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:29:33.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-05-08T05:29:33.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.96:0/861613' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:29:33.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: Standby manager daemon z restarted 2024-05-08T05:29:33.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: Standby manager daemon z started 2024-05-08T05:29:33.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-05-08T05:29:33.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-05-08T05:29:33.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-05-08T05:29:33.817 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:33 smithi059 ceph-mon[30304]: from='mgr.? 172.21.15.143:0/2803589113' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-05-08T05:29:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:33.832+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mon.1 v2:172.21.15.143:3300/0 8 ==== mgrmap(e 21) ==== 149337+0+0 (secure 0 0 0) 0x7f04080e02b0 con 0x7f0418105dc0 2024-05-08T05:29:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:33.832+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 msgr2=0x7f04040b4db0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:29:33.832 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:33.832+0000 7f040ffff640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/546809822,v1:172.21.15.59:6801/546809822] conn(0x7f04040b28f0 0x7f04040b4db0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:34.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe85506f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe85506f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe85506f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe85506f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe85506f700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe857874700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe857874700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe857874700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe857874700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.337 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.079+0000 7fe857874700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: Active manager daemon x restarted 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: Activating manager daemon x 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: mgrmap e21: x(active, starting, since 0.0128011s), standbys: y, z 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:29:34.744 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:29:34.745 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: Manager daemon x is now available 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:29:34.746 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:34 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:29:34.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-05-08T05:29:34.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: Active manager daemon x restarted 2024-05-08T05:29:34.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: Activating manager daemon x 2024-05-08T05:29:34.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T05:29:34.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: mgrmap e21: x(active, starting, since 0.0128011s), standbys: y, z 2024-05-08T05:29:34.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:29:34.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:29:34.781 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:29:34.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:29:34.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: Manager daemon x is now available 2024-05-08T05:29:34.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:29:34.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:29:34.783 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:34 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:29:34.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: mgrmap e20: x(active, since 11m), standbys: y, z 2024-05-08T05:29:34.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: Active manager daemon x restarted 2024-05-08T05:29:34.837 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: Activating manager daemon x 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: osdmap e91: 12 total, 12 up, 12 in 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: mgrmap e21: x(active, starting, since 0.0128011s), standbys: y, z 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:29:34.838 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-05-08T05:29:34.839 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: Manager daemon x is now available 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-05-08T05:29:34.840 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:34 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-05-08T05:29:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.838+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mon.1 v2:172.21.15.143:3300/0 9 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f0408062590 con 0x7f0418105dc0 2024-05-08T05:29:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.839+0000 7f040ffff640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0404039d80 0x7f040403a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.839+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}) -- 0x7f0418062e90 con 0x7f0404039d80 2024-05-08T05:29:34.841 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.840+0000 7f0416d9d640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0404039d80 0x7f040403a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:34.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.842+0000 7f0416d9d640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0404039d80 0x7f040403a160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0400009230 tx=0x7f04000314a0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:34.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.862+0000 7f040ffff640 1 -- 172.21.15.59:0/1585234235 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+3 (secure 0 0 0) 0x7f0418062e90 con 0x7f0404039d80 2024-05-08T05:29:34.862 INFO:teuthology.orchestra.run.smithi059.stdout:{} 2024-05-08T05:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.864+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0404039d80 msgr2=0x7f040403a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.864+0000 7f041d050640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0404039d80 0x7f040403a160 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f0400009230 tx=0x7f04000314a0 comp rx=0 tx=0).stop 2024-05-08T05:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.864+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0418105dc0 msgr2=0x7f04181991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.864+0000 7f041d050640 1 --2- 172.21.15.59:0/1585234235 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0418105dc0 0x7f04181991e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f040800e9f0 tx=0x7f040800eeb0 comp rx=0 tx=0).stop 2024-05-08T05:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.864+0000 7f041759e640 1 -- 172.21.15.59:0/1585234235 reap_dead start 2024-05-08T05:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.864+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 shutdown_connections 2024-05-08T05:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.864+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 >> 172.21.15.59:0/1585234235 conn(0x7f04180fce80 msgr2=0x7f0418103d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.864+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 shutdown_connections 2024-05-08T05:29:34.865 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:34.865+0000 7f041d050640 1 -- 172.21.15.59:0/1585234235 wait complete. 2024-05-08T05:29:34.878 INFO:teuthology.run_tasks:Running task check-counter... 2024-05-08T05:29:34.890 INFO:teuthology.run_tasks:Running task exec... 2024-05-08T05:29:34.961 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T05:29:34.961 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:29:34.961 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/log_level debug' 2024-05-08T05:29:35.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 -- 172.21.15.59:0/1766484566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e081034c0 msgr2=0x7f2e08109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/1766484566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e081034c0 0x7f2e08109d50 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2df8009900 tx=0x7f2df802a410 comp rx=0 tx=0).stop 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 -- 172.21.15.59:0/1766484566 shutdown_connections 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/1766484566 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e081034c0 0x7f2e08109d50 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/1766484566 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e08102b00 0x7f2e08102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/1766484566 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e08101910 0x7f2e08101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 -- 172.21.15.59:0/1766484566 >> 172.21.15.59:0/1766484566 conn(0x7f2e080fce80 msgr2=0x7f2e080ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 -- 172.21.15.59:0/1766484566 shutdown_connections 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 -- 172.21.15.59:0/1766484566 wait complete. 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 Processor -- start 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.068+0000 7f2e0fb38640 1 -- start start 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0fb38640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e08101910 0x7f2e08198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0fb38640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e08102b00 0x7f2e081990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0fb38640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e081034c0 0x7f2e081a0110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0fb38640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810e560 con 0x7f2e08102b00 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0fb38640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810e3e0 con 0x7f2e08101910 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0fb38640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2e0810e6e0 con 0x7f2e081034c0 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d0d3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e08102b00 0x7f2e081990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d0d3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e08102b00 0x7f2e081990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49468/0 (socket says 172.21.15.59:49468) 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d0d3640 1 -- 172.21.15.59:0/142826838 learned_addr learned my addr 172.21.15.59:0/142826838 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0e0d5640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e081034c0 0x7f2e081a0110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d8d4640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e08101910 0x7f2e08198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d0d3640 1 -- 172.21.15.59:0/142826838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e081034c0 msgr2=0x7f2e081a0110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d0d3640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e081034c0 0x7f2e081a0110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d0d3640 1 -- 172.21.15.59:0/142826838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e08101910 msgr2=0x7f2e08198b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d0d3640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e08101910 0x7f2e08198b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d0d3640 1 -- 172.21.15.59:0/142826838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e081a0780 con 0x7f2e08102b00 2024-05-08T05:29:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0e0d5640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e081034c0 0x7f2e081a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0d0d3640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e08102b00 0x7f2e081990a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00eb90 tx=0x7f2dfc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e06ffd640 1 -- 172.21.15.59:0/142826838 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc004d20 con 0x7f2e08102b00 2024-05-08T05:29:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e06ffd640 1 -- 172.21.15.59:0/142826838 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2dfc00ee90 con 0x7f2e08102b00 2024-05-08T05:29:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e081a0a70 con 0x7f2e08102b00 2024-05-08T05:29:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.069+0000 7f2e06ffd640 1 -- 172.21.15.59:0/142826838 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2dfc010670 con 0x7f2e08102b00 2024-05-08T05:29:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.070+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e081a0fc0 con 0x7f2e08102b00 2024-05-08T05:29:35.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.070+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e08108c30 con 0x7f2e08102b00 2024-05-08T05:29:35.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.072+0000 7f2e06ffd640 1 -- 172.21.15.59:0/142826838 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f2dfc005020 con 0x7f2e08102b00 2024-05-08T05:29:35.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.072+0000 7f2e06ffd640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2df00b2710 0x7f2df00b4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.073+0000 7f2e0d8d4640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2df00b2710 0x7f2df00b4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.073+0000 7f2e06ffd640 1 -- 172.21.15.59:0/142826838 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f2dfc0e29c0 con 0x7f2e08102b00 2024-05-08T05:29:35.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.073+0000 7f2e0d8d4640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2df00b2710 0x7f2df00b4bd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e00005fd0 tx=0x7f2e00005850 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:35.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.077+0000 7f2e06ffd640 1 -- 172.21.15.59:0/142826838 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2dfc0aa540 con 0x7f2e08102b00 2024-05-08T05:29:35.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.849+0000 7fe852069700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:35.154 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.849+0000 7fe852069700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:35.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.849+0000 7fe852069700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:35.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.849+0000 7fe852069700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:35.155 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:29:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:29:34.849+0000 7fe852069700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:29:35.279 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.278+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f2e0802db50 con 0x7f2e08102b00 2024-05-08T05:29:35.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.282+0000 7f2e06ffd640 1 -- 172.21.15.59:0/142826838 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v28)=0 v28) ==== 129+0+0 (secure 0 0 0) 0x7f2dfc0a9e40 con 0x7f2e08102b00 2024-05-08T05:29:35.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2df00b2710 msgr2=0x7f2df00b4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2df00b2710 0x7f2df00b4bd0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2e00005fd0 tx=0x7f2e00005850 comp rx=0 tx=0).stop 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e08102b00 msgr2=0x7f2e081990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e08102b00 0x7f2e081990a0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2dfc00eb90 tx=0x7f2dfc00c6a0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 shutdown_connections 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2df00b2710 0x7f2df00b4bd0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e081034c0 0x7f2e081a0110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e08102b00 0x7f2e081990a0 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 --2- 172.21.15.59:0/142826838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e08101910 0x7f2e08198b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 >> 172.21.15.59:0/142826838 conn(0x7f2e080fce80 msgr2=0x7f2e08105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 shutdown_connections 2024-05-08T05:29:35.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.285+0000 7f2e0fb38640 1 -- 172.21.15.59:0/142826838 wait complete. 2024-05-08T05:29:35.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-05-08T05:29:35.407 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.404+0000 7fb82c472640 1 -- 172.21.15.59:0/3297454537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 msgr2=0x7fb824109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.404+0000 7fb82c472640 1 --2- 172.21.15.59:0/3297454537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 0x7fb824109d50 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fb81c009ca0 tx=0x7fb81c02a8c0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 -- 172.21.15.59:0/3297454537 shutdown_connections 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 --2- 172.21.15.59:0/3297454537 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 0x7fb824109d50 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 --2- 172.21.15.59:0/3297454537 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb824102b00 0x7fb824102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 --2- 172.21.15.59:0/3297454537 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824101910 0x7fb824101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 -- 172.21.15.59:0/3297454537 >> 172.21.15.59:0/3297454537 conn(0x7fb8240fce80 msgr2=0x7fb8240ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 -- 172.21.15.59:0/3297454537 shutdown_connections 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 -- 172.21.15.59:0/3297454537 wait complete. 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 Processor -- start 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 -- start start 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb824101910 0x7fb824105920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.408 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824102b00 0x7fb824103fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 0x7fb8241044f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb82410d420 con 0x7fb8241034c0 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb82410d2a0 con 0x7fb824102b00 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82c472640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb82410d5a0 con 0x7fb824101910 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82aa0f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 0x7fb8241044f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82aa0f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 0x7fb8241044f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49478/0 (socket says 172.21.15.59:49478) 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.405+0000 7fb82aa0f640 1 -- 172.21.15.59:0/1259591823 learned_addr learned my addr 172.21.15.59:0/1259591823 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb829a0d640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824102b00 0x7fb824103fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb82a20e640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb824101910 0x7fb824105920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb82aa0f640 1 -- 172.21.15.59:0/1259591823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb824101910 msgr2=0x7fb824105920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb82aa0f640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb824101910 0x7fb824105920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb82aa0f640 1 -- 172.21.15.59:0/1259591823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824102b00 msgr2=0x7fb824103fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb82aa0f640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824102b00 0x7fb824103fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.409 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb82aa0f640 1 -- 172.21.15.59:0/1259591823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb824104db0 con 0x7fb8241034c0 2024-05-08T05:29:35.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb829a0d640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824102b00 0x7fb824103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:35.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb82aa0f640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 0x7fb8241044f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0097b0 tx=0x7fb81c031840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:35.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb8177fe640 1 -- 172.21.15.59:0/1259591823 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c003bf0 con 0x7fb8241034c0 2024-05-08T05:29:35.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb8177fe640 1 -- 172.21.15.59:0/1259591823 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb81c003d90 con 0x7fb8241034c0 2024-05-08T05:29:35.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8241a2c60 con 0x7fb8241034c0 2024-05-08T05:29:35.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb8177fe640 1 -- 172.21.15.59:0/1259591823 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb81c02fc70 con 0x7fb8241034c0 2024-05-08T05:29:35.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.406+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8241a3180 con 0x7fb8241034c0 2024-05-08T05:29:35.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.407+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb824050740 con 0x7fb8241034c0 2024-05-08T05:29:35.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.408+0000 7fb8177fe640 1 -- 172.21.15.59:0/1259591823 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fb81c004b40 con 0x7fb8241034c0 2024-05-08T05:29:35.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.409+0000 7fb8177fe640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb8100b2690 0x7fb8100b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.409+0000 7fb82a20e640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb8100b2690 0x7fb8100b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.409+0000 7fb8177fe640 1 -- 172.21.15.59:0/1259591823 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fb81c103930 con 0x7fb8241034c0 2024-05-08T05:29:35.411 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.410+0000 7fb82a20e640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb8100b2690 0x7fb8100b4b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb824102960 tx=0x7fb820005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:35.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.413+0000 7fb8177fe640 1 -- 172.21.15.59:0/1259591823 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb81c0cb4b0 con 0x7fb8241034c0 2024-05-08T05:29:35.606 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.606+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fb8241a3460 con 0x7fb8241034c0 2024-05-08T05:29:35.610 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.610+0000 7fb8177fe640 1 -- 172.21.15.59:0/1259591823 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7fb81c0cadb0 con 0x7fb8241034c0 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb8100b2690 msgr2=0x7fb8100b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb8100b2690 0x7fb8100b4b50 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fb824102960 tx=0x7fb820005f90 comp rx=0 tx=0).stop 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 msgr2=0x7fb8241044f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 0x7fb8241044f0 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fb81c0097b0 tx=0x7fb81c031840 comp rx=0 tx=0).stop 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 shutdown_connections 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb8100b2690 0x7fb8100b4b50 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb8241034c0 0x7fb8241044f0 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb824102b00 0x7fb824103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 --2- 172.21.15.59:0/1259591823 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb824101910 0x7fb824105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 >> 172.21.15.59:0/1259591823 conn(0x7fb8240fce80 msgr2=0x7fb82406a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.612+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 shutdown_connections 2024-05-08T05:29:35.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.613+0000 7fb82c472640 1 -- 172.21.15.59:0/1259591823 wait complete. 2024-05-08T05:29:35.624 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-05-08T05:29:35.734 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 -- 172.21.15.59:0/3208264690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0564103570 msgr2=0x7f0564109e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 --2- 172.21.15.59:0/3208264690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0564103570 0x7f0564109e00 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f055c009ca0 tx=0x7f055c02a8c0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 -- 172.21.15.59:0/3208264690 shutdown_connections 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 --2- 172.21.15.59:0/3208264690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0564103570 0x7f0564109e00 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 --2- 172.21.15.59:0/3208264690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0564102bb0 0x7f0564103030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 --2- 172.21.15.59:0/3208264690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f05641019c0 0x7f0564101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 -- 172.21.15.59:0/3208264690 >> 172.21.15.59:0/3208264690 conn(0x7f05640fcf30 msgr2=0x7f05640ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 -- 172.21.15.59:0/3208264690 shutdown_connections 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 -- 172.21.15.59:0/3208264690 wait complete. 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 Processor -- start 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.731+0000 7f056abaf640 1 -- start start 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056abaf640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05641019c0 0x7f0564198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.735 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056abaf640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0564102bb0 0x7f0564199080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056abaf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0564103570 0x7f05641a00f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056abaf640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f056410d4d0 con 0x7f05641019c0 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056abaf640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f056410d350 con 0x7f0564103570 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056abaf640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f056410d650 con 0x7f0564102bb0 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056894b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05641019c0 0x7f0564198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056894b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05641019c0 0x7f0564198b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49506/0 (socket says 172.21.15.59:49506) 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056894b640 1 -- 172.21.15.59:0/2616376350 learned_addr learned my addr 172.21.15.59:0/2616376350 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f0563fff640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0564102bb0 0x7f0564199080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056914c640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0564103570 0x7f05641a00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056894b640 1 -- 172.21.15.59:0/2616376350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0564102bb0 msgr2=0x7f0564199080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056894b640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0564102bb0 0x7f0564199080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056894b640 1 -- 172.21.15.59:0/2616376350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0564103570 msgr2=0x7f05641a00f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056894b640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0564103570 0x7f05641a00f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056894b640 1 -- 172.21.15.59:0/2616376350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05641a0760 con 0x7f05641019c0 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056914c640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0564103570 0x7f05641a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056894b640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05641019c0 0x7f0564198b40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f055400e9c0 tx=0x7f055400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f0561ffb640 1 -- 172.21.15.59:0/2616376350 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f055400cd80 con 0x7f05641019c0 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f0561ffb640 1 -- 172.21.15.59:0/2616376350 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f05540035f0 con 0x7f05641019c0 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05641a0a50 con 0x7f05641019c0 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.732+0000 7f0561ffb640 1 -- 172.21.15.59:0/2616376350 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0554010690 con 0x7f05641019c0 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.733+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05641a0f50 con 0x7f05641019c0 2024-05-08T05:29:35.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.734+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f05641a1180 con 0x7f05641019c0 2024-05-08T05:29:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.735+0000 7f0561ffb640 1 -- 172.21.15.59:0/2616376350 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f0554004000 con 0x7f05641019c0 2024-05-08T05:29:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.736+0000 7f0561ffb640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f054c0b2690 0x7f054c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:35.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.736+0000 7f0563fff640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f054c0b2690 0x7f054c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.736+0000 7f0561ffb640 1 -- 172.21.15.59:0/2616376350 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f05540e2860 con 0x7f05641019c0 2024-05-08T05:29:35.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.737+0000 7f0563fff640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f054c0b2690 0x7f054c0b4b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f056419a050 tx=0x7f0558005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:35.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.742+0000 7f0561ffb640 1 -- 172.21.15.59:0/2616376350 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f05540aa3e0 con 0x7f05641019c0 2024-05-08T05:29:35.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.955+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f05641a1660 con 0x7f05641019c0 2024-05-08T05:29:35.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.961+0000 7f0561ffb640 1 -- 172.21.15.59:0/2616376350 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7f05540a9ce0 con 0x7f05641019c0 2024-05-08T05:29:35.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.963+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f054c0b2690 msgr2=0x7f054c0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.963+0000 7f056abaf640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f054c0b2690 0x7f054c0b4b50 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f056419a050 tx=0x7f0558005f90 comp rx=0 tx=0).stop 2024-05-08T05:29:35.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.963+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05641019c0 msgr2=0x7f0564198b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:35.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.963+0000 7f056abaf640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05641019c0 0x7f0564198b40 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f055400e9c0 tx=0x7f055400ee80 comp rx=0 tx=0).stop 2024-05-08T05:29:35.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.964+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 shutdown_connections 2024-05-08T05:29:35.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.964+0000 7f056abaf640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f054c0b2690 0x7f054c0b4b50 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.964+0000 7f056abaf640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0564103570 0x7f05641a00f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.964+0000 7f056abaf640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0564102bb0 0x7f0564199080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.964+0000 7f056abaf640 1 --2- 172.21.15.59:0/2616376350 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f05641019c0 0x7f0564198b40 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.964+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 >> 172.21.15.59:0/2616376350 conn(0x7f05640fcf30 msgr2=0x7f05640fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.964+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 shutdown_connections 2024-05-08T05:29:35.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:35.964+0000 7f056abaf640 1 -- 172.21.15.59:0/2616376350 wait complete. 2024-05-08T05:29:35.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-05-08T05:29:36.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:35 smithi059 ceph-mon[30304]: [08/May/2024:05:29:34] ENGINE Bus STARTING 2024-05-08T05:29:36.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:35 smithi059 ceph-mon[30304]: [08/May/2024:05:29:34] ENGINE Serving on http://172.21.15.59:8765 2024-05-08T05:29:36.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:35 smithi059 ceph-mon[30304]: [08/May/2024:05:29:34] ENGINE Serving on https://172.21.15.59:7150 2024-05-08T05:29:36.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:35 smithi059 ceph-mon[30304]: [08/May/2024:05:29:34] ENGINE Bus STARTED 2024-05-08T05:29:36.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:35 smithi059 ceph-mon[30304]: mgrmap e22: x(active, since 1.01948s), standbys: y, z 2024-05-08T05:29:36.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:35 smithi059 ceph-mon[30304]: from='client.24976 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:36.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:35 smithi059 ceph-mon[30304]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:29:36.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:35 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/142826838' entity='client.admin' 2024-05-08T05:29:36.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:35 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1259591823' entity='client.admin' 2024-05-08T05:29:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.098+0000 7f48a5fa4640 1 -- 172.21.15.59:0/936409579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a01034c0 msgr2=0x7f48a0109d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.098+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/936409579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a01034c0 0x7f48a0109d50 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4898009ca0 tx=0x7f489802a8c0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.098+0000 7f48a5fa4640 1 -- 172.21.15.59:0/936409579 shutdown_connections 2024-05-08T05:29:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.098+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/936409579 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a01034c0 0x7f48a0109d50 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.098+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/936409579 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48a0102b00 0x7f48a0102f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.098+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/936409579 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a0101910 0x7f48a0101d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.098+0000 7f48a5fa4640 1 -- 172.21.15.59:0/936409579 >> 172.21.15.59:0/936409579 conn(0x7f48a00fce80 msgr2=0x7f48a00ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.098+0000 7f48a5fa4640 1 -- 172.21.15.59:0/936409579 shutdown_connections 2024-05-08T05:29:36.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.098+0000 7f48a5fa4640 1 -- 172.21.15.59:0/936409579 wait complete. 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f48a5fa4640 1 Processor -- start 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f48a5fa4640 1 -- start start 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f48a5fa4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48a0101910 0x7f48a0197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f48a5fa4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0102b00 0x7f48a0198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f48a5fa4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a01034c0 0x7f48a019f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f48a5fa4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f48a010d440 con 0x7f48a0102b00 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f48a5fa4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f48a010d2c0 con 0x7f48a0101910 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f48a5fa4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f48a010d5c0 con 0x7f48a01034c0 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f489effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0102b00 0x7f48a0198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f489f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48a0101910 0x7f48a0197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f489effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0102b00 0x7f48a0198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49546/0 (socket says 172.21.15.59:49546) 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f489effd640 1 -- 172.21.15.59:0/1865699025 learned_addr learned my addr 172.21.15.59:0/1865699025 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.099+0000 7f489ffff640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a01034c0 0x7f48a019f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f489effd640 1 -- 172.21.15.59:0/1865699025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a01034c0 msgr2=0x7f48a019f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f489effd640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a01034c0 0x7f48a019f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f489effd640 1 -- 172.21.15.59:0/1865699025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48a0101910 msgr2=0x7f48a0197ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f489effd640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48a0101910 0x7f48a0197ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f489effd640 1 -- 172.21.15.59:0/1865699025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48a019f6b0 con 0x7f48a0102b00 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f489effd640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0102b00 0x7f48a0198020 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f489400e9f0 tx=0x7f489400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f489cff9640 1 -- 172.21.15.59:0/1865699025 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f489400cd80 con 0x7f48a0102b00 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48a006a0c0 con 0x7f48a0102b00 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f489cff9640 1 -- 172.21.15.59:0/1865699025 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48940108e0 con 0x7f48a0102b00 2024-05-08T05:29:36.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f489cff9640 1 -- 172.21.15.59:0/1865699025 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4894018980 con 0x7f48a0102b00 2024-05-08T05:29:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.100+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48a006a610 con 0x7f48a0102b00 2024-05-08T05:29:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.101+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48a0050740 con 0x7f48a0102b00 2024-05-08T05:29:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.102+0000 7f489cff9640 1 -- 172.21.15.59:0/1865699025 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7f4894010430 con 0x7f48a0102b00 2024-05-08T05:29:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.103+0000 7f489cff9640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f48880b2690 0x7f48880b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.103+0000 7f489cff9640 1 -- 172.21.15.59:0/1865699025 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f48940e25c0 con 0x7f48a0102b00 2024-05-08T05:29:36.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.103+0000 7f489f7fe640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f48880b2690 0x7f48880b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.104+0000 7f489f7fe640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f48880b2690 0x7f48880b4b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f48a0102960 tx=0x7f4890005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:36.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.108+0000 7f489cff9640 1 -- 172.21.15.59:0/1865699025 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f48940a9af0 con 0x7f48a0102b00 2024-05-08T05:29:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:35 smithi143 ceph-mon[34337]: [08/May/2024:05:29:34] ENGINE Bus STARTING 2024-05-08T05:29:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:35 smithi143 ceph-mon[34337]: [08/May/2024:05:29:34] ENGINE Serving on http://172.21.15.59:8765 2024-05-08T05:29:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:35 smithi143 ceph-mon[34337]: [08/May/2024:05:29:34] ENGINE Serving on https://172.21.15.59:7150 2024-05-08T05:29:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:35 smithi143 ceph-mon[34337]: [08/May/2024:05:29:34] ENGINE Bus STARTED 2024-05-08T05:29:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:35 smithi143 ceph-mon[34337]: mgrmap e22: x(active, since 1.01948s), standbys: y, z 2024-05-08T05:29:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:35 smithi143 ceph-mon[34337]: from='client.24976 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:35 smithi143 ceph-mon[34337]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:29:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:35 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/142826838' entity='client.admin' 2024-05-08T05:29:36.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:35 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1259591823' entity='client.admin' 2024-05-08T05:29:36.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:35 smithi096 ceph-mon[34253]: [08/May/2024:05:29:34] ENGINE Bus STARTING 2024-05-08T05:29:36.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:35 smithi096 ceph-mon[34253]: [08/May/2024:05:29:34] ENGINE Serving on http://172.21.15.59:8765 2024-05-08T05:29:36.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:35 smithi096 ceph-mon[34253]: [08/May/2024:05:29:34] ENGINE Serving on https://172.21.15.59:7150 2024-05-08T05:29:36.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:35 smithi096 ceph-mon[34253]: [08/May/2024:05:29:34] ENGINE Bus STARTED 2024-05-08T05:29:36.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:35 smithi096 ceph-mon[34253]: mgrmap e22: x(active, since 1.01948s), standbys: y, z 2024-05-08T05:29:36.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:35 smithi096 ceph-mon[34253]: from='client.24976 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup pin", "vol_name": "cephfs", "group_name": "qa", "pin_type": "random", "pin_setting": "0.10", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:36.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:35 smithi096 ceph-mon[34253]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:29:36.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:35 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/142826838' entity='client.admin' 2024-05-08T05:29:36.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:35 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1259591823' entity='client.admin' 2024-05-08T05:29:36.350 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.350+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f48a006a870 con 0x7f48880b2690 2024-05-08T05:29:36.410 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.409+0000 7f489cff9640 1 -- 172.21.15.59:0/1865699025 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f48a006a870 con 0x7f48880b2690 2024-05-08T05:29:36.410 INFO:teuthology.orchestra.run.smithi059.stdout:Schedule set for path / 2024-05-08T05:29:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f48880b2690 msgr2=0x7f48880b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.412 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f48880b2690 0x7f48880b4b50 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f48a0102960 tx=0x7f4890005f90 comp rx=0 tx=0).stop 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0102b00 msgr2=0x7f48a0198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0102b00 0x7f48a0198020 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f489400e9f0 tx=0x7f489400eeb0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 shutdown_connections 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f48880b2690 0x7f48880b4b50 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f48a01034c0 0x7f48a019f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f48a0102b00 0x7f48a0198020 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 --2- 172.21.15.59:0/1865699025 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f48a0101910 0x7f48a0197ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 >> 172.21.15.59:0/1865699025 conn(0x7f48a00fce80 msgr2=0x7f48a0105e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 shutdown_connections 2024-05-08T05:29:36.413 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.412+0000 7f48a5fa4640 1 -- 172.21.15.59:0/1865699025 wait complete. 2024-05-08T05:29:36.424 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-05-08T05:29:36.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.533+0000 7fca73023640 1 -- 172.21.15.59:0/3146010763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c101280 msgr2=0x7fca6c1016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.533+0000 7fca73023640 1 --2- 172.21.15.59:0/3146010763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c101280 0x7fca6c1016a0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7fca60009900 tx=0x7fca6002a410 comp rx=0 tx=0).stop 2024-05-08T05:29:36.536 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.533+0000 7fca73023640 1 -- 172.21.15.59:0/3146010763 shutdown_connections 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.533+0000 7fca73023640 1 --2- 172.21.15.59:0/3146010763 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca6c102580 0x7fca6c102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.533+0000 7fca73023640 1 --2- 172.21.15.59:0/3146010763 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca6c101be0 0x7fca6c102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.533+0000 7fca73023640 1 --2- 172.21.15.59:0/3146010763 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c101280 0x7fca6c1016a0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.533+0000 7fca73023640 1 -- 172.21.15.59:0/3146010763 >> 172.21.15.59:0/3146010763 conn(0x7fca6c0fce80 msgr2=0x7fca6c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.533+0000 7fca73023640 1 -- 172.21.15.59:0/3146010763 shutdown_connections 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.533+0000 7fca73023640 1 -- 172.21.15.59:0/3146010763 wait complete. 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.534+0000 7fca73023640 1 Processor -- start 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.534+0000 7fca73023640 1 -- start start 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.534+0000 7fca73023640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca6c101280 0x7fca6c19adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.534+0000 7fca73023640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca6c101be0 0x7fca6c19b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.534+0000 7fca73023640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c102580 0x7fca6c1a2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.534+0000 7fca73023640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c110700 con 0x7fca6c102580 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.534+0000 7fca73023640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c110580 con 0x7fca6c101280 2024-05-08T05:29:36.537 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.534+0000 7fca73023640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fca6c110880 con 0x7fca6c101be0 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.534+0000 7fca715c0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c102580 0x7fca6c1a2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca70dbf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca6c101280 0x7fca6c19adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca715c0640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c102580 0x7fca6c1a2330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49562/0 (socket says 172.21.15.59:49562) 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca70dbf640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca6c101280 0x7fca6c19adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:43148/0 (socket says 172.21.15.59:43148) 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca715c0640 1 -- 172.21.15.59:0/1346864824 learned_addr learned my addr 172.21.15.59:0/1346864824 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca6bfff640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca6c101be0 0x7fca6c19b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca715c0640 1 -- 172.21.15.59:0/1346864824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca6c101be0 msgr2=0x7fca6c19b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca715c0640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca6c101be0 0x7fca6c19b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca715c0640 1 -- 172.21.15.59:0/1346864824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca6c101280 msgr2=0x7fca6c19adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca715c0640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca6c101280 0x7fca6c19adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca715c0640 1 -- 172.21.15.59:0/1346864824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca6c1a29d0 con 0x7fca6c102580 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca6bfff640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca6c101be0 0x7fca6c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:36.538 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca70dbf640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca6c101280 0x7fca6c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca715c0640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c102580 0x7fca6c1a2330 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fca6400eaa0 tx=0x7fca6400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca69ffb640 1 -- 172.21.15.59:0/1346864824 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca6400cdb0 con 0x7fca6c102580 2024-05-08T05:29:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca69ffb640 1 -- 172.21.15.59:0/1346864824 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca64005020 con 0x7fca6c102580 2024-05-08T05:29:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca6c1a2cc0 con 0x7fca6c102580 2024-05-08T05:29:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.535+0000 7fca69ffb640 1 -- 172.21.15.59:0/1346864824 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca640116d0 con 0x7fca6c102580 2024-05-08T05:29:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.536+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca6c1a31c0 con 0x7fca6c102580 2024-05-08T05:29:36.539 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.537+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca6c10bfd0 con 0x7fca6c102580 2024-05-08T05:29:36.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.539+0000 7fca69ffb640 1 -- 172.21.15.59:0/1346864824 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 22) ==== 149464+0+0 (secure 0 0 0) 0x7fca640051d0 con 0x7fca6c102580 2024-05-08T05:29:36.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.540+0000 7fca69ffb640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fca5c0b2690 0x7fca5c0b4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.540+0000 7fca70dbf640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fca5c0b2690 0x7fca5c0b4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.540 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.540+0000 7fca69ffb640 1 -- 172.21.15.59:0/1346864824 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fca640e28c0 con 0x7fca6c102580 2024-05-08T05:29:36.541 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.541+0000 7fca70dbf640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fca5c0b2690 0x7fca5c0b4b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca60009230 tx=0x7fca600314a0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:36.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.544+0000 7fca69ffb640 1 -- 172.21.15.59:0/1346864824 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fca64016030 con 0x7fca6c102580 2024-05-08T05:29:36.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.623+0000 7fca69ffb640 1 -- 172.21.15.59:0/1346864824 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7fca640a5f40 con 0x7fca6c102580 2024-05-08T05:29:36.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.773+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7fca6c062e90 con 0x7fca5c0b2690 2024-05-08T05:29:36.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.836+0000 7fca69ffb640 1 -- 172.21.15.59:0/1346864824 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7fca6c062e90 con 0x7fca5c0b2690 2024-05-08T05:29:36.837 INFO:teuthology.orchestra.run.smithi059.stdout:Retention added to path / 2024-05-08T05:29:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fca5c0b2690 msgr2=0x7fca5c0b4b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fca5c0b2690 0x7fca5c0b4b50 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fca60009230 tx=0x7fca600314a0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c102580 msgr2=0x7fca6c1a2330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c102580 0x7fca6c1a2330 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7fca6400eaa0 tx=0x7fca6400ef60 comp rx=0 tx=0).stop 2024-05-08T05:29:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 shutdown_connections 2024-05-08T05:29:36.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fca5c0b2690 0x7fca5c0b4b50 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fca6c102580 0x7fca6c1a2330 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fca6c101be0 0x7fca6c19b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 --2- 172.21.15.59:0/1346864824 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fca6c101280 0x7fca6c19adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 >> 172.21.15.59:0/1346864824 conn(0x7fca6c0fce80 msgr2=0x7fca6c1001e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 shutdown_connections 2024-05-08T05:29:36.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.839+0000 7fca73023640 1 -- 172.21.15.59:0/1346864824 wait complete. 2024-05-08T05:29:36.851 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T05:29:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.960+0000 7f10ab62a640 1 -- 172.21.15.59:0/2775179949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4102580 msgr2=0x7f10a4102a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.960+0000 7f10ab62a640 1 --2- 172.21.15.59:0/2775179949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4102580 0x7f10a4102a00 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f109c009ca0 tx=0x7f109c02a8c0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.960+0000 7f10ab62a640 1 -- 172.21.15.59:0/2775179949 shutdown_connections 2024-05-08T05:29:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.960+0000 7f10ab62a640 1 --2- 172.21.15.59:0/2775179949 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4102580 0x7f10a4102a00 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.960+0000 7f10ab62a640 1 --2- 172.21.15.59:0/2775179949 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10a4101be0 0x7f10a4102040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.960+0000 7f10ab62a640 1 --2- 172.21.15.59:0/2775179949 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10a4101280 0x7f10a41016a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.960+0000 7f10ab62a640 1 -- 172.21.15.59:0/2775179949 >> 172.21.15.59:0/2775179949 conn(0x7f10a40fce80 msgr2=0x7f10a40ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.960+0000 7f10ab62a640 1 -- 172.21.15.59:0/2775179949 shutdown_connections 2024-05-08T05:29:36.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.960+0000 7f10ab62a640 1 -- 172.21.15.59:0/2775179949 wait complete. 2024-05-08T05:29:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10ab62a640 1 Processor -- start 2024-05-08T05:29:36.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10ab62a640 1 -- start start 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10ab62a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10a4101280 0x7f10a419ad30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10ab62a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4101be0 0x7f10a419b270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10ab62a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10a4102580 0x7f10a41a22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10ab62a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f10a4110730 con 0x7f10a4101be0 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10ab62a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f10a41105b0 con 0x7f10a4101280 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10ab62a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f10a41108b0 con 0x7f10a4102580 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10a8bc5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4101be0 0x7f10a419b270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10a8bc5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4101be0 0x7f10a419b270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49574/0 (socket says 172.21.15.59:49574) 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10a93c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10a4101280 0x7f10a419ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.961+0000 7f10a8bc5640 1 -- 172.21.15.59:0/4058604638 learned_addr learned my addr 172.21.15.59:0/4058604638 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10a9bc7640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10a4102580 0x7f10a41a22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10a93c6640 1 -- 172.21.15.59:0/4058604638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10a4102580 msgr2=0x7f10a41a22a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10a93c6640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10a4102580 0x7f10a41a22a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10a93c6640 1 -- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4101be0 msgr2=0x7f10a419b270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10a93c6640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4101be0 0x7f10a419b270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10a93c6640 1 -- 172.21.15.59:0/4058604638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10a41a2940 con 0x7f10a4101280 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10a8bc5640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4101be0 0x7f10a419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10a93c6640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10a4101280 0x7f10a419ad30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10a000e9f0 tx=0x7f10a000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f109a7fc640 1 -- 172.21.15.59:0/4058604638 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a000ccc0 con 0x7f10a4101280 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10a41a2c30 con 0x7f10a4101280 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f109a7fc640 1 -- 172.21.15.59:0/4058604638 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a00108e0 con 0x7f10a4101280 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f109a7fc640 1 -- 172.21.15.59:0/4058604638 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0018880 con 0x7f10a4101280 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.962+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10a41a31d0 con 0x7f10a4101280 2024-05-08T05:29:36.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.963+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f10a410bfd0 con 0x7f10a4101280 2024-05-08T05:29:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.965+0000 7f109a7fc640 1 -- 172.21.15.59:0/4058604638 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7f10a0010430 con 0x7f10a4101280 2024-05-08T05:29:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.965+0000 7f109a7fc640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f10900b2a90 0x7f10900b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.965+0000 7f10a8bc5640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f10900b2a90 0x7f10900b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.966+0000 7f109a7fc640 1 -- 172.21.15.59:0/4058604638 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f10a00e2ed0 con 0x7f10a4101280 2024-05-08T05:29:36.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.966+0000 7f10a8bc5640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f10900b2a90 0x7f10900b4f50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1094011500 tx=0x7f1094005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:36.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:36.970+0000 7f109a7fc640 1 -- 172.21.15.59:0/4058604638 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f10a00a9d80 con 0x7f10a4101280 2024-05-08T05:29:37.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.195+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f10a4062df0 con 0x7f10900b2a90 2024-05-08T05:29:37.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.203+0000 7f109a7fc640 1 -- 172.21.15.59:0/4058604638 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7f10a4062df0 con 0x7f10900b2a90 2024-05-08T05:29:37.203 INFO:teuthology.orchestra.run.smithi059.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T05:29:36", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f10900b2a90 msgr2=0x7f10900b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f10900b2a90 0x7f10900b4f50 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f1094011500 tx=0x7f1094005f90 comp rx=0 tx=0).stop 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10a4101280 msgr2=0x7f10a419ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10a4101280 0x7f10a419ad30 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f10a000e9f0 tx=0x7f10a000eeb0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 shutdown_connections 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f10900b2a90 0x7f10900b4f50 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f10a4102580 0x7f10a41a22a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f10a4101be0 0x7f10a419b270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 --2- 172.21.15.59:0/4058604638 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f10a4101280 0x7f10a419ad30 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 >> 172.21.15.59:0/4058604638 conn(0x7f10a40fce80 msgr2=0x7f10a40fe980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 shutdown_connections 2024-05-08T05:29:37.206 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.205+0000 7f10ab62a640 1 -- 172.21.15.59:0/4058604638 wait complete. 2024-05-08T05:29:37.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:36 smithi059 ceph-mon[30304]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:29:37.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:36 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:36 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:36 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2616376350' entity='client.admin' 2024-05-08T05:29:37.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:36 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:36 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:36 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:36 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:36 smithi059 ceph-mon[30304]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-08T05:29:37.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T05:29:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:36 smithi143 ceph-mon[34337]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:29:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:36 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:36 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:36 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2616376350' entity='client.admin' 2024-05-08T05:29:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:36 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:36 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:36 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:36 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:36 smithi143 ceph-mon[34337]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-08T05:29:37.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:36 smithi096 ceph-mon[34253]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:29:37.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:36 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:36 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:36 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2616376350' entity='client.admin' 2024-05-08T05:29:37.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:36 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:36 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:36 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:36 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:37.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:36 smithi096 ceph-mon[34253]: mgrmap e23: x(active, since 2s), standbys: y, z 2024-05-08T05:29:37.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.324+0000 7f6370d89640 1 -- 172.21.15.59:0/3891909288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c105e20 msgr2=0x7f636c108210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:37.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.324+0000 7f6370d89640 1 --2- 172.21.15.59:0/3891909288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c105e20 0x7f636c108210 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f6360009900 tx=0x7f636002a410 comp rx=0 tx=0).stop 2024-05-08T05:29:37.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.325+0000 7f6370d89640 1 -- 172.21.15.59:0/3891909288 shutdown_connections 2024-05-08T05:29:37.327 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.325+0000 7f6370d89640 1 --2- 172.21.15.59:0/3891909288 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f636c108750 0x7f636c10ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.325+0000 7f6370d89640 1 --2- 172.21.15.59:0/3891909288 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c105e20 0x7f636c108210 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.325+0000 7f6370d89640 1 --2- 172.21.15.59:0/3891909288 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f636c1034f0 0x7f636c1058e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.325+0000 7f6370d89640 1 -- 172.21.15.59:0/3891909288 >> 172.21.15.59:0/3891909288 conn(0x7f636c0fce80 msgr2=0x7f636c0ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.325+0000 7f6370d89640 1 -- 172.21.15.59:0/3891909288 shutdown_connections 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.325+0000 7f6370d89640 1 -- 172.21.15.59:0/3891909288 wait complete. 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.325+0000 7f6370d89640 1 Processor -- start 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.325+0000 7f6370d89640 1 -- start start 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f6370d89640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f636c1034f0 0x7f636c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f6370d89640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f636c105e20 0x7f636c199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f6370d89640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c108750 0x7f636c1a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f6370d89640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f636c10e580 con 0x7f636c108750 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f6370d89640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f636c10e400 con 0x7f636c105e20 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f6370d89640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f636c10e700 con 0x7f636c1034f0 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636ad9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c108750 0x7f636c1a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:37.328 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636ad9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c108750 0x7f636c1a01a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49588/0 (socket says 172.21.15.59:49588) 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636ad9d640 1 -- 172.21.15.59:0/3211461992 learned_addr learned my addr 172.21.15.59:0/3211461992 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636a59c640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f636c1034f0 0x7f636c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f6369d9b640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f636c105e20 0x7f636c199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636ad9d640 1 -- 172.21.15.59:0/3211461992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f636c1034f0 msgr2=0x7f636c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636ad9d640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f636c1034f0 0x7f636c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636ad9d640 1 -- 172.21.15.59:0/3211461992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f636c105e20 msgr2=0x7f636c199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636ad9d640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f636c105e20 0x7f636c199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636ad9d640 1 -- 172.21.15.59:0/3211461992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f636c1a0840 con 0x7f636c108750 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636a59c640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f636c1034f0 0x7f636c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f6369d9b640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f636c105e20 0x7f636c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f636ad9d640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c108750 0x7f636c1a01a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f636400efd0 tx=0x7f636400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.326+0000 7f635b7fe640 1 -- 172.21.15.59:0/3211461992 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6364003b70 con 0x7f636c108750 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.327+0000 7f635b7fe640 1 -- 172.21.15.59:0/3211461992 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6364003d10 con 0x7f636c108750 2024-05-08T05:29:37.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.327+0000 7f635b7fe640 1 -- 172.21.15.59:0/3211461992 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63640108a0 con 0x7f636c108750 2024-05-08T05:29:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.327+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f636c1a0b30 con 0x7f636c108750 2024-05-08T05:29:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.327+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f636c1a1080 con 0x7f636c108750 2024-05-08T05:29:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.328+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f636c0fe970 con 0x7f636c108750 2024-05-08T05:29:37.330 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.330+0000 7f635b7fe640 1 -- 172.21.15.59:0/3211461992 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 23) ==== 149823+0+0 (secure 0 0 0) 0x7f6364004b40 con 0x7f636c108750 2024-05-08T05:29:37.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.331+0000 7f635b7fe640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f63540b2b10 0x7f63540b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:37.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.331+0000 7f635b7fe640 1 -- 172.21.15.59:0/3211461992 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f63640e3320 con 0x7f636c108750 2024-05-08T05:29:37.331 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.331+0000 7f636a59c640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f63540b2b10 0x7f63540b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:37.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.332+0000 7f636a59c640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f63540b2b10 0x7f63540b4fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f636c19a100 tx=0x7f635c005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:37.335 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.334+0000 7f635b7fe640 1 -- 172.21.15.59:0/3211461992 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f63640aa820 con 0x7f636c108750 2024-05-08T05:29:37.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.563+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f636c062df0 con 0x7f63540b2b10 2024-05-08T05:29:37.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.570+0000 7f635b7fe640 1 -- 172.21.15.59:0/3211461992 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f636c062df0 con 0x7f63540b2b10 2024-05-08T05:29:37.570 INFO:teuthology.orchestra.run.smithi059.stdout:/ 1m 6m3h 2024-05-08T05:29:37.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.572+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f63540b2b10 msgr2=0x7f63540b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:37.573 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.572+0000 7f6370d89640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f63540b2b10 0x7f63540b4fd0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f636c19a100 tx=0x7f635c005f90 comp rx=0 tx=0).stop 2024-05-08T05:29:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.572+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c108750 msgr2=0x7f636c1a01a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.572+0000 7f6370d89640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c108750 0x7f636c1a01a0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f636400efd0 tx=0x7f636400c6a0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.572+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 shutdown_connections 2024-05-08T05:29:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.572+0000 7f6370d89640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f63540b2b10 0x7f63540b4fd0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.572+0000 7f6370d89640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f636c108750 0x7f636c1a01a0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.572+0000 7f6370d89640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f636c105e20 0x7f636c199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.572+0000 7f6370d89640 1 --2- 172.21.15.59:0/3211461992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f636c1034f0 0x7f636c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.573+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 >> 172.21.15.59:0/3211461992 conn(0x7f636c0fce80 msgr2=0x7f636c103e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:37.574 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.573+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 shutdown_connections 2024-05-08T05:29:37.575 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:37.573+0000 7f6370d89640 1 -- 172.21.15.59:0/3211461992 wait complete. 2024-05-08T05:29:37.584 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-05-08T05:29:37.612 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-05-08T05:29:37.624 INFO:teuthology.run_tasks:Running task background_exec... 2024-05-08T05:29:37.644 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:29:37.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-05-08T05:29:37.654 INFO:teuthology.run_tasks:Running task quiescer... 2024-05-08T05:29:37.667 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-05-08T05:29:37.667 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:29:37.933 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:38.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:38.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi143 to 176.2M 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: Unable to set osd_memory_target on smithi143 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:29:38.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:38 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:29:38.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:38.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:38.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-08T05:29:38.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:29:38.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi143 to 176.2M 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: Unable to set osd_memory_target on smithi143 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.497 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.497 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:29:38.497 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:38 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='client.15360 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='client.15366 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi143 to 176.2M 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: Unable to set osd_memory_target on smithi143 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='client.25058 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:29:38.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:38 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Unable to set osd_memory_target on smithi059 to 185879859: error parsing value: Value '185879859' is below minimum 939524096 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:29:39.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:29:39.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:29:39.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:29:39.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-08T05:29:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Unable to set osd_memory_target on smithi059 to 185879859: error parsing value: Value '185879859' is below minimum 939524096 2024-05-08T05:29:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-08T05:29:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-08T05:29:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:29:39.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:29:39.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:29:39.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Unable to set osd_memory_target on smithi059 to 185879859: error parsing value: Value '185879859' is below minimum 939524096 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Updating smithi059:/etc/ceph/ceph.conf 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Updating smithi096:/etc/ceph/ceph.conf 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Updating smithi143:/etc/ceph/ceph.conf 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='client.15378 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.conf 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Updating smithi059:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Updating smithi143:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: Updating smithi096:/etc/ceph/ceph.client.admin.keyring 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: mgrmap e24: x(active, since 4s), standbys: y, z 2024-05-08T05:29:39.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:29:39.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:29:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.778+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3913651111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44108870 msgr2=0x7f2e441103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:39.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.778+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3913651111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44108870 0x7f2e441103f0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f2e3c00b0d0 tx=0x7f2e3c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.778+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3913651111 shutdown_connections 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.778+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3913651111 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44108870 0x7f2e441103f0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.778+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3913651111 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44103780 0x7f2e44103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.778+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3913651111 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e44104130 0x7f2e44108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.778+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3913651111 >> 172.21.15.59:0/3913651111 conn(0x7f2e4407a610 msgr2=0x7f2e4407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.778+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3913651111 shutdown_connections 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.778+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3913651111 wait complete. 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.779+0000 7f2e4c69a700 1 Processor -- start 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.779+0000 7f2e4c69a700 1 -- start start 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.779+0000 7f2e4c69a700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44103780 0x7f2e4419f740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.779+0000 7f2e4c69a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44104130 0x7f2e4419fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:39.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e4c69a700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e44108870 0x7f2e44199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e4c69a700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2e441136b0 con 0x7f2e44104130 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e4c69a700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44113530 con 0x7f2e44103780 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e4c69a700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2e44113830 con 0x7f2e44108870 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e49c35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44104130 0x7f2e4419fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e49c35700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44104130 0x7f2e4419fc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53758/0 (socket says 172.21.15.59:53758) 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e49c35700 1 -- 172.21.15.59:0/3047899457 learned_addr learned my addr 172.21.15.59:0/3047899457 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e4a436700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44103780 0x7f2e4419f740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e4ac37700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e44108870 0x7f2e44199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e49c35700 1 -- 172.21.15.59:0/3047899457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e44108870 msgr2=0x7f2e44199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e49c35700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e44108870 0x7f2e44199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e49c35700 1 -- 172.21.15.59:0/3047899457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44103780 msgr2=0x7f2e4419f740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e49c35700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44103780 0x7f2e4419f740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e49c35700 1 -- 172.21.15.59:0/3047899457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4419a0b0 con 0x7f2e44104130 2024-05-08T05:29:39.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e49c35700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44104130 0x7f2e4419fc80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400b530 tx=0x7f2e3400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e3b7fe700 1 -- 172.21.15.59:0/3047899457 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34015020 con 0x7f2e44104130 2024-05-08T05:29:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e3b7fe700 1 -- 172.21.15.59:0/3047899457 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e34011ce0 con 0x7f2e44104130 2024-05-08T05:29:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e3b7fe700 1 -- 172.21.15.59:0/3047899457 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e34010e40 con 0x7f2e44104130 2024-05-08T05:29:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.780+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e4419a3a0 con 0x7f2e44104130 2024-05-08T05:29:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.781+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e441a6610 con 0x7f2e44104130 2024-05-08T05:29:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.781+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e4406a190 con 0x7f2e44104130 2024-05-08T05:29:39.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.783+0000 7f2e3b7fe700 1 -- 172.21.15.59:0/3047899457 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f2e34018020 con 0x7f2e44104130 2024-05-08T05:29:39.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.784+0000 7f2e3b7fe700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2e300b2b10 0x7f2e300b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:39.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.784+0000 7f2e4a436700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2e300b2b10 0x7f2e300b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:39.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.784+0000 7f2e3b7fe700 1 -- 172.21.15.59:0/3047899457 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f2e340e2850 con 0x7f2e44104130 2024-05-08T05:29:39.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.784+0000 7f2e4a436700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2e300b2b10 0x7f2e300b4fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e441083b0 tx=0x7f2e4000b410 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:39.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:39.787+0000 7f2e3b7fe700 1 -- 172.21.15.59:0/3047899457 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f2e3401f020 con 0x7f2e44104130 2024-05-08T05:29:40.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.000+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2e440638e0 con 0x7f2e300b2b10 2024-05-08T05:29:40.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.010+0000 7f2e3b7fe700 1 -- 172.21.15.59:0/3047899457 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339717 (secure 0 0 0) 0x7f2e440638e0 con 0x7f2e300b2b10 2024-05-08T05:29:40.016 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:29:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2e300b2b10 msgr2=0x7f2e300b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2e300b2b10 0x7f2e300b4fd0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2e441083b0 tx=0x7f2e4000b410 comp rx=0 tx=0).stop 2024-05-08T05:29:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44104130 msgr2=0x7f2e4419fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44104130 0x7f2e4419fc80 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2e3400b530 tx=0x7f2e3400b840 comp rx=0 tx=0).stop 2024-05-08T05:29:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 shutdown_connections 2024-05-08T05:29:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2e300b2b10 0x7f2e300b4fd0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2e44108870 0x7f2e44199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:40.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2e44103780 0x7f2e4419f740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:40.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 --2- 172.21.15.59:0/3047899457 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2e44104130 0x7f2e4419fc80 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:40.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 >> 172.21.15.59:0/3047899457 conn(0x7f2e4407a610 msgr2=0x7f2e44101640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:40.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 shutdown_connections 2024-05-08T05:29:40.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:40.014+0000 7f2e4c69a700 1 -- 172.21.15.59:0/3047899457 wait complete. 2024-05-08T05:29:40.021 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T05:29:40.318 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:40 smithi059 ceph-mon[30304]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:29:40.319 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:40 smithi059 ceph-mon[30304]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:29:40.319 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:40 smithi059 ceph-mon[30304]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:29:40.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:40 smithi143 ceph-mon[34337]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:29:40.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:40 smithi143 ceph-mon[34337]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:29:40.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:40 smithi143 ceph-mon[34337]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:29:40.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:40 smithi096 ceph-mon[34253]: Updating smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:29:40.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:40 smithi096 ceph-mon[34253]: Updating smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:29:40.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:40 smithi096 ceph-mon[34253]: Updating smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/config/ceph.client.admin.keyring 2024-05-08T05:29:40.800 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":6,"stamp":"2024-05-08T05:29:39.840343+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1195747,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2101,"num_read_kb":1199,"num_write":515,"num_write_kb":3634,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951840,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":909,"ondisk_log_size":909,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361752,"kb_used_data":12564,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580072,"statfs":{"total":1151940427776,"available":1151569993728,"internally_reserved":0,"allocated":12865536,"data_stored":10116942,"data_compressed":614687,"data_compressed_allocated":831488,"data_compressed_original":4673536,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.991088"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088374+0000","last_change":"2024-05-08T05:25:25.667895+0000","last_active":"2024-05-08T05:29:34.088374+0000","last_peered":"2024-05-08T05:29:34.088374+0000","last_clean":"2024-05-08T05:29:34.088374+0000","last_became_active":"2024-05-08T05:25:25.667563+0000","last_became_peered":"2024-05-08T05:25:25.667563+0000","last_unstale":"2024-05-08T05:29:34.088374+0000","last_undegraded":"2024-05-08T05:29:34.088374+0000","last_fullsized":"2024-05-08T05:29:34.088374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:47.677082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088642+0000","last_change":"2024-05-08T05:25:25.673346+0000","last_active":"2024-05-08T05:29:34.088642+0000","last_peered":"2024-05-08T05:29:34.088642+0000","last_clean":"2024-05-08T05:29:34.088642+0000","last_became_active":"2024-05-08T05:25:25.673011+0000","last_became_peered":"2024-05-08T05:25:25.673011+0000","last_unstale":"2024-05-08T05:29:34.088642+0000","last_undegraded":"2024-05-08T05:29:34.088642+0000","last_fullsized":"2024-05-08T05:29:34.088642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:21:05.091177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.089071+0000","last_change":"2024-05-08T05:25:25.670469+0000","last_active":"2024-05-08T05:29:34.089071+0000","last_peered":"2024-05-08T05:29:34.089071+0000","last_clean":"2024-05-08T05:29:34.089071+0000","last_became_active":"2024-05-08T05:25:25.669700+0000","last_became_peered":"2024-05-08T05:25:25.669700+0000","last_unstale":"2024-05-08T05:29:34.089071+0000","last_undegraded":"2024-05-08T05:29:34.089071+0000","last_fullsized":"2024-05-08T05:29:34.089071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:06.046633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093206+0000","last_change":"2024-05-08T05:25:25.659937+0000","last_active":"2024-05-08T05:29:34.093206+0000","last_peered":"2024-05-08T05:29:34.093206+0000","last_clean":"2024-05-08T05:29:34.093206+0000","last_became_active":"2024-05-08T05:25:25.658892+0000","last_became_peered":"2024-05-08T05:25:25.658892+0000","last_unstale":"2024-05-08T05:29:34.093206+0000","last_undegraded":"2024-05-08T05:29:34.093206+0000","last_fullsized":"2024-05-08T05:29:34.093206+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:10.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925563+0000","last_change":"2024-05-08T05:25:25.669132+0000","last_active":"2024-05-08T05:29:33.925563+0000","last_peered":"2024-05-08T05:29:33.925563+0000","last_clean":"2024-05-08T05:29:33.925563+0000","last_became_active":"2024-05-08T05:25:25.668556+0000","last_became_peered":"2024-05-08T05:25:25.668556+0000","last_unstale":"2024-05-08T05:29:33.925563+0000","last_undegraded":"2024-05-08T05:29:33.925563+0000","last_fullsized":"2024-05-08T05:29:33.925563+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:28.443763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936400+0000","last_change":"2024-05-08T05:25:25.668978+0000","last_active":"2024-05-08T05:29:33.936400+0000","last_peered":"2024-05-08T05:29:33.936400+0000","last_clean":"2024-05-08T05:29:33.936400+0000","last_became_active":"2024-05-08T05:25:25.668647+0000","last_became_peered":"2024-05-08T05:25:25.668647+0000","last_unstale":"2024-05-08T05:29:33.936400+0000","last_undegraded":"2024-05-08T05:29:33.936400+0000","last_fullsized":"2024-05-08T05:29:33.936400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:07:00.962620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085103+0000","last_change":"2024-05-08T05:25:25.664784+0000","last_active":"2024-05-08T05:29:34.085103+0000","last_peered":"2024-05-08T05:29:34.085103+0000","last_clean":"2024-05-08T05:29:34.085103+0000","last_became_active":"2024-05-08T05:25:25.664390+0000","last_became_peered":"2024-05-08T05:25:25.664390+0000","last_unstale":"2024-05-08T05:29:34.085103+0000","last_undegraded":"2024-05-08T05:29:34.085103+0000","last_fullsized":"2024-05-08T05:29:34.085103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:08:35.358796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925616+0000","last_change":"2024-05-08T05:25:25.677139+0000","last_active":"2024-05-08T05:29:33.925616+0000","last_peered":"2024-05-08T05:29:33.925616+0000","last_clean":"2024-05-08T05:29:33.925616+0000","last_became_active":"2024-05-08T05:25:25.676444+0000","last_became_peered":"2024-05-08T05:25:25.676444+0000","last_unstale":"2024-05-08T05:29:33.925616+0000","last_undegraded":"2024-05-08T05:29:33.925616+0000","last_fullsized":"2024-05-08T05:29:33.925616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:56.704152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826525+0000","last_change":"2024-05-08T05:25:25.675593+0000","last_active":"2024-05-08T05:29:33.826525+0000","last_peered":"2024-05-08T05:29:33.826525+0000","last_clean":"2024-05-08T05:29:33.826525+0000","last_became_active":"2024-05-08T05:25:25.675136+0000","last_became_peered":"2024-05-08T05:25:25.675136+0000","last_unstale":"2024-05-08T05:29:33.826525+0000","last_undegraded":"2024-05-08T05:29:33.826525+0000","last_fullsized":"2024-05-08T05:29:33.826525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:49.042956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936724+0000","last_change":"2024-05-08T05:25:25.669384+0000","last_active":"2024-05-08T05:29:33.936724+0000","last_peered":"2024-05-08T05:29:33.936724+0000","last_clean":"2024-05-08T05:29:33.936724+0000","last_became_active":"2024-05-08T05:25:25.668861+0000","last_became_peered":"2024-05-08T05:25:25.668861+0000","last_unstale":"2024-05-08T05:29:33.936724+0000","last_undegraded":"2024-05-08T05:29:33.936724+0000","last_fullsized":"2024-05-08T05:29:33.936724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:57.491242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085723+0000","last_change":"2024-05-08T05:25:25.669463+0000","last_active":"2024-05-08T05:29:34.085723+0000","last_peered":"2024-05-08T05:29:34.085723+0000","last_clean":"2024-05-08T05:29:34.085723+0000","last_became_active":"2024-05-08T05:25:25.668672+0000","last_became_peered":"2024-05-08T05:25:25.668672+0000","last_unstale":"2024-05-08T05:29:34.085723+0000","last_undegraded":"2024-05-08T05:29:34.085723+0000","last_fullsized":"2024-05-08T05:29:34.085723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:30.779473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093195+0000","last_change":"2024-05-08T05:25:25.670378+0000","last_active":"2024-05-08T05:29:34.093195+0000","last_peered":"2024-05-08T05:29:34.093195+0000","last_clean":"2024-05-08T05:29:34.093195+0000","last_became_active":"2024-05-08T05:25:25.669801+0000","last_became_peered":"2024-05-08T05:25:25.669801+0000","last_unstale":"2024-05-08T05:29:34.093195+0000","last_undegraded":"2024-05-08T05:29:34.093195+0000","last_fullsized":"2024-05-08T05:29:34.093195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:49.614163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680129+0000","last_change":"2024-05-08T05:25:25.670874+0000","last_active":"2024-05-08T05:29:34.680129+0000","last_peered":"2024-05-08T05:29:34.680129+0000","last_clean":"2024-05-08T05:29:34.680129+0000","last_became_active":"2024-05-08T05:25:25.670557+0000","last_became_peered":"2024-05-08T05:25:25.670557+0000","last_unstale":"2024-05-08T05:29:34.680129+0000","last_undegraded":"2024-05-08T05:29:34.680129+0000","last_fullsized":"2024-05-08T05:29:34.680129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:58.594223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088286+0000","last_change":"2024-05-08T05:25:25.670015+0000","last_active":"2024-05-08T05:29:34.088286+0000","last_peered":"2024-05-08T05:29:34.088286+0000","last_clean":"2024-05-08T05:29:34.088286+0000","last_became_active":"2024-05-08T05:25:25.669493+0000","last_became_peered":"2024-05-08T05:25:25.669493+0000","last_unstale":"2024-05-08T05:29:34.088286+0000","last_undegraded":"2024-05-08T05:29:34.088286+0000","last_fullsized":"2024-05-08T05:29:34.088286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:49.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088404+0000","last_change":"2024-05-08T05:25:25.659651+0000","last_active":"2024-05-08T05:29:34.088404+0000","last_peered":"2024-05-08T05:29:34.088404+0000","last_clean":"2024-05-08T05:29:34.088404+0000","last_became_active":"2024-05-08T05:25:25.659308+0000","last_became_peered":"2024-05-08T05:25:25.659308+0000","last_unstale":"2024-05-08T05:29:34.088404+0000","last_undegraded":"2024-05-08T05:29:34.088404+0000","last_fullsized":"2024-05-08T05:29:34.088404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:49.945474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825750+0000","last_change":"2024-05-08T05:25:25.663696+0000","last_active":"2024-05-08T05:29:33.825750+0000","last_peered":"2024-05-08T05:29:33.825750+0000","last_clean":"2024-05-08T05:29:33.825750+0000","last_became_active":"2024-05-08T05:25:25.662590+0000","last_became_peered":"2024-05-08T05:25:25.662590+0000","last_unstale":"2024-05-08T05:29:33.825750+0000","last_undegraded":"2024-05-08T05:29:33.825750+0000","last_fullsized":"2024-05-08T05:29:33.825750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:29:21.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088209+0000","last_change":"2024-05-08T05:25:25.668552+0000","last_active":"2024-05-08T05:29:34.088209+0000","last_peered":"2024-05-08T05:29:34.088209+0000","last_clean":"2024-05-08T05:29:34.088209+0000","last_became_active":"2024-05-08T05:25:25.668185+0000","last_became_peered":"2024-05-08T05:25:25.668185+0000","last_unstale":"2024-05-08T05:29:34.088209+0000","last_undegraded":"2024-05-08T05:29:34.088209+0000","last_fullsized":"2024-05-08T05:29:34.088209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:25.762909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825129+0000","last_change":"2024-05-08T05:25:25.668259+0000","last_active":"2024-05-08T05:29:33.825129+0000","last_peered":"2024-05-08T05:29:33.825129+0000","last_clean":"2024-05-08T05:29:33.825129+0000","last_became_active":"2024-05-08T05:25:25.667767+0000","last_became_peered":"2024-05-08T05:25:25.667767+0000","last_unstale":"2024-05-08T05:29:33.825129+0000","last_undegraded":"2024-05-08T05:29:33.825129+0000","last_fullsized":"2024-05-08T05:29:33.825129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:36:34.984139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827417+0000","last_change":"2024-05-08T05:25:25.663381+0000","last_active":"2024-05-08T05:29:33.827417+0000","last_peered":"2024-05-08T05:29:33.827417+0000","last_clean":"2024-05-08T05:29:33.827417+0000","last_became_active":"2024-05-08T05:25:25.663019+0000","last_became_peered":"2024-05-08T05:25:25.663019+0000","last_unstale":"2024-05-08T05:29:33.827417+0000","last_undegraded":"2024-05-08T05:29:33.827417+0000","last_fullsized":"2024-05-08T05:29:33.827417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:58.440446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826825+0000","last_change":"2024-05-08T05:25:37.052656+0000","last_active":"2024-05-08T05:29:33.826825+0000","last_peered":"2024-05-08T05:29:33.826825+0000","last_clean":"2024-05-08T05:29:33.826825+0000","last_became_active":"2024-05-08T05:25:37.052237+0000","last_became_peered":"2024-05-08T05:25:37.052237+0000","last_unstale":"2024-05-08T05:29:33.826825+0000","last_undegraded":"2024-05-08T05:29:33.826825+0000","last_fullsized":"2024-05-08T05:29:33.826825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:54.354488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"90'14","reported_seq":249,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.678043+0000","last_change":"2024-05-08T05:25:22.649492+0000","last_active":"2024-05-08T05:29:34.678043+0000","last_peered":"2024-05-08T05:29:34.678043+0000","last_clean":"2024-05-08T05:29:34.678043+0000","last_became_active":"2024-05-08T05:25:22.649132+0000","last_became_peered":"2024-05-08T05:25:22.649132+0000","last_unstale":"2024-05-08T05:29:34.678043+0000","last_undegraded":"2024-05-08T05:29:34.678043+0000","last_fullsized":"2024-05-08T05:29:34.678043+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:24.474848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":15,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926798+0000","last_change":"2024-05-08T05:25:25.667811+0000","last_active":"2024-05-08T05:29:33.926798+0000","last_peered":"2024-05-08T05:29:33.926798+0000","last_clean":"2024-05-08T05:29:33.926798+0000","last_became_active":"2024-05-08T05:25:25.667449+0000","last_became_peered":"2024-05-08T05:25:25.667449+0000","last_unstale":"2024-05-08T05:29:33.926798+0000","last_undegraded":"2024-05-08T05:29:33.926798+0000","last_fullsized":"2024-05-08T05:29:33.926798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:29.293702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092852+0000","last_change":"2024-05-08T05:26:05.308251+0000","last_active":"2024-05-08T05:29:34.092852+0000","last_peered":"2024-05-08T05:29:34.092852+0000","last_clean":"2024-05-08T05:29:34.092852+0000","last_became_active":"2024-05-08T05:26:05.307622+0000","last_became_peered":"2024-05-08T05:26:05.307622+0000","last_unstale":"2024-05-08T05:29:34.092852+0000","last_undegraded":"2024-05-08T05:29:34.092852+0000","last_fullsized":"2024-05-08T05:29:34.092852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:51.116094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926878+0000","last_change":"2024-05-08T05:25:22.748130+0000","last_active":"2024-05-08T05:29:33.926878+0000","last_peered":"2024-05-08T05:29:33.926878+0000","last_clean":"2024-05-08T05:29:33.926878+0000","last_became_active":"2024-05-08T05:25:22.747769+0000","last_became_peered":"2024-05-08T05:25:22.747769+0000","last_unstale":"2024-05-08T05:29:33.926878+0000","last_undegraded":"2024-05-08T05:29:33.926878+0000","last_fullsized":"2024-05-08T05:29:33.926878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:04.086851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088665+0000","last_change":"2024-05-08T05:25:25.672114+0000","last_active":"2024-05-08T05:29:34.088665+0000","last_peered":"2024-05-08T05:29:34.088665+0000","last_clean":"2024-05-08T05:29:34.088665+0000","last_became_active":"2024-05-08T05:25:25.671761+0000","last_became_peered":"2024-05-08T05:25:25.671761+0000","last_unstale":"2024-05-08T05:29:34.088665+0000","last_undegraded":"2024-05-08T05:29:34.088665+0000","last_fullsized":"2024-05-08T05:29:34.088665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:40.684565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092958+0000","last_change":"2024-05-08T05:25:37.058375+0000","last_active":"2024-05-08T05:29:34.092958+0000","last_peered":"2024-05-08T05:29:34.092958+0000","last_clean":"2024-05-08T05:29:34.092958+0000","last_became_active":"2024-05-08T05:25:37.057844+0000","last_became_peered":"2024-05-08T05:25:37.057844+0000","last_unstale":"2024-05-08T05:29:34.092958+0000","last_undegraded":"2024-05-08T05:29:34.092958+0000","last_fullsized":"2024-05-08T05:29:34.092958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:51:42.881830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088188+0000","last_change":"2024-05-08T05:27:05.576781+0000","last_active":"2024-05-08T05:29:34.088188+0000","last_peered":"2024-05-08T05:29:34.088188+0000","last_clean":"2024-05-08T05:29:34.088188+0000","last_became_active":"2024-05-08T05:27:05.566420+0000","last_became_peered":"2024-05-08T05:27:05.566420+0000","last_unstale":"2024-05-08T05:29:34.088188+0000","last_undegraded":"2024-05-08T05:29:34.088188+0000","last_fullsized":"2024-05-08T05:29:34.088188+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:09.385135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826181+0000","last_change":"2024-05-08T05:25:25.673300+0000","last_active":"2024-05-08T05:29:33.826181+0000","last_peered":"2024-05-08T05:29:33.826181+0000","last_clean":"2024-05-08T05:29:33.826181+0000","last_became_active":"2024-05-08T05:25:25.672881+0000","last_became_peered":"2024-05-08T05:25:25.672881+0000","last_unstale":"2024-05-08T05:29:33.826181+0000","last_undegraded":"2024-05-08T05:29:33.826181+0000","last_fullsized":"2024-05-08T05:29:33.826181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:16.111775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092846+0000","last_change":"2024-05-08T05:25:37.058903+0000","last_active":"2024-05-08T05:29:34.092846+0000","last_peered":"2024-05-08T05:29:34.092846+0000","last_clean":"2024-05-08T05:29:34.092846+0000","last_became_active":"2024-05-08T05:25:37.058508+0000","last_became_peered":"2024-05-08T05:25:37.058508+0000","last_unstale":"2024-05-08T05:29:34.092846+0000","last_undegraded":"2024-05-08T05:29:34.092846+0000","last_fullsized":"2024-05-08T05:29:34.092846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:15:16.605748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925877+0000","last_change":"2024-05-08T05:25:22.646191+0000","last_active":"2024-05-08T05:29:33.925877+0000","last_peered":"2024-05-08T05:29:33.925877+0000","last_clean":"2024-05-08T05:29:33.925877+0000","last_became_active":"2024-05-08T05:25:22.645187+0000","last_became_peered":"2024-05-08T05:25:22.645187+0000","last_unstale":"2024-05-08T05:29:33.925877+0000","last_undegraded":"2024-05-08T05:29:33.925877+0000","last_fullsized":"2024-05-08T05:29:33.925877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:13.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933506+0000","last_change":"2024-05-08T05:26:05.308149+0000","last_active":"2024-05-08T05:29:33.933506+0000","last_peered":"2024-05-08T05:29:33.933506+0000","last_clean":"2024-05-08T05:29:33.933506+0000","last_became_active":"2024-05-08T05:26:05.307602+0000","last_became_peered":"2024-05-08T05:26:05.307602+0000","last_unstale":"2024-05-08T05:29:33.933506+0000","last_undegraded":"2024-05-08T05:29:33.933506+0000","last_fullsized":"2024-05-08T05:29:33.933506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:02.791894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827181+0000","last_change":"2024-05-08T05:25:37.059052+0000","last_active":"2024-05-08T05:29:33.827181+0000","last_peered":"2024-05-08T05:29:33.827181+0000","last_clean":"2024-05-08T05:29:33.827181+0000","last_became_active":"2024-05-08T05:25:37.058545+0000","last_became_peered":"2024-05-08T05:25:37.058545+0000","last_unstale":"2024-05-08T05:29:33.827181+0000","last_undegraded":"2024-05-08T05:29:33.827181+0000","last_fullsized":"2024-05-08T05:29:33.827181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:31.521284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":248,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.679156+0000","last_change":"2024-05-08T05:25:22.648874+0000","last_active":"2024-05-08T05:29:34.679156+0000","last_peered":"2024-05-08T05:29:34.679156+0000","last_clean":"2024-05-08T05:29:34.679156+0000","last_became_active":"2024-05-08T05:25:22.648477+0000","last_became_peered":"2024-05-08T05:25:22.648477+0000","last_unstale":"2024-05-08T05:29:34.679156+0000","last_undegraded":"2024-05-08T05:29:34.679156+0000","last_fullsized":"2024-05-08T05:29:34.679156+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:08.075904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":192,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925572+0000","last_change":"2024-05-08T05:25:25.662449+0000","last_active":"2024-05-08T05:29:33.925572+0000","last_peered":"2024-05-08T05:29:33.925572+0000","last_clean":"2024-05-08T05:29:33.925572+0000","last_became_active":"2024-05-08T05:25:25.662089+0000","last_became_peered":"2024-05-08T05:25:25.662089+0000","last_unstale":"2024-05-08T05:29:33.925572+0000","last_undegraded":"2024-05-08T05:29:33.925572+0000","last_fullsized":"2024-05-08T05:29:33.925572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:31.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827128+0000","last_change":"2024-05-08T05:25:37.055548+0000","last_active":"2024-05-08T05:29:33.827128+0000","last_peered":"2024-05-08T05:29:33.827128+0000","last_clean":"2024-05-08T05:29:33.827128+0000","last_became_active":"2024-05-08T05:25:37.055138+0000","last_became_peered":"2024-05-08T05:25:37.055138+0000","last_unstale":"2024-05-08T05:29:33.827128+0000","last_undegraded":"2024-05-08T05:29:33.827128+0000","last_fullsized":"2024-05-08T05:29:33.827128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:45.697831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936365+0000","last_change":"2024-05-08T05:25:22.648072+0000","last_active":"2024-05-08T05:29:33.936365+0000","last_peered":"2024-05-08T05:29:33.936365+0000","last_clean":"2024-05-08T05:29:33.936365+0000","last_became_active":"2024-05-08T05:25:22.647457+0000","last_became_peered":"2024-05-08T05:25:22.647457+0000","last_unstale":"2024-05-08T05:29:33.936365+0000","last_undegraded":"2024-05-08T05:29:33.936365+0000","last_fullsized":"2024-05-08T05:29:33.936365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:17.476479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933534+0000","last_change":"2024-05-08T05:25:25.662698+0000","last_active":"2024-05-08T05:29:33.933534+0000","last_peered":"2024-05-08T05:29:33.933534+0000","last_clean":"2024-05-08T05:29:33.933534+0000","last_became_active":"2024-05-08T05:25:25.662368+0000","last_became_peered":"2024-05-08T05:25:25.662368+0000","last_unstale":"2024-05-08T05:29:33.933534+0000","last_undegraded":"2024-05-08T05:29:33.933534+0000","last_fullsized":"2024-05-08T05:29:33.933534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:38.511037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088201+0000","last_change":"2024-05-08T05:25:37.054407+0000","last_active":"2024-05-08T05:29:34.088201+0000","last_peered":"2024-05-08T05:29:34.088201+0000","last_clean":"2024-05-08T05:29:34.088201+0000","last_became_active":"2024-05-08T05:25:37.053999+0000","last_became_peered":"2024-05-08T05:25:37.053999+0000","last_unstale":"2024-05-08T05:29:34.088201+0000","last_undegraded":"2024-05-08T05:29:34.088201+0000","last_fullsized":"2024-05-08T05:29:34.088201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:16.942601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680002+0000","last_change":"2024-05-08T05:25:22.643654+0000","last_active":"2024-05-08T05:29:34.680002+0000","last_peered":"2024-05-08T05:29:34.680002+0000","last_clean":"2024-05-08T05:29:34.680002+0000","last_became_active":"2024-05-08T05:25:22.643315+0000","last_became_peered":"2024-05-08T05:25:22.643315+0000","last_unstale":"2024-05-08T05:29:34.680002+0000","last_undegraded":"2024-05-08T05:29:34.680002+0000","last_fullsized":"2024-05-08T05:29:34.680002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:53.514629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925576+0000","last_change":"2024-05-08T05:25:25.663148+0000","last_active":"2024-05-08T05:29:33.925576+0000","last_peered":"2024-05-08T05:29:33.925576+0000","last_clean":"2024-05-08T05:29:33.925576+0000","last_became_active":"2024-05-08T05:25:25.662770+0000","last_became_peered":"2024-05-08T05:25:25.662770+0000","last_unstale":"2024-05-08T05:29:33.925576+0000","last_undegraded":"2024-05-08T05:29:33.925576+0000","last_fullsized":"2024-05-08T05:29:33.925576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:46.458233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088500+0000","last_change":"2024-05-08T05:25:37.054265+0000","last_active":"2024-05-08T05:29:34.088500+0000","last_peered":"2024-05-08T05:29:34.088500+0000","last_clean":"2024-05-08T05:29:34.088500+0000","last_became_active":"2024-05-08T05:25:37.053579+0000","last_became_peered":"2024-05-08T05:25:37.053579+0000","last_unstale":"2024-05-08T05:29:34.088500+0000","last_undegraded":"2024-05-08T05:29:34.088500+0000","last_fullsized":"2024-05-08T05:29:34.088500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:21.748715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933115+0000","last_change":"2024-05-08T05:25:22.648083+0000","last_active":"2024-05-08T05:29:33.933115+0000","last_peered":"2024-05-08T05:29:33.933115+0000","last_clean":"2024-05-08T05:29:33.933115+0000","last_became_active":"2024-05-08T05:25:22.647650+0000","last_became_peered":"2024-05-08T05:25:22.647650+0000","last_unstale":"2024-05-08T05:29:33.933115+0000","last_undegraded":"2024-05-08T05:29:33.933115+0000","last_fullsized":"2024-05-08T05:29:33.933115+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:56.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.824690+0000","last_change":"2024-05-08T05:25:25.668290+0000","last_active":"2024-05-08T05:29:33.824690+0000","last_peered":"2024-05-08T05:29:33.824690+0000","last_clean":"2024-05-08T05:29:33.824690+0000","last_became_active":"2024-05-08T05:25:25.667888+0000","last_became_peered":"2024-05-08T05:25:25.667888+0000","last_unstale":"2024-05-08T05:29:33.824690+0000","last_undegraded":"2024-05-08T05:29:33.824690+0000","last_fullsized":"2024-05-08T05:29:33.824690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:19.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925789+0000","last_change":"2024-05-08T05:25:37.057639+0000","last_active":"2024-05-08T05:29:33.925789+0000","last_peered":"2024-05-08T05:29:33.925789+0000","last_clean":"2024-05-08T05:29:33.925789+0000","last_became_active":"2024-05-08T05:25:37.056848+0000","last_became_peered":"2024-05-08T05:25:37.056848+0000","last_unstale":"2024-05-08T05:29:33.925789+0000","last_undegraded":"2024-05-08T05:29:33.925789+0000","last_fullsized":"2024-05-08T05:29:33.925789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:37.137224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":428,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.678448+0000","last_change":"2024-05-08T05:25:22.649658+0000","last_active":"2024-05-08T05:29:34.678448+0000","last_peered":"2024-05-08T05:29:34.678448+0000","last_clean":"2024-05-08T05:29:34.678448+0000","last_became_active":"2024-05-08T05:25:22.648771+0000","last_became_peered":"2024-05-08T05:25:22.648771+0000","last_unstale":"2024-05-08T05:29:34.678448+0000","last_undegraded":"2024-05-08T05:29:34.678448+0000","last_fullsized":"2024-05-08T05:29:34.678448+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:54.412710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":374,"num_read_kb":374,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826502+0000","last_change":"2024-05-08T05:25:25.673931+0000","last_active":"2024-05-08T05:29:33.826502+0000","last_peered":"2024-05-08T05:29:33.826502+0000","last_clean":"2024-05-08T05:29:33.826502+0000","last_became_active":"2024-05-08T05:25:25.673585+0000","last_became_peered":"2024-05-08T05:25:25.673585+0000","last_unstale":"2024-05-08T05:29:33.826502+0000","last_undegraded":"2024-05-08T05:29:33.826502+0000","last_fullsized":"2024-05-08T05:29:33.826502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:27.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.824556+0000","last_change":"2024-05-08T05:25:37.047651+0000","last_active":"2024-05-08T05:29:33.824556+0000","last_peered":"2024-05-08T05:29:33.824556+0000","last_clean":"2024-05-08T05:29:33.824556+0000","last_became_active":"2024-05-08T05:25:37.047246+0000","last_became_peered":"2024-05-08T05:25:37.047246+0000","last_unstale":"2024-05-08T05:29:33.824556+0000","last_undegraded":"2024-05-08T05:29:33.824556+0000","last_fullsized":"2024-05-08T05:29:33.824556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:22.300076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085560+0000","last_change":"2024-05-08T05:25:22.646973+0000","last_active":"2024-05-08T05:29:34.085560+0000","last_peered":"2024-05-08T05:29:34.085560+0000","last_clean":"2024-05-08T05:29:34.085560+0000","last_became_active":"2024-05-08T05:25:22.646232+0000","last_became_peered":"2024-05-08T05:25:22.646232+0000","last_unstale":"2024-05-08T05:29:34.085560+0000","last_undegraded":"2024-05-08T05:29:34.085560+0000","last_fullsized":"2024-05-08T05:29:34.085560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:38.147470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088524+0000","last_change":"2024-05-08T05:25:25.661373+0000","last_active":"2024-05-08T05:29:34.088524+0000","last_peered":"2024-05-08T05:29:34.088524+0000","last_clean":"2024-05-08T05:29:34.088524+0000","last_became_active":"2024-05-08T05:25:25.661080+0000","last_became_peered":"2024-05-08T05:25:25.661080+0000","last_unstale":"2024-05-08T05:29:34.088524+0000","last_undegraded":"2024-05-08T05:29:34.088524+0000","last_fullsized":"2024-05-08T05:29:34.088524+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:18.718610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093071+0000","last_change":"2024-05-08T05:25:37.051797+0000","last_active":"2024-05-08T05:29:34.093071+0000","last_peered":"2024-05-08T05:29:34.093071+0000","last_clean":"2024-05-08T05:29:34.093071+0000","last_became_active":"2024-05-08T05:25:37.051381+0000","last_became_peered":"2024-05-08T05:25:37.051381+0000","last_unstale":"2024-05-08T05:29:34.093071+0000","last_undegraded":"2024-05-08T05:29:34.093071+0000","last_fullsized":"2024-05-08T05:29:34.093071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:34.337512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933456+0000","last_change":"2024-05-08T05:25:22.748510+0000","last_active":"2024-05-08T05:29:33.933456+0000","last_peered":"2024-05-08T05:29:33.933456+0000","last_clean":"2024-05-08T05:29:33.933456+0000","last_became_active":"2024-05-08T05:25:22.748140+0000","last_became_peered":"2024-05-08T05:25:22.748140+0000","last_unstale":"2024-05-08T05:29:33.933456+0000","last_undegraded":"2024-05-08T05:29:33.933456+0000","last_fullsized":"2024-05-08T05:29:33.933456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:32.325653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826339+0000","last_change":"2024-05-08T05:25:25.671261+0000","last_active":"2024-05-08T05:29:33.826339+0000","last_peered":"2024-05-08T05:29:33.826339+0000","last_clean":"2024-05-08T05:29:33.826339+0000","last_became_active":"2024-05-08T05:25:25.670937+0000","last_became_peered":"2024-05-08T05:25:25.670937+0000","last_unstale":"2024-05-08T05:29:33.826339+0000","last_undegraded":"2024-05-08T05:29:33.826339+0000","last_fullsized":"2024-05-08T05:29:33.826339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:41.375908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936237+0000","last_change":"2024-05-08T05:26:05.150754+0000","last_active":"2024-05-08T05:29:33.936237+0000","last_peered":"2024-05-08T05:29:33.936237+0000","last_clean":"2024-05-08T05:29:33.936237+0000","last_became_active":"2024-05-08T05:26:05.150331+0000","last_became_peered":"2024-05-08T05:26:05.150331+0000","last_unstale":"2024-05-08T05:29:33.936237+0000","last_undegraded":"2024-05-08T05:29:33.936237+0000","last_fullsized":"2024-05-08T05:29:33.936237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:23.309103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'42","reported_seq":475,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.707527+0000","last_change":"2024-05-08T05:25:22.645627+0000","last_active":"2024-05-08T05:29:34.707527+0000","last_peered":"2024-05-08T05:29:34.707527+0000","last_clean":"2024-05-08T05:29:34.707527+0000","last_became_active":"2024-05-08T05:25:22.645243+0000","last_became_peered":"2024-05-08T05:25:22.645243+0000","last_unstale":"2024-05-08T05:29:34.707527+0000","last_undegraded":"2024-05-08T05:29:34.707527+0000","last_fullsized":"2024-05-08T05:29:34.707527+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:23.256903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":69907,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":387,"num_read_kb":263,"num_write":44,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925853+0000","last_change":"2024-05-08T05:25:25.661258+0000","last_active":"2024-05-08T05:29:33.925853+0000","last_peered":"2024-05-08T05:29:33.925853+0000","last_clean":"2024-05-08T05:29:33.925853+0000","last_became_active":"2024-05-08T05:25:25.660925+0000","last_became_peered":"2024-05-08T05:25:25.660925+0000","last_unstale":"2024-05-08T05:29:33.925853+0000","last_undegraded":"2024-05-08T05:29:33.925853+0000","last_fullsized":"2024-05-08T05:29:33.925853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:58.764540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933810+0000","last_change":"2024-05-08T05:25:37.053539+0000","last_active":"2024-05-08T05:29:33.933810+0000","last_peered":"2024-05-08T05:29:33.933810+0000","last_clean":"2024-05-08T05:29:33.933810+0000","last_became_active":"2024-05-08T05:25:37.053102+0000","last_became_peered":"2024-05-08T05:25:37.053102+0000","last_unstale":"2024-05-08T05:29:33.933810+0000","last_undegraded":"2024-05-08T05:29:33.933810+0000","last_fullsized":"2024-05-08T05:29:33.933810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:48.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827052+0000","last_change":"2024-05-08T05:25:22.646651+0000","last_active":"2024-05-08T05:29:33.827052+0000","last_peered":"2024-05-08T05:29:33.827052+0000","last_clean":"2024-05-08T05:29:33.827052+0000","last_became_active":"2024-05-08T05:25:22.646285+0000","last_became_peered":"2024-05-08T05:25:22.646285+0000","last_unstale":"2024-05-08T05:29:33.827052+0000","last_undegraded":"2024-05-08T05:29:33.827052+0000","last_fullsized":"2024-05-08T05:29:33.827052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.843858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680138+0000","last_change":"2024-05-08T05:25:25.666018+0000","last_active":"2024-05-08T05:29:34.680138+0000","last_peered":"2024-05-08T05:29:34.680138+0000","last_clean":"2024-05-08T05:29:34.680138+0000","last_became_active":"2024-05-08T05:25:25.663609+0000","last_became_peered":"2024-05-08T05:25:25.663609+0000","last_unstale":"2024-05-08T05:29:34.680138+0000","last_undegraded":"2024-05-08T05:29:34.680138+0000","last_fullsized":"2024-05-08T05:29:34.680138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:36.223121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087978+0000","last_change":"2024-05-08T05:25:37.052907+0000","last_active":"2024-05-08T05:29:34.087978+0000","last_peered":"2024-05-08T05:29:34.087978+0000","last_clean":"2024-05-08T05:29:34.087978+0000","last_became_active":"2024-05-08T05:25:37.052506+0000","last_became_peered":"2024-05-08T05:25:37.052506+0000","last_unstale":"2024-05-08T05:29:34.087978+0000","last_undegraded":"2024-05-08T05:29:34.087978+0000","last_fullsized":"2024-05-08T05:29:34.087978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:41.356996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933355+0000","last_change":"2024-05-08T05:25:22.646671+0000","last_active":"2024-05-08T05:29:33.933355+0000","last_peered":"2024-05-08T05:29:33.933355+0000","last_clean":"2024-05-08T05:29:33.933355+0000","last_became_active":"2024-05-08T05:25:22.646202+0000","last_became_peered":"2024-05-08T05:25:22.646202+0000","last_unstale":"2024-05-08T05:29:33.933355+0000","last_undegraded":"2024-05-08T05:29:33.933355+0000","last_fullsized":"2024-05-08T05:29:33.933355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:59.224960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085357+0000","last_change":"2024-05-08T05:25:25.666969+0000","last_active":"2024-05-08T05:29:34.085357+0000","last_peered":"2024-05-08T05:29:34.085357+0000","last_clean":"2024-05-08T05:29:34.085357+0000","last_became_active":"2024-05-08T05:25:25.666482+0000","last_became_peered":"2024-05-08T05:25:25.666482+0000","last_unstale":"2024-05-08T05:29:34.085357+0000","last_undegraded":"2024-05-08T05:29:34.085357+0000","last_fullsized":"2024-05-08T05:29:34.085357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:07.819116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825127+0000","last_change":"2024-05-08T05:25:37.055807+0000","last_active":"2024-05-08T05:29:33.825127+0000","last_peered":"2024-05-08T05:29:33.825127+0000","last_clean":"2024-05-08T05:29:33.825127+0000","last_became_active":"2024-05-08T05:25:37.054778+0000","last_became_peered":"2024-05-08T05:25:37.054778+0000","last_unstale":"2024-05-08T05:29:33.825127+0000","last_undegraded":"2024-05-08T05:29:33.825127+0000","last_fullsized":"2024-05-08T05:29:33.825127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:33.767885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'9","reported_seq":253,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:38.685427+0000","last_change":"2024-05-08T05:25:22.647093+0000","last_active":"2024-05-08T05:29:38.685427+0000","last_peered":"2024-05-08T05:29:38.685427+0000","last_clean":"2024-05-08T05:29:38.685427+0000","last_became_active":"2024-05-08T05:25:22.646733+0000","last_became_peered":"2024-05-08T05:25:22.646733+0000","last_unstale":"2024-05-08T05:29:38.685427+0000","last_undegraded":"2024-05-08T05:29:38.685427+0000","last_fullsized":"2024-05-08T05:29:38.685427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:04.346568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1819,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":199,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085169+0000","last_change":"2024-05-08T05:25:25.666565+0000","last_active":"2024-05-08T05:29:34.085169+0000","last_peered":"2024-05-08T05:29:34.085169+0000","last_clean":"2024-05-08T05:29:34.085169+0000","last_became_active":"2024-05-08T05:25:25.666138+0000","last_became_peered":"2024-05-08T05:25:25.666138+0000","last_unstale":"2024-05-08T05:29:34.085169+0000","last_undegraded":"2024-05-08T05:29:34.085169+0000","last_fullsized":"2024-05-08T05:29:34.085169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:05.396530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680587+0000","last_change":"2024-05-08T05:25:37.060933+0000","last_active":"2024-05-08T05:29:34.680587+0000","last_peered":"2024-05-08T05:29:34.680587+0000","last_clean":"2024-05-08T05:29:34.680587+0000","last_became_active":"2024-05-08T05:25:37.060388+0000","last_became_peered":"2024-05-08T05:25:37.060388+0000","last_unstale":"2024-05-08T05:29:34.680587+0000","last_undegraded":"2024-05-08T05:29:34.680587+0000","last_fullsized":"2024-05-08T05:29:34.680587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:09.776051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":81,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925413+0000","last_change":"2024-05-08T05:26:05.156863+0000","last_active":"2024-05-08T05:29:33.925413+0000","last_peered":"2024-05-08T05:29:33.925413+0000","last_clean":"2024-05-08T05:29:33.925413+0000","last_became_active":"2024-05-08T05:26:05.149008+0000","last_became_peered":"2024-05-08T05:26:05.149008+0000","last_unstale":"2024-05-08T05:29:33.925413+0000","last_undegraded":"2024-05-08T05:29:33.925413+0000","last_fullsized":"2024-05-08T05:29:33.925413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:56.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933168+0000","last_change":"2024-05-08T05:25:25.670415+0000","last_active":"2024-05-08T05:29:33.933168+0000","last_peered":"2024-05-08T05:29:33.933168+0000","last_clean":"2024-05-08T05:29:33.933168+0000","last_became_active":"2024-05-08T05:25:25.670095+0000","last_became_peered":"2024-05-08T05:25:25.670095+0000","last_unstale":"2024-05-08T05:29:33.933168+0000","last_undegraded":"2024-05-08T05:29:33.933168+0000","last_fullsized":"2024-05-08T05:29:33.933168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.489203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925444+0000","last_change":"2024-05-08T05:26:05.307721+0000","last_active":"2024-05-08T05:29:33.925444+0000","last_peered":"2024-05-08T05:29:33.925444+0000","last_clean":"2024-05-08T05:29:33.925444+0000","last_became_active":"2024-05-08T05:26:05.307279+0000","last_became_peered":"2024-05-08T05:26:05.307279+0000","last_unstale":"2024-05-08T05:29:33.925444+0000","last_undegraded":"2024-05-08T05:29:33.925444+0000","last_fullsized":"2024-05-08T05:29:33.925444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:08.857770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925842+0000","last_change":"2024-05-08T05:25:22.644279+0000","last_active":"2024-05-08T05:29:33.925842+0000","last_peered":"2024-05-08T05:29:33.925842+0000","last_clean":"2024-05-08T05:29:33.925842+0000","last_became_active":"2024-05-08T05:25:22.643900+0000","last_became_peered":"2024-05-08T05:25:22.643900+0000","last_unstale":"2024-05-08T05:29:33.925842+0000","last_undegraded":"2024-05-08T05:29:33.925842+0000","last_fullsized":"2024-05-08T05:29:33.925842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:32.435843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088135+0000","last_change":"2024-05-08T05:25:25.670130+0000","last_active":"2024-05-08T05:29:34.088135+0000","last_peered":"2024-05-08T05:29:34.088135+0000","last_clean":"2024-05-08T05:29:34.088135+0000","last_became_active":"2024-05-08T05:25:25.669784+0000","last_became_peered":"2024-05-08T05:25:25.669784+0000","last_unstale":"2024-05-08T05:29:34.088135+0000","last_undegraded":"2024-05-08T05:29:34.088135+0000","last_fullsized":"2024-05-08T05:29:34.088135+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:07.188883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925515+0000","last_change":"2024-05-08T05:26:05.308804+0000","last_active":"2024-05-08T05:29:33.925515+0000","last_peered":"2024-05-08T05:29:33.925515+0000","last_clean":"2024-05-08T05:29:33.925515+0000","last_became_active":"2024-05-08T05:26:05.308226+0000","last_became_peered":"2024-05-08T05:26:05.308226+0000","last_unstale":"2024-05-08T05:29:33.925515+0000","last_undegraded":"2024-05-08T05:29:33.925515+0000","last_fullsized":"2024-05-08T05:29:33.925515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:33.442023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.824979+0000","last_change":"2024-05-08T05:25:22.748409+0000","last_active":"2024-05-08T05:29:33.824979+0000","last_peered":"2024-05-08T05:29:33.824979+0000","last_clean":"2024-05-08T05:29:33.824979+0000","last_became_active":"2024-05-08T05:25:22.748062+0000","last_became_peered":"2024-05-08T05:25:22.748062+0000","last_unstale":"2024-05-08T05:29:33.824979+0000","last_undegraded":"2024-05-08T05:29:33.824979+0000","last_fullsized":"2024-05-08T05:29:33.824979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:42.147084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936494+0000","last_change":"2024-05-08T05:25:25.677057+0000","last_active":"2024-05-08T05:29:33.936494+0000","last_peered":"2024-05-08T05:29:33.936494+0000","last_clean":"2024-05-08T05:29:33.936494+0000","last_became_active":"2024-05-08T05:25:25.676697+0000","last_became_peered":"2024-05-08T05:25:25.676697+0000","last_unstale":"2024-05-08T05:29:33.936494+0000","last_undegraded":"2024-05-08T05:29:33.936494+0000","last_fullsized":"2024-05-08T05:29:33.936494+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:40.786327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680220+0000","last_change":"2024-05-08T05:25:37.053903+0000","last_active":"2024-05-08T05:29:34.680220+0000","last_peered":"2024-05-08T05:29:34.680220+0000","last_clean":"2024-05-08T05:29:34.680220+0000","last_became_active":"2024-05-08T05:25:37.053375+0000","last_became_peered":"2024-05-08T05:25:37.053375+0000","last_unstale":"2024-05-08T05:29:34.680220+0000","last_undegraded":"2024-05-08T05:29:34.680220+0000","last_fullsized":"2024-05-08T05:29:34.680220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:19.374355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"90'7","reported_seq":245,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.678456+0000","last_change":"2024-05-08T05:25:22.646931+0000","last_active":"2024-05-08T05:29:34.678456+0000","last_peered":"2024-05-08T05:29:34.678456+0000","last_clean":"2024-05-08T05:29:34.678456+0000","last_became_active":"2024-05-08T05:25:22.646599+0000","last_became_peered":"2024-05-08T05:25:22.646599+0000","last_unstale":"2024-05-08T05:29:34.678456+0000","last_undegraded":"2024-05-08T05:29:34.678456+0000","last_fullsized":"2024-05-08T05:29:34.678456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:55.873089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":190,"num_read_kb":3,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936332+0000","last_change":"2024-05-08T05:26:05.147977+0000","last_active":"2024-05-08T05:29:33.936332+0000","last_peered":"2024-05-08T05:29:33.936332+0000","last_clean":"2024-05-08T05:29:33.936332+0000","last_became_active":"2024-05-08T05:26:05.147637+0000","last_became_peered":"2024-05-08T05:26:05.147637+0000","last_unstale":"2024-05-08T05:29:33.936332+0000","last_undegraded":"2024-05-08T05:29:33.936332+0000","last_fullsized":"2024-05-08T05:29:33.936332+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:59.671306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680429+0000","last_change":"2024-05-08T05:25:37.057913+0000","last_active":"2024-05-08T05:29:34.680429+0000","last_peered":"2024-05-08T05:29:34.680429+0000","last_clean":"2024-05-08T05:29:34.680429+0000","last_became_active":"2024-05-08T05:25:37.057474+0000","last_became_peered":"2024-05-08T05:25:37.057474+0000","last_unstale":"2024-05-08T05:29:34.680429+0000","last_undegraded":"2024-05-08T05:29:34.680429+0000","last_fullsized":"2024-05-08T05:29:34.680429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:19.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933446+0000","last_change":"2024-05-08T05:25:25.663594+0000","last_active":"2024-05-08T05:29:33.933446+0000","last_peered":"2024-05-08T05:29:33.933446+0000","last_clean":"2024-05-08T05:29:33.933446+0000","last_became_active":"2024-05-08T05:25:25.663275+0000","last_became_peered":"2024-05-08T05:25:25.663275+0000","last_unstale":"2024-05-08T05:29:33.933446+0000","last_undegraded":"2024-05-08T05:29:33.933446+0000","last_fullsized":"2024-05-08T05:29:33.933446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:20.581166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":243,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680604+0000","last_change":"2024-05-08T05:25:22.748316+0000","last_active":"2024-05-08T05:29:34.680604+0000","last_peered":"2024-05-08T05:29:34.680604+0000","last_clean":"2024-05-08T05:29:34.680604+0000","last_became_active":"2024-05-08T05:25:22.747819+0000","last_became_peered":"2024-05-08T05:25:22.747819+0000","last_unstale":"2024-05-08T05:29:34.680604+0000","last_undegraded":"2024-05-08T05:29:34.680604+0000","last_fullsized":"2024-05-08T05:29:34.680604+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:50:42.689853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":188,"num_read_kb":188,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085575+0000","last_change":"2024-05-08T05:25:37.058271+0000","last_active":"2024-05-08T05:29:34.085575+0000","last_peered":"2024-05-08T05:29:34.085575+0000","last_clean":"2024-05-08T05:29:34.085575+0000","last_became_active":"2024-05-08T05:25:37.057290+0000","last_became_peered":"2024-05-08T05:25:37.057290+0000","last_unstale":"2024-05-08T05:29:34.085575+0000","last_undegraded":"2024-05-08T05:29:34.085575+0000","last_fullsized":"2024-05-08T05:29:34.085575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:47.369909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925931+0000","last_change":"2024-05-08T05:25:25.666000+0000","last_active":"2024-05-08T05:29:33.925931+0000","last_peered":"2024-05-08T05:29:33.925931+0000","last_clean":"2024-05-08T05:29:33.925931+0000","last_became_active":"2024-05-08T05:25:25.665663+0000","last_became_peered":"2024-05-08T05:25:25.665663+0000","last_unstale":"2024-05-08T05:29:33.925931+0000","last_undegraded":"2024-05-08T05:29:33.925931+0000","last_fullsized":"2024-05-08T05:29:33.925931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:44.328879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093038+0000","last_change":"2024-05-08T05:25:22.648770+0000","last_active":"2024-05-08T05:29:34.093038+0000","last_peered":"2024-05-08T05:29:34.093038+0000","last_clean":"2024-05-08T05:29:34.093038+0000","last_became_active":"2024-05-08T05:25:22.648119+0000","last_became_peered":"2024-05-08T05:25:22.648119+0000","last_unstale":"2024-05-08T05:29:34.093038+0000","last_undegraded":"2024-05-08T05:29:34.093038+0000","last_fullsized":"2024-05-08T05:29:34.093038+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:55.813080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"21'92","reported_seq":122,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085023+0000","last_change":"2024-05-08T05:23:55.820436+0000","last_active":"2024-05-08T05:29:34.085023+0000","last_peered":"2024-05-08T05:29:34.085023+0000","last_clean":"2024-05-08T05:29:34.085023+0000","last_became_active":"2024-05-08T05:23:55.814786+0000","last_became_peered":"2024-05-08T05:23:55.814786+0000","last_unstale":"2024-05-08T05:29:34.085023+0000","last_undegraded":"2024-05-08T05:29:34.085023+0000","last_fullsized":"2024-05-08T05:29:34.085023+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_clean_scrub_stamp":"2024-05-08T05:20:32.659283+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:11.169455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933610+0000","last_change":"2024-05-08T05:25:37.057287+0000","last_active":"2024-05-08T05:29:33.933610+0000","last_peered":"2024-05-08T05:29:33.933610+0000","last_clean":"2024-05-08T05:29:33.933610+0000","last_became_active":"2024-05-08T05:25:37.056871+0000","last_became_peered":"2024-05-08T05:25:37.056871+0000","last_unstale":"2024-05-08T05:29:33.933610+0000","last_undegraded":"2024-05-08T05:29:33.933610+0000","last_fullsized":"2024-05-08T05:29:33.933610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:19.004199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.679912+0000","last_change":"2024-05-08T05:25:25.664795+0000","last_active":"2024-05-08T05:29:34.679912+0000","last_peered":"2024-05-08T05:29:34.679912+0000","last_clean":"2024-05-08T05:29:34.679912+0000","last_became_active":"2024-05-08T05:25:25.664444+0000","last_became_peered":"2024-05-08T05:25:25.664444+0000","last_unstale":"2024-05-08T05:29:34.679912+0000","last_undegraded":"2024-05-08T05:29:34.679912+0000","last_fullsized":"2024-05-08T05:29:34.679912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:27.172530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085169+0000","last_change":"2024-05-08T05:25:22.645205+0000","last_active":"2024-05-08T05:29:34.085169+0000","last_peered":"2024-05-08T05:29:34.085169+0000","last_clean":"2024-05-08T05:29:34.085169+0000","last_became_active":"2024-05-08T05:25:22.644720+0000","last_became_peered":"2024-05-08T05:25:22.644720+0000","last_unstale":"2024-05-08T05:29:34.085169+0000","last_undegraded":"2024-05-08T05:29:34.085169+0000","last_fullsized":"2024-05-08T05:29:34.085169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:05.630188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087942+0000","last_change":"2024-05-08T05:25:37.055151+0000","last_active":"2024-05-08T05:29:34.087942+0000","last_peered":"2024-05-08T05:29:34.087942+0000","last_clean":"2024-05-08T05:29:34.087942+0000","last_became_active":"2024-05-08T05:25:37.054737+0000","last_became_peered":"2024-05-08T05:25:37.054737+0000","last_unstale":"2024-05-08T05:29:34.087942+0000","last_undegraded":"2024-05-08T05:29:34.087942+0000","last_fullsized":"2024-05-08T05:29:34.087942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:49.135453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088311+0000","last_change":"2024-05-08T05:26:05.148665+0000","last_active":"2024-05-08T05:29:34.088311+0000","last_peered":"2024-05-08T05:29:34.088311+0000","last_clean":"2024-05-08T05:29:34.088311+0000","last_became_active":"2024-05-08T05:26:05.148283+0000","last_became_peered":"2024-05-08T05:26:05.148283+0000","last_unstale":"2024-05-08T05:29:34.088311+0000","last_undegraded":"2024-05-08T05:29:34.088311+0000","last_fullsized":"2024-05-08T05:29:34.088311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:26.197521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680351+0000","last_change":"2024-05-08T05:25:25.669130+0000","last_active":"2024-05-08T05:29:34.680351+0000","last_peered":"2024-05-08T05:29:34.680351+0000","last_clean":"2024-05-08T05:29:34.680351+0000","last_became_active":"2024-05-08T05:25:25.668812+0000","last_became_peered":"2024-05-08T05:25:25.668812+0000","last_unstale":"2024-05-08T05:29:34.680351+0000","last_undegraded":"2024-05-08T05:29:34.680351+0000","last_fullsized":"2024-05-08T05:29:34.680351+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:34.514248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925554+0000","last_change":"2024-05-08T05:25:37.057146+0000","last_active":"2024-05-08T05:29:33.925554+0000","last_peered":"2024-05-08T05:29:33.925554+0000","last_clean":"2024-05-08T05:29:33.925554+0000","last_became_active":"2024-05-08T05:25:37.056706+0000","last_became_peered":"2024-05-08T05:25:37.056706+0000","last_unstale":"2024-05-08T05:29:33.925554+0000","last_undegraded":"2024-05-08T05:29:33.925554+0000","last_fullsized":"2024-05-08T05:29:33.925554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:09.724088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'51","reported_seq":120,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.048962+0000","last_change":"2024-05-08T05:25:22.644573+0000","last_active":"2024-05-08T05:29:34.048962+0000","last_peered":"2024-05-08T05:29:34.048962+0000","last_clean":"2024-05-08T05:29:34.048962+0000","last_became_active":"2024-05-08T05:25:22.644231+0000","last_became_peered":"2024-05-08T05:25:22.644231+0000","last_unstale":"2024-05-08T05:29:34.048962+0000","last_undegraded":"2024-05-08T05:29:34.048962+0000","last_fullsized":"2024-05-08T05:29:34.048962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":51,"log_dups_size":0,"ondisk_log_size":51,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:02.931823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":91,"num_write_kb":838,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925815+0000","last_change":"2024-05-08T05:25:25.671036+0000","last_active":"2024-05-08T05:29:33.925815+0000","last_peered":"2024-05-08T05:29:33.925815+0000","last_clean":"2024-05-08T05:29:33.925815+0000","last_became_active":"2024-05-08T05:25:25.669170+0000","last_became_peered":"2024-05-08T05:25:25.669170+0000","last_unstale":"2024-05-08T05:29:33.925815+0000","last_undegraded":"2024-05-08T05:29:33.925815+0000","last_fullsized":"2024-05-08T05:29:33.925815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:18.963232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826699+0000","last_change":"2024-05-08T05:25:37.050656+0000","last_active":"2024-05-08T05:29:33.826699+0000","last_peered":"2024-05-08T05:29:33.826699+0000","last_clean":"2024-05-08T05:29:33.826699+0000","last_became_active":"2024-05-08T05:25:37.050051+0000","last_became_peered":"2024-05-08T05:25:37.050051+0000","last_unstale":"2024-05-08T05:29:33.826699+0000","last_undegraded":"2024-05-08T05:29:33.826699+0000","last_fullsized":"2024-05-08T05:29:33.826699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:56:07.316216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093378+0000","last_change":"2024-05-08T05:25:23.614874+0000","last_active":"2024-05-08T05:29:34.093378+0000","last_peered":"2024-05-08T05:29:34.093378+0000","last_clean":"2024-05-08T05:29:34.093378+0000","last_became_active":"2024-05-08T05:25:23.614550+0000","last_became_peered":"2024-05-08T05:25:23.614550+0000","last_unstale":"2024-05-08T05:29:34.093378+0000","last_undegraded":"2024-05-08T05:29:34.093378+0000","last_fullsized":"2024-05-08T05:29:34.093378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:10.183477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088730+0000","last_change":"2024-05-08T05:25:25.658710+0000","last_active":"2024-05-08T05:29:34.088730+0000","last_peered":"2024-05-08T05:29:34.088730+0000","last_clean":"2024-05-08T05:29:34.088730+0000","last_became_active":"2024-05-08T05:25:25.658393+0000","last_became_peered":"2024-05-08T05:25:25.658393+0000","last_unstale":"2024-05-08T05:29:34.088730+0000","last_undegraded":"2024-05-08T05:29:34.088730+0000","last_fullsized":"2024-05-08T05:29:34.088730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:02.629256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092991+0000","last_change":"2024-05-08T05:25:37.059976+0000","last_active":"2024-05-08T05:29:34.092991+0000","last_peered":"2024-05-08T05:29:34.092991+0000","last_clean":"2024-05-08T05:29:34.092991+0000","last_became_active":"2024-05-08T05:25:37.059474+0000","last_became_peered":"2024-05-08T05:25:37.059474+0000","last_unstale":"2024-05-08T05:29:34.092991+0000","last_undegraded":"2024-05-08T05:29:34.092991+0000","last_fullsized":"2024-05-08T05:29:34.092991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:51.946242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825833+0000","last_change":"2024-05-08T05:25:22.748271+0000","last_active":"2024-05-08T05:29:33.825833+0000","last_peered":"2024-05-08T05:29:33.825833+0000","last_clean":"2024-05-08T05:29:33.825833+0000","last_became_active":"2024-05-08T05:25:22.747915+0000","last_became_peered":"2024-05-08T05:25:22.747915+0000","last_unstale":"2024-05-08T05:29:33.825833+0000","last_undegraded":"2024-05-08T05:29:33.825833+0000","last_fullsized":"2024-05-08T05:29:33.825833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:35.262298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925478+0000","last_change":"2024-05-08T05:25:25.668722+0000","last_active":"2024-05-08T05:29:33.925478+0000","last_peered":"2024-05-08T05:29:33.925478+0000","last_clean":"2024-05-08T05:29:33.925478+0000","last_became_active":"2024-05-08T05:25:25.668223+0000","last_became_peered":"2024-05-08T05:25:25.668223+0000","last_unstale":"2024-05-08T05:29:33.925478+0000","last_undegraded":"2024-05-08T05:29:33.925478+0000","last_fullsized":"2024-05-08T05:29:33.925478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:43.511567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825206+0000","last_change":"2024-05-08T05:25:37.056131+0000","last_active":"2024-05-08T05:29:33.825206+0000","last_peered":"2024-05-08T05:29:33.825206+0000","last_clean":"2024-05-08T05:29:33.825206+0000","last_became_active":"2024-05-08T05:25:37.055424+0000","last_became_peered":"2024-05-08T05:25:37.055424+0000","last_unstale":"2024-05-08T05:29:33.825206+0000","last_undegraded":"2024-05-08T05:29:33.825206+0000","last_fullsized":"2024-05-08T05:29:33.825206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:43.279408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925896+0000","last_change":"2024-05-08T05:25:22.643908+0000","last_active":"2024-05-08T05:29:33.925896+0000","last_peered":"2024-05-08T05:29:33.925896+0000","last_clean":"2024-05-08T05:29:33.925896+0000","last_became_active":"2024-05-08T05:25:22.643542+0000","last_became_peered":"2024-05-08T05:25:22.643542+0000","last_unstale":"2024-05-08T05:29:33.925896+0000","last_undegraded":"2024-05-08T05:29:33.925896+0000","last_fullsized":"2024-05-08T05:29:33.925896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:01.671285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087815+0000","last_change":"2024-05-08T05:25:25.667843+0000","last_active":"2024-05-08T05:29:34.087815+0000","last_peered":"2024-05-08T05:29:34.087815+0000","last_clean":"2024-05-08T05:29:34.087815+0000","last_became_active":"2024-05-08T05:25:25.667462+0000","last_became_peered":"2024-05-08T05:25:25.667462+0000","last_unstale":"2024-05-08T05:29:34.087815+0000","last_undegraded":"2024-05-08T05:29:34.087815+0000","last_fullsized":"2024-05-08T05:29:34.087815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:51.325104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092927+0000","last_change":"2024-05-08T05:25:37.053017+0000","last_active":"2024-05-08T05:29:34.092927+0000","last_peered":"2024-05-08T05:29:34.092927+0000","last_clean":"2024-05-08T05:29:34.092927+0000","last_became_active":"2024-05-08T05:25:37.052601+0000","last_became_peered":"2024-05-08T05:25:37.052601+0000","last_unstale":"2024-05-08T05:29:34.092927+0000","last_undegraded":"2024-05-08T05:29:34.092927+0000","last_fullsized":"2024-05-08T05:29:34.092927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:35.393835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826911+0000","last_change":"2024-05-08T05:25:22.644564+0000","last_active":"2024-05-08T05:29:33.826911+0000","last_peered":"2024-05-08T05:29:33.826911+0000","last_clean":"2024-05-08T05:29:33.826911+0000","last_became_active":"2024-05-08T05:25:22.644170+0000","last_became_peered":"2024-05-08T05:25:22.644170+0000","last_unstale":"2024-05-08T05:29:33.826911+0000","last_undegraded":"2024-05-08T05:29:33.826911+0000","last_fullsized":"2024-05-08T05:29:33.826911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:24.757574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926667+0000","last_change":"2024-05-08T05:25:25.676983+0000","last_active":"2024-05-08T05:29:33.926667+0000","last_peered":"2024-05-08T05:29:33.926667+0000","last_clean":"2024-05-08T05:29:33.926667+0000","last_became_active":"2024-05-08T05:25:25.676610+0000","last_became_peered":"2024-05-08T05:25:25.676610+0000","last_unstale":"2024-05-08T05:29:33.926667+0000","last_undegraded":"2024-05-08T05:29:33.926667+0000","last_fullsized":"2024-05-08T05:29:33.926667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:39.137935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092999+0000","last_change":"2024-05-08T05:25:37.059381+0000","last_active":"2024-05-08T05:29:34.092999+0000","last_peered":"2024-05-08T05:29:34.092999+0000","last_clean":"2024-05-08T05:29:34.092999+0000","last_became_active":"2024-05-08T05:25:37.056851+0000","last_became_peered":"2024-05-08T05:25:37.056851+0000","last_unstale":"2024-05-08T05:29:34.092999+0000","last_undegraded":"2024-05-08T05:29:34.092999+0000","last_fullsized":"2024-05-08T05:29:34.092999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:19.339331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925636+0000","last_change":"2024-05-08T05:25:22.648099+0000","last_active":"2024-05-08T05:29:33.925636+0000","last_peered":"2024-05-08T05:29:33.925636+0000","last_clean":"2024-05-08T05:29:33.925636+0000","last_became_active":"2024-05-08T05:25:22.647736+0000","last_became_peered":"2024-05-08T05:25:22.647736+0000","last_unstale":"2024-05-08T05:29:33.925636+0000","last_undegraded":"2024-05-08T05:29:33.925636+0000","last_fullsized":"2024-05-08T05:29:33.925636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:12.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085452+0000","last_change":"2024-05-08T05:25:25.667565+0000","last_active":"2024-05-08T05:29:34.085452+0000","last_peered":"2024-05-08T05:29:34.085452+0000","last_clean":"2024-05-08T05:29:34.085452+0000","last_became_active":"2024-05-08T05:25:25.667217+0000","last_became_peered":"2024-05-08T05:25:25.667217+0000","last_unstale":"2024-05-08T05:29:34.085452+0000","last_undegraded":"2024-05-08T05:29:34.085452+0000","last_fullsized":"2024-05-08T05:29:34.085452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:05.228227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825918+0000","last_change":"2024-05-08T05:25:37.056923+0000","last_active":"2024-05-08T05:29:33.825918+0000","last_peered":"2024-05-08T05:29:33.825918+0000","last_clean":"2024-05-08T05:29:33.825918+0000","last_became_active":"2024-05-08T05:25:37.056496+0000","last_became_peered":"2024-05-08T05:25:37.056496+0000","last_unstale":"2024-05-08T05:29:33.825918+0000","last_undegraded":"2024-05-08T05:29:33.825918+0000","last_fullsized":"2024-05-08T05:29:33.825918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:21.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'8","reported_seq":246,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.678997+0000","last_change":"2024-05-08T05:25:23.614900+0000","last_active":"2024-05-08T05:29:34.678997+0000","last_peered":"2024-05-08T05:29:34.678997+0000","last_clean":"2024-05-08T05:29:34.678997+0000","last_became_active":"2024-05-08T05:25:23.614510+0000","last_became_peered":"2024-05-08T05:25:23.614510+0000","last_unstale":"2024-05-08T05:29:34.678997+0000","last_undegraded":"2024-05-08T05:29:34.678997+0000","last_fullsized":"2024-05-08T05:29:34.678997+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:19.539656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2093,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":192,"num_read_kb":5,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085833+0000","last_change":"2024-05-08T05:25:25.669016+0000","last_active":"2024-05-08T05:29:34.085833+0000","last_peered":"2024-05-08T05:29:34.085833+0000","last_clean":"2024-05-08T05:29:34.085833+0000","last_became_active":"2024-05-08T05:25:25.667809+0000","last_became_peered":"2024-05-08T05:25:25.667809+0000","last_unstale":"2024-05-08T05:29:34.085833+0000","last_undegraded":"2024-05-08T05:29:34.085833+0000","last_fullsized":"2024-05-08T05:29:34.085833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:56.475622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088613+0000","last_change":"2024-05-08T05:25:37.053630+0000","last_active":"2024-05-08T05:29:34.088613+0000","last_peered":"2024-05-08T05:29:34.088613+0000","last_clean":"2024-05-08T05:29:34.088613+0000","last_became_active":"2024-05-08T05:25:37.053069+0000","last_became_peered":"2024-05-08T05:25:37.053069+0000","last_unstale":"2024-05-08T05:29:34.088613+0000","last_undegraded":"2024-05-08T05:29:34.088613+0000","last_fullsized":"2024-05-08T05:29:34.088613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:49:56.693447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'41","reported_seq":96,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.046857+0000","last_change":"2024-05-08T05:25:22.645284+0000","last_active":"2024-05-08T05:29:34.046857+0000","last_peered":"2024-05-08T05:29:34.046857+0000","last_clean":"2024-05-08T05:29:34.046857+0000","last_became_active":"2024-05-08T05:25:22.644897+0000","last_became_peered":"2024-05-08T05:25:22.644897+0000","last_unstale":"2024-05-08T05:29:34.046857+0000","last_undegraded":"2024-05-08T05:29:34.046857+0000","last_fullsized":"2024-05-08T05:29:34.046857+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:53:31.423412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085358+0000","last_change":"2024-05-08T05:25:25.664930+0000","last_active":"2024-05-08T05:29:34.085358+0000","last_peered":"2024-05-08T05:29:34.085358+0000","last_clean":"2024-05-08T05:29:34.085358+0000","last_became_active":"2024-05-08T05:25:25.664395+0000","last_became_peered":"2024-05-08T05:25:25.664395+0000","last_unstale":"2024-05-08T05:29:34.085358+0000","last_undegraded":"2024-05-08T05:29:34.085358+0000","last_fullsized":"2024-05-08T05:29:34.085358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:37.971474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933025+0000","last_change":"2024-05-08T05:25:37.053258+0000","last_active":"2024-05-08T05:29:33.933025+0000","last_peered":"2024-05-08T05:29:33.933025+0000","last_clean":"2024-05-08T05:29:33.933025+0000","last_became_active":"2024-05-08T05:25:37.052786+0000","last_became_peered":"2024-05-08T05:25:37.052786+0000","last_unstale":"2024-05-08T05:29:33.933025+0000","last_undegraded":"2024-05-08T05:29:33.933025+0000","last_fullsized":"2024-05-08T05:29:33.933025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:07.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680512+0000","last_change":"2024-05-08T05:27:05.770358+0000","last_active":"2024-05-08T05:29:34.680512+0000","last_peered":"2024-05-08T05:29:34.680512+0000","last_clean":"2024-05-08T05:29:34.680512+0000","last_became_active":"2024-05-08T05:27:05.766778+0000","last_became_peered":"2024-05-08T05:27:05.766778+0000","last_unstale":"2024-05-08T05:29:34.680512+0000","last_undegraded":"2024-05-08T05:29:34.680512+0000","last_fullsized":"2024-05-08T05:29:34.680512+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:48:52.941549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085900+0000","last_change":"2024-05-08T05:25:25.663771+0000","last_active":"2024-05-08T05:29:34.085900+0000","last_peered":"2024-05-08T05:29:34.085900+0000","last_clean":"2024-05-08T05:29:34.085900+0000","last_became_active":"2024-05-08T05:25:25.663281+0000","last_became_peered":"2024-05-08T05:25:25.663281+0000","last_unstale":"2024-05-08T05:29:34.085900+0000","last_undegraded":"2024-05-08T05:29:34.085900+0000","last_fullsized":"2024-05-08T05:29:34.085900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:43.892781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087991+0000","last_change":"2024-05-08T05:25:25.665446+0000","last_active":"2024-05-08T05:29:34.087991+0000","last_peered":"2024-05-08T05:29:34.087991+0000","last_clean":"2024-05-08T05:29:34.087991+0000","last_became_active":"2024-05-08T05:25:25.664389+0000","last_became_peered":"2024-05-08T05:25:25.664389+0000","last_unstale":"2024-05-08T05:29:34.087991+0000","last_undegraded":"2024-05-08T05:29:34.087991+0000","last_fullsized":"2024-05-08T05:29:34.087991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826995+0000","last_change":"2024-05-08T05:26:05.149870+0000","last_active":"2024-05-08T05:29:33.826995+0000","last_peered":"2024-05-08T05:29:33.826995+0000","last_clean":"2024-05-08T05:29:33.826995+0000","last_became_active":"2024-05-08T05:26:05.149476+0000","last_became_peered":"2024-05-08T05:26:05.149476+0000","last_unstale":"2024-05-08T05:29:33.826995+0000","last_undegraded":"2024-05-08T05:29:33.826995+0000","last_fullsized":"2024-05-08T05:29:33.826995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:40:43.401832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826865+0000","last_change":"2024-05-08T05:25:25.667836+0000","last_active":"2024-05-08T05:29:33.826865+0000","last_peered":"2024-05-08T05:29:33.826865+0000","last_clean":"2024-05-08T05:29:33.826865+0000","last_became_active":"2024-05-08T05:25:25.667496+0000","last_became_peered":"2024-05-08T05:25:25.667496+0000","last_unstale":"2024-05-08T05:29:33.826865+0000","last_undegraded":"2024-05-08T05:29:33.826865+0000","last_fullsized":"2024-05-08T05:29:33.826865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:01.482837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.089014+0000","last_change":"2024-05-08T05:25:25.667279+0000","last_active":"2024-05-08T05:29:34.089014+0000","last_peered":"2024-05-08T05:29:34.089014+0000","last_clean":"2024-05-08T05:29:34.089014+0000","last_became_active":"2024-05-08T05:25:25.666960+0000","last_became_peered":"2024-05-08T05:25:25.666960+0000","last_unstale":"2024-05-08T05:29:34.089014+0000","last_undegraded":"2024-05-08T05:29:34.089014+0000","last_fullsized":"2024-05-08T05:29:34.089014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:06.335850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680319+0000","last_change":"2024-05-08T05:25:25.667644+0000","last_active":"2024-05-08T05:29:34.680319+0000","last_peered":"2024-05-08T05:29:34.680319+0000","last_clean":"2024-05-08T05:29:34.680319+0000","last_became_active":"2024-05-08T05:25:25.667133+0000","last_became_peered":"2024-05-08T05:25:25.667133+0000","last_unstale":"2024-05-08T05:29:34.680319+0000","last_undegraded":"2024-05-08T05:29:34.680319+0000","last_fullsized":"2024-05-08T05:29:34.680319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:03.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087979+0000","last_change":"2024-05-08T05:25:25.670761+0000","last_active":"2024-05-08T05:29:34.087979+0000","last_peered":"2024-05-08T05:29:34.087979+0000","last_clean":"2024-05-08T05:29:34.087979+0000","last_became_active":"2024-05-08T05:25:25.670342+0000","last_became_peered":"2024-05-08T05:25:25.670342+0000","last_unstale":"2024-05-08T05:29:34.087979+0000","last_undegraded":"2024-05-08T05:29:34.087979+0000","last_fullsized":"2024-05-08T05:29:34.087979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:08.069332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827277+0000","last_change":"2024-05-08T05:25:25.664512+0000","last_active":"2024-05-08T05:29:33.827277+0000","last_peered":"2024-05-08T05:29:33.827277+0000","last_clean":"2024-05-08T05:29:33.827277+0000","last_became_active":"2024-05-08T05:25:25.664191+0000","last_became_peered":"2024-05-08T05:25:25.664191+0000","last_unstale":"2024-05-08T05:29:33.827277+0000","last_undegraded":"2024-05-08T05:29:33.827277+0000","last_fullsized":"2024-05-08T05:29:33.827277+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:07.586637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.927122+0000","last_change":"2024-05-08T05:25:25.675711+0000","last_active":"2024-05-08T05:29:33.927122+0000","last_peered":"2024-05-08T05:29:33.927122+0000","last_clean":"2024-05-08T05:29:33.927122+0000","last_became_active":"2024-05-08T05:25:25.671890+0000","last_became_peered":"2024-05-08T05:25:25.671890+0000","last_unstale":"2024-05-08T05:29:33.927122+0000","last_undegraded":"2024-05-08T05:29:33.927122+0000","last_fullsized":"2024-05-08T05:29:33.927122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:57:49.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933145+0000","last_change":"2024-05-08T05:25:25.662603+0000","last_active":"2024-05-08T05:29:33.933145+0000","last_peered":"2024-05-08T05:29:33.933145+0000","last_clean":"2024-05-08T05:29:33.933145+0000","last_became_active":"2024-05-08T05:25:25.662287+0000","last_became_peered":"2024-05-08T05:25:25.662287+0000","last_unstale":"2024-05-08T05:29:33.933145+0000","last_undegraded":"2024-05-08T05:29:33.933145+0000","last_fullsized":"2024-05-08T05:29:33.933145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:02.152557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926090+0000","last_change":"2024-05-08T05:25:25.667309+0000","last_active":"2024-05-08T05:29:33.926090+0000","last_peered":"2024-05-08T05:29:33.926090+0000","last_clean":"2024-05-08T05:29:33.926090+0000","last_became_active":"2024-05-08T05:25:25.666968+0000","last_became_peered":"2024-05-08T05:25:25.666968+0000","last_unstale":"2024-05-08T05:29:33.926090+0000","last_undegraded":"2024-05-08T05:29:33.926090+0000","last_fullsized":"2024-05-08T05:29:33.926090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:28.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826130+0000","last_change":"2024-05-08T05:25:25.673579+0000","last_active":"2024-05-08T05:29:33.826130+0000","last_peered":"2024-05-08T05:29:33.826130+0000","last_clean":"2024-05-08T05:29:33.826130+0000","last_became_active":"2024-05-08T05:25:25.673130+0000","last_became_peered":"2024-05-08T05:25:25.673130+0000","last_unstale":"2024-05-08T05:29:33.826130+0000","last_undegraded":"2024-05-08T05:29:33.826130+0000","last_fullsized":"2024-05-08T05:29:33.826130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:01.291319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680150+0000","last_change":"2024-05-08T05:25:25.667359+0000","last_active":"2024-05-08T05:29:34.680150+0000","last_peered":"2024-05-08T05:29:34.680150+0000","last_clean":"2024-05-08T05:29:34.680150+0000","last_became_active":"2024-05-08T05:25:25.667044+0000","last_became_peered":"2024-05-08T05:25:25.667044+0000","last_unstale":"2024-05-08T05:29:34.680150+0000","last_undegraded":"2024-05-08T05:29:34.680150+0000","last_fullsized":"2024-05-08T05:29:34.680150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:54.973053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088126+0000","last_change":"2024-05-08T05:26:05.150407+0000","last_active":"2024-05-08T05:29:34.088126+0000","last_peered":"2024-05-08T05:29:34.088126+0000","last_clean":"2024-05-08T05:29:34.088126+0000","last_became_active":"2024-05-08T05:26:05.149916+0000","last_became_peered":"2024-05-08T05:26:05.149916+0000","last_unstale":"2024-05-08T05:29:34.088126+0000","last_undegraded":"2024-05-08T05:29:34.088126+0000","last_fullsized":"2024-05-08T05:29:34.088126+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:05:57.075928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107400,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":605145,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2020,"num_read_kb":1130,"num_write":390,"num_write_kb":1482,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":753664,"data_stored":2015760,"data_compressed":85095,"data_compressed_allocated":110592,"data_compressed_original":1732608,"omap_allocated":0,"internal_metadata":0},"log_size":315,"ondisk_log_size":315,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678081,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":531921,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874310,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28176,"kb_used_data":1160,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716976,"statfs":{"total":95995035648,"available":95966183424,"internally_reserved":0,"allocated":1187840,"data_stored":1320440,"data_compressed":77386,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037836,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28060,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717092,"statfs":{"total":95995035648,"available":95966302208,"internally_reserved":0,"allocated":1064960,"data_stored":928128,"data_compressed":54576,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234065,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32204,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712948,"statfs":{"total":95995035648,"available":95962058752,"internally_reserved":0,"allocated":1048576,"data_stored":924298,"data_compressed":54576,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462997,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1085440,"data_stored":1130377,"data_compressed":71230,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626523,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":516986,"data_compressed":32462,"data_compressed_allocated":45056,"data_compressed_original":90112,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":601091,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":529090,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1072,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1097728,"data_stored":599602,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182507,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1247476,"data_compressed":69921,"data_compressed_allocated":94208,"data_compressed_original":798720,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986279,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":531737,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822752,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32264,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712888,"statfs":{"total":95995035648,"available":95961997312,"internally_reserved":0,"allocated":1114112,"data_stored":1255796,"data_compressed":77386,"data_compressed_allocated":102400,"data_compressed_original":811008,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":75493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":74004,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":135734,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1254,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19146,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":143360,"data_stored":204474,"data_compressed":6156,"data_compressed_allocated":8192,"data_compressed_original":122880,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2227,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:29:40.806 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:29:41.062 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:41.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:41 smithi059 ceph-mon[30304]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 1 op/s 2024-05-08T05:29:41.327 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:41 smithi059 ceph-mon[30304]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:29:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:41 smithi143 ceph-mon[34337]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 1 op/s 2024-05-08T05:29:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:41 smithi143 ceph-mon[34337]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:29:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:41 smithi096 ceph-mon[34253]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 0 B/s rd, 1 op/s 2024-05-08T05:29:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:41 smithi096 ceph-mon[34253]: from='client.15384 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:29:42.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.824+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/155891228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4104130 msgr2=0x7f0fa4108140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:42.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.824+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/155891228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4104130 0x7f0fa4108140 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c0099c0 tx=0x7f0f8c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:29:42.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.824+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/155891228 shutdown_connections 2024-05-08T05:29:42.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.824+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/155891228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa4108870 0x7f0fa41103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:42.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.824+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/155891228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa4103780 0x7f0fa4103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:42.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.824+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/155891228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4104130 0x7f0fa4108140 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:42.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.824+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/155891228 >> 172.21.15.59:0/155891228 conn(0x7f0fa407a610 msgr2=0x7f0fa407aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:42.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.824+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/155891228 shutdown_connections 2024-05-08T05:29:42.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.824+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/155891228 wait complete. 2024-05-08T05:29:42.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa8aa3700 1 Processor -- start 2024-05-08T05:29:42.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa8aa3700 1 -- start start 2024-05-08T05:29:42.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa8aa3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4103780 0x7f0fa419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:42.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa8aa3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa4104130 0x7f0fa419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:42.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa8aa3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa4108870 0x7f0fa4199930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:42.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa8aa3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa4113630 con 0x7f0fa4103780 2024-05-08T05:29:42.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa8aa3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41134b0 con 0x7f0fa4108870 2024-05-08T05:29:42.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa8aa3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0fa41137b0 con 0x7f0fa4104130 2024-05-08T05:29:42.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4103780 0x7f0fa419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:42.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4103780 0x7f0fa419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53790/0 (socket says 172.21.15.59:53790) 2024-05-08T05:29:42.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa259c700 1 -- 172.21.15.59:0/2108495818 learned_addr learned my addr 172.21.15.59:0/2108495818 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:42.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa1d9b700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa4104130 0x7f0fa419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:42.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.825+0000 7f0fa2d9d700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa4108870 0x7f0fa4199930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:42.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0fa259c700 1 -- 172.21.15.59:0/2108495818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa4104130 msgr2=0x7f0fa419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:42.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0fa259c700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa4104130 0x7f0fa419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:42.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0fa259c700 1 -- 172.21.15.59:0/2108495818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa4108870 msgr2=0x7f0fa4199930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:42.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0fa259c700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa4108870 0x7f0fa4199930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:42.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0fa259c700 1 -- 172.21.15.59:0/2108495818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fa419a090 con 0x7f0fa4103780 2024-05-08T05:29:42.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0fa259c700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4103780 0x7f0fa419f720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f940060b0 tx=0x7f0f9400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:42.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0f9b7fe700 1 -- 172.21.15.59:0/2108495818 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f9401a070 con 0x7f0fa4103780 2024-05-08T05:29:42.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fa419a380 con 0x7f0fa4103780 2024-05-08T05:29:42.850 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0f9b7fe700 1 -- 172.21.15.59:0/2108495818 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f9400ddd0 con 0x7f0fa4103780 2024-05-08T05:29:42.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0f9b7fe700 1 -- 172.21.15.59:0/2108495818 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f94014720 con 0x7f0fa4103780 2024-05-08T05:29:42.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.826+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fa41a65f0 con 0x7f0fa4103780 2024-05-08T05:29:42.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.827+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fa4104c00 con 0x7f0fa4103780 2024-05-08T05:29:42.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.829+0000 7f0f9b7fe700 1 -- 172.21.15.59:0/2108495818 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f0f9401d070 con 0x7f0fa4103780 2024-05-08T05:29:42.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.829+0000 7f0f9b7fe700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0f900b2b10 0x7f0f900b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:42.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.830+0000 7f0fa1d9b700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0f900b2b10 0x7f0f900b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:42.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.830+0000 7f0f9b7fe700 1 -- 172.21.15.59:0/2108495818 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0f940e2990 con 0x7f0fa4103780 2024-05-08T05:29:42.851 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.830+0000 7f0fa1d9b700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0f900b2b10 0x7f0f900b4fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009990 tx=0x7f0f8c009650 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:42.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:42.833+0000 7f0f9b7fe700 1 -- 172.21.15.59:0/2108495818 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f0f94010070 con 0x7f0fa4103780 2024-05-08T05:29:43.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.044+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0fa40638e0 con 0x7f0f900b2b10 2024-05-08T05:29:43.054 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.054+0000 7f0f9b7fe700 1 -- 172.21.15.59:0/2108495818 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339750 (secure 0 0 0) 0x7f0fa40638e0 con 0x7f0f900b2b10 2024-05-08T05:29:43.059 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:29:43.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0f900b2b10 msgr2=0x7f0f900b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:43.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0f900b2b10 0x7f0f900b4fd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0f8c009990 tx=0x7f0f8c009650 comp rx=0 tx=0).stop 2024-05-08T05:29:43.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4103780 msgr2=0x7f0fa419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:43.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4103780 0x7f0fa419f720 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f0f940060b0 tx=0x7f0f9400d3b0 comp rx=0 tx=0).stop 2024-05-08T05:29:43.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 shutdown_connections 2024-05-08T05:29:43.061 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0f900b2b10 0x7f0f900b4fd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:43.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0fa4108870 0x7f0fa4199930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:43.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0fa4103780 0x7f0fa419f720 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:43.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 --2- 172.21.15.59:0/2108495818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0fa4104130 0x7f0fa419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:43.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 >> 172.21.15.59:0/2108495818 conn(0x7f0fa407a610 msgr2=0x7f0fa4101810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:43.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 shutdown_connections 2024-05-08T05:29:43.062 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:43.058+0000 7f0fa8aa3700 1 -- 172.21.15.59:0/2108495818 wait complete. 2024-05-08T05:29:43.062 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T05:29:43.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:43 smithi059 ceph-mon[30304]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 132 KiB/s wr, 73 op/s 2024-05-08T05:29:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:43 smithi143 ceph-mon[34337]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 132 KiB/s wr, 73 op/s 2024-05-08T05:29:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:43 smithi096 ceph-mon[34253]: pgmap v7: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 132 KiB/s wr, 73 op/s 2024-05-08T05:29:43.772 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":7,"stamp":"2024-05-08T05:29:41.840813+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1396072,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2466,"num_read_kb":1639,"num_write":660,"num_write_kb":4556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951840,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1001,"ondisk_log_size":1001,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361824,"kb_used_data":12632,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580000,"statfs":{"total":1151940427776,"available":1151569920000,"internally_reserved":0,"allocated":12935168,"data_stored":10527458,"data_compressed":636033,"data_compressed_allocated":860160,"data_compressed_original":5050368,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200325,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":370,"num_read_kb":440,"num_write":145,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.991558"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088374+0000","last_change":"2024-05-08T05:25:25.667895+0000","last_active":"2024-05-08T05:29:34.088374+0000","last_peered":"2024-05-08T05:29:34.088374+0000","last_clean":"2024-05-08T05:29:34.088374+0000","last_became_active":"2024-05-08T05:25:25.667563+0000","last_became_peered":"2024-05-08T05:25:25.667563+0000","last_unstale":"2024-05-08T05:29:34.088374+0000","last_undegraded":"2024-05-08T05:29:34.088374+0000","last_fullsized":"2024-05-08T05:29:34.088374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:47.677082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088642+0000","last_change":"2024-05-08T05:25:25.673346+0000","last_active":"2024-05-08T05:29:34.088642+0000","last_peered":"2024-05-08T05:29:34.088642+0000","last_clean":"2024-05-08T05:29:34.088642+0000","last_became_active":"2024-05-08T05:25:25.673011+0000","last_became_peered":"2024-05-08T05:25:25.673011+0000","last_unstale":"2024-05-08T05:29:34.088642+0000","last_undegraded":"2024-05-08T05:29:34.088642+0000","last_fullsized":"2024-05-08T05:29:34.088642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:21:05.091177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.089071+0000","last_change":"2024-05-08T05:25:25.670469+0000","last_active":"2024-05-08T05:29:34.089071+0000","last_peered":"2024-05-08T05:29:34.089071+0000","last_clean":"2024-05-08T05:29:34.089071+0000","last_became_active":"2024-05-08T05:25:25.669700+0000","last_became_peered":"2024-05-08T05:25:25.669700+0000","last_unstale":"2024-05-08T05:29:34.089071+0000","last_undegraded":"2024-05-08T05:29:34.089071+0000","last_fullsized":"2024-05-08T05:29:34.089071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:06.046633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093206+0000","last_change":"2024-05-08T05:25:25.659937+0000","last_active":"2024-05-08T05:29:34.093206+0000","last_peered":"2024-05-08T05:29:34.093206+0000","last_clean":"2024-05-08T05:29:34.093206+0000","last_became_active":"2024-05-08T05:25:25.658892+0000","last_became_peered":"2024-05-08T05:25:25.658892+0000","last_unstale":"2024-05-08T05:29:34.093206+0000","last_undegraded":"2024-05-08T05:29:34.093206+0000","last_fullsized":"2024-05-08T05:29:34.093206+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:10.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925563+0000","last_change":"2024-05-08T05:25:25.669132+0000","last_active":"2024-05-08T05:29:33.925563+0000","last_peered":"2024-05-08T05:29:33.925563+0000","last_clean":"2024-05-08T05:29:33.925563+0000","last_became_active":"2024-05-08T05:25:25.668556+0000","last_became_peered":"2024-05-08T05:25:25.668556+0000","last_unstale":"2024-05-08T05:29:33.925563+0000","last_undegraded":"2024-05-08T05:29:33.925563+0000","last_fullsized":"2024-05-08T05:29:33.925563+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:28.443763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936400+0000","last_change":"2024-05-08T05:25:25.668978+0000","last_active":"2024-05-08T05:29:33.936400+0000","last_peered":"2024-05-08T05:29:33.936400+0000","last_clean":"2024-05-08T05:29:33.936400+0000","last_became_active":"2024-05-08T05:25:25.668647+0000","last_became_peered":"2024-05-08T05:25:25.668647+0000","last_unstale":"2024-05-08T05:29:33.936400+0000","last_undegraded":"2024-05-08T05:29:33.936400+0000","last_fullsized":"2024-05-08T05:29:33.936400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:07:00.962620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085103+0000","last_change":"2024-05-08T05:25:25.664784+0000","last_active":"2024-05-08T05:29:34.085103+0000","last_peered":"2024-05-08T05:29:34.085103+0000","last_clean":"2024-05-08T05:29:34.085103+0000","last_became_active":"2024-05-08T05:25:25.664390+0000","last_became_peered":"2024-05-08T05:25:25.664390+0000","last_unstale":"2024-05-08T05:29:34.085103+0000","last_undegraded":"2024-05-08T05:29:34.085103+0000","last_fullsized":"2024-05-08T05:29:34.085103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:08:35.358796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925616+0000","last_change":"2024-05-08T05:25:25.677139+0000","last_active":"2024-05-08T05:29:33.925616+0000","last_peered":"2024-05-08T05:29:33.925616+0000","last_clean":"2024-05-08T05:29:33.925616+0000","last_became_active":"2024-05-08T05:25:25.676444+0000","last_became_peered":"2024-05-08T05:25:25.676444+0000","last_unstale":"2024-05-08T05:29:33.925616+0000","last_undegraded":"2024-05-08T05:29:33.925616+0000","last_fullsized":"2024-05-08T05:29:33.925616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:56.704152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826525+0000","last_change":"2024-05-08T05:25:25.675593+0000","last_active":"2024-05-08T05:29:33.826525+0000","last_peered":"2024-05-08T05:29:33.826525+0000","last_clean":"2024-05-08T05:29:33.826525+0000","last_became_active":"2024-05-08T05:25:25.675136+0000","last_became_peered":"2024-05-08T05:25:25.675136+0000","last_unstale":"2024-05-08T05:29:33.826525+0000","last_undegraded":"2024-05-08T05:29:33.826525+0000","last_fullsized":"2024-05-08T05:29:33.826525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:49.042956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936724+0000","last_change":"2024-05-08T05:25:25.669384+0000","last_active":"2024-05-08T05:29:33.936724+0000","last_peered":"2024-05-08T05:29:33.936724+0000","last_clean":"2024-05-08T05:29:33.936724+0000","last_became_active":"2024-05-08T05:25:25.668861+0000","last_became_peered":"2024-05-08T05:25:25.668861+0000","last_unstale":"2024-05-08T05:29:33.936724+0000","last_undegraded":"2024-05-08T05:29:33.936724+0000","last_fullsized":"2024-05-08T05:29:33.936724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:57.491242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085723+0000","last_change":"2024-05-08T05:25:25.669463+0000","last_active":"2024-05-08T05:29:34.085723+0000","last_peered":"2024-05-08T05:29:34.085723+0000","last_clean":"2024-05-08T05:29:34.085723+0000","last_became_active":"2024-05-08T05:25:25.668672+0000","last_became_peered":"2024-05-08T05:25:25.668672+0000","last_unstale":"2024-05-08T05:29:34.085723+0000","last_undegraded":"2024-05-08T05:29:34.085723+0000","last_fullsized":"2024-05-08T05:29:34.085723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:30.779473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093195+0000","last_change":"2024-05-08T05:25:25.670378+0000","last_active":"2024-05-08T05:29:34.093195+0000","last_peered":"2024-05-08T05:29:34.093195+0000","last_clean":"2024-05-08T05:29:34.093195+0000","last_became_active":"2024-05-08T05:25:25.669801+0000","last_became_peered":"2024-05-08T05:25:25.669801+0000","last_unstale":"2024-05-08T05:29:34.093195+0000","last_undegraded":"2024-05-08T05:29:34.093195+0000","last_fullsized":"2024-05-08T05:29:34.093195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:49.614163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680129+0000","last_change":"2024-05-08T05:25:25.670874+0000","last_active":"2024-05-08T05:29:34.680129+0000","last_peered":"2024-05-08T05:29:34.680129+0000","last_clean":"2024-05-08T05:29:34.680129+0000","last_became_active":"2024-05-08T05:25:25.670557+0000","last_became_peered":"2024-05-08T05:25:25.670557+0000","last_unstale":"2024-05-08T05:29:34.680129+0000","last_undegraded":"2024-05-08T05:29:34.680129+0000","last_fullsized":"2024-05-08T05:29:34.680129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:58.594223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088286+0000","last_change":"2024-05-08T05:25:25.670015+0000","last_active":"2024-05-08T05:29:34.088286+0000","last_peered":"2024-05-08T05:29:34.088286+0000","last_clean":"2024-05-08T05:29:34.088286+0000","last_became_active":"2024-05-08T05:25:25.669493+0000","last_became_peered":"2024-05-08T05:25:25.669493+0000","last_unstale":"2024-05-08T05:29:34.088286+0000","last_undegraded":"2024-05-08T05:29:34.088286+0000","last_fullsized":"2024-05-08T05:29:34.088286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:49.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088404+0000","last_change":"2024-05-08T05:25:25.659651+0000","last_active":"2024-05-08T05:29:34.088404+0000","last_peered":"2024-05-08T05:29:34.088404+0000","last_clean":"2024-05-08T05:29:34.088404+0000","last_became_active":"2024-05-08T05:25:25.659308+0000","last_became_peered":"2024-05-08T05:25:25.659308+0000","last_unstale":"2024-05-08T05:29:34.088404+0000","last_undegraded":"2024-05-08T05:29:34.088404+0000","last_fullsized":"2024-05-08T05:29:34.088404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:49.945474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825750+0000","last_change":"2024-05-08T05:25:25.663696+0000","last_active":"2024-05-08T05:29:33.825750+0000","last_peered":"2024-05-08T05:29:33.825750+0000","last_clean":"2024-05-08T05:29:33.825750+0000","last_became_active":"2024-05-08T05:25:25.662590+0000","last_became_peered":"2024-05-08T05:25:25.662590+0000","last_unstale":"2024-05-08T05:29:33.825750+0000","last_undegraded":"2024-05-08T05:29:33.825750+0000","last_fullsized":"2024-05-08T05:29:33.825750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:29:21.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088209+0000","last_change":"2024-05-08T05:25:25.668552+0000","last_active":"2024-05-08T05:29:34.088209+0000","last_peered":"2024-05-08T05:29:34.088209+0000","last_clean":"2024-05-08T05:29:34.088209+0000","last_became_active":"2024-05-08T05:25:25.668185+0000","last_became_peered":"2024-05-08T05:25:25.668185+0000","last_unstale":"2024-05-08T05:29:34.088209+0000","last_undegraded":"2024-05-08T05:29:34.088209+0000","last_fullsized":"2024-05-08T05:29:34.088209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:25.762909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825129+0000","last_change":"2024-05-08T05:25:25.668259+0000","last_active":"2024-05-08T05:29:33.825129+0000","last_peered":"2024-05-08T05:29:33.825129+0000","last_clean":"2024-05-08T05:29:33.825129+0000","last_became_active":"2024-05-08T05:25:25.667767+0000","last_became_peered":"2024-05-08T05:25:25.667767+0000","last_unstale":"2024-05-08T05:29:33.825129+0000","last_undegraded":"2024-05-08T05:29:33.825129+0000","last_fullsized":"2024-05-08T05:29:33.825129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:36:34.984139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827417+0000","last_change":"2024-05-08T05:25:25.663381+0000","last_active":"2024-05-08T05:29:33.827417+0000","last_peered":"2024-05-08T05:29:33.827417+0000","last_clean":"2024-05-08T05:29:33.827417+0000","last_became_active":"2024-05-08T05:25:25.663019+0000","last_became_peered":"2024-05-08T05:25:25.663019+0000","last_unstale":"2024-05-08T05:29:33.827417+0000","last_undegraded":"2024-05-08T05:29:33.827417+0000","last_fullsized":"2024-05-08T05:29:33.827417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:58.440446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826825+0000","last_change":"2024-05-08T05:25:37.052656+0000","last_active":"2024-05-08T05:29:33.826825+0000","last_peered":"2024-05-08T05:29:33.826825+0000","last_clean":"2024-05-08T05:29:33.826825+0000","last_became_active":"2024-05-08T05:25:37.052237+0000","last_became_peered":"2024-05-08T05:25:37.052237+0000","last_unstale":"2024-05-08T05:29:33.826825+0000","last_undegraded":"2024-05-08T05:29:33.826825+0000","last_fullsized":"2024-05-08T05:29:33.826825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:54.354488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'15","reported_seq":255,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:38.683127+0000","last_change":"2024-05-08T05:25:22.649492+0000","last_active":"2024-05-08T05:29:38.683127+0000","last_peered":"2024-05-08T05:29:38.683127+0000","last_clean":"2024-05-08T05:29:38.683127+0000","last_became_active":"2024-05-08T05:25:22.649132+0000","last_became_peered":"2024-05-08T05:25:22.649132+0000","last_unstale":"2024-05-08T05:29:38.683127+0000","last_undegraded":"2024-05-08T05:29:38.683127+0000","last_fullsized":"2024-05-08T05:29:38.683127+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:24.474848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":193,"num_read_kb":193,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926798+0000","last_change":"2024-05-08T05:25:25.667811+0000","last_active":"2024-05-08T05:29:33.926798+0000","last_peered":"2024-05-08T05:29:33.926798+0000","last_clean":"2024-05-08T05:29:33.926798+0000","last_became_active":"2024-05-08T05:25:25.667449+0000","last_became_peered":"2024-05-08T05:25:25.667449+0000","last_unstale":"2024-05-08T05:29:33.926798+0000","last_undegraded":"2024-05-08T05:29:33.926798+0000","last_fullsized":"2024-05-08T05:29:33.926798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:29.293702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092852+0000","last_change":"2024-05-08T05:26:05.308251+0000","last_active":"2024-05-08T05:29:34.092852+0000","last_peered":"2024-05-08T05:29:34.092852+0000","last_clean":"2024-05-08T05:29:34.092852+0000","last_became_active":"2024-05-08T05:26:05.307622+0000","last_became_peered":"2024-05-08T05:26:05.307622+0000","last_unstale":"2024-05-08T05:29:34.092852+0000","last_undegraded":"2024-05-08T05:29:34.092852+0000","last_fullsized":"2024-05-08T05:29:34.092852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:51.116094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926878+0000","last_change":"2024-05-08T05:25:22.748130+0000","last_active":"2024-05-08T05:29:33.926878+0000","last_peered":"2024-05-08T05:29:33.926878+0000","last_clean":"2024-05-08T05:29:33.926878+0000","last_became_active":"2024-05-08T05:25:22.747769+0000","last_became_peered":"2024-05-08T05:25:22.747769+0000","last_unstale":"2024-05-08T05:29:33.926878+0000","last_undegraded":"2024-05-08T05:29:33.926878+0000","last_fullsized":"2024-05-08T05:29:33.926878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:04.086851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088665+0000","last_change":"2024-05-08T05:25:25.672114+0000","last_active":"2024-05-08T05:29:34.088665+0000","last_peered":"2024-05-08T05:29:34.088665+0000","last_clean":"2024-05-08T05:29:34.088665+0000","last_became_active":"2024-05-08T05:25:25.671761+0000","last_became_peered":"2024-05-08T05:25:25.671761+0000","last_unstale":"2024-05-08T05:29:34.088665+0000","last_undegraded":"2024-05-08T05:29:34.088665+0000","last_fullsized":"2024-05-08T05:29:34.088665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:40.684565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092958+0000","last_change":"2024-05-08T05:25:37.058375+0000","last_active":"2024-05-08T05:29:34.092958+0000","last_peered":"2024-05-08T05:29:34.092958+0000","last_clean":"2024-05-08T05:29:34.092958+0000","last_became_active":"2024-05-08T05:25:37.057844+0000","last_became_peered":"2024-05-08T05:25:37.057844+0000","last_unstale":"2024-05-08T05:29:34.092958+0000","last_undegraded":"2024-05-08T05:29:34.092958+0000","last_fullsized":"2024-05-08T05:29:34.092958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:51:42.881830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088188+0000","last_change":"2024-05-08T05:27:05.576781+0000","last_active":"2024-05-08T05:29:34.088188+0000","last_peered":"2024-05-08T05:29:34.088188+0000","last_clean":"2024-05-08T05:29:34.088188+0000","last_became_active":"2024-05-08T05:27:05.566420+0000","last_became_peered":"2024-05-08T05:27:05.566420+0000","last_unstale":"2024-05-08T05:29:34.088188+0000","last_undegraded":"2024-05-08T05:29:34.088188+0000","last_fullsized":"2024-05-08T05:29:34.088188+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:09.385135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826181+0000","last_change":"2024-05-08T05:25:25.673300+0000","last_active":"2024-05-08T05:29:33.826181+0000","last_peered":"2024-05-08T05:29:33.826181+0000","last_clean":"2024-05-08T05:29:33.826181+0000","last_became_active":"2024-05-08T05:25:25.672881+0000","last_became_peered":"2024-05-08T05:25:25.672881+0000","last_unstale":"2024-05-08T05:29:33.826181+0000","last_undegraded":"2024-05-08T05:29:33.826181+0000","last_fullsized":"2024-05-08T05:29:33.826181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:16.111775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092846+0000","last_change":"2024-05-08T05:25:37.058903+0000","last_active":"2024-05-08T05:29:34.092846+0000","last_peered":"2024-05-08T05:29:34.092846+0000","last_clean":"2024-05-08T05:29:34.092846+0000","last_became_active":"2024-05-08T05:25:37.058508+0000","last_became_peered":"2024-05-08T05:25:37.058508+0000","last_unstale":"2024-05-08T05:29:34.092846+0000","last_undegraded":"2024-05-08T05:29:34.092846+0000","last_fullsized":"2024-05-08T05:29:34.092846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:15:16.605748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925877+0000","last_change":"2024-05-08T05:25:22.646191+0000","last_active":"2024-05-08T05:29:33.925877+0000","last_peered":"2024-05-08T05:29:33.925877+0000","last_clean":"2024-05-08T05:29:33.925877+0000","last_became_active":"2024-05-08T05:25:22.645187+0000","last_became_peered":"2024-05-08T05:25:22.645187+0000","last_unstale":"2024-05-08T05:29:33.925877+0000","last_undegraded":"2024-05-08T05:29:33.925877+0000","last_fullsized":"2024-05-08T05:29:33.925877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:13.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933506+0000","last_change":"2024-05-08T05:26:05.308149+0000","last_active":"2024-05-08T05:29:33.933506+0000","last_peered":"2024-05-08T05:29:33.933506+0000","last_clean":"2024-05-08T05:29:33.933506+0000","last_became_active":"2024-05-08T05:26:05.307602+0000","last_became_peered":"2024-05-08T05:26:05.307602+0000","last_unstale":"2024-05-08T05:29:33.933506+0000","last_undegraded":"2024-05-08T05:29:33.933506+0000","last_fullsized":"2024-05-08T05:29:33.933506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:02.791894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827181+0000","last_change":"2024-05-08T05:25:37.059052+0000","last_active":"2024-05-08T05:29:33.827181+0000","last_peered":"2024-05-08T05:29:33.827181+0000","last_clean":"2024-05-08T05:29:33.827181+0000","last_became_active":"2024-05-08T05:25:37.058545+0000","last_became_peered":"2024-05-08T05:25:37.058545+0000","last_unstale":"2024-05-08T05:29:33.827181+0000","last_undegraded":"2024-05-08T05:29:33.827181+0000","last_fullsized":"2024-05-08T05:29:33.827181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:31.521284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":253,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:38.684136+0000","last_change":"2024-05-08T05:25:22.648874+0000","last_active":"2024-05-08T05:29:38.684136+0000","last_peered":"2024-05-08T05:29:38.684136+0000","last_clean":"2024-05-08T05:29:38.684136+0000","last_became_active":"2024-05-08T05:25:22.648477+0000","last_became_peered":"2024-05-08T05:25:22.648477+0000","last_unstale":"2024-05-08T05:29:38.684136+0000","last_undegraded":"2024-05-08T05:29:38.684136+0000","last_fullsized":"2024-05-08T05:29:38.684136+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:08.075904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":197,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925572+0000","last_change":"2024-05-08T05:25:25.662449+0000","last_active":"2024-05-08T05:29:33.925572+0000","last_peered":"2024-05-08T05:29:33.925572+0000","last_clean":"2024-05-08T05:29:33.925572+0000","last_became_active":"2024-05-08T05:25:25.662089+0000","last_became_peered":"2024-05-08T05:25:25.662089+0000","last_unstale":"2024-05-08T05:29:33.925572+0000","last_undegraded":"2024-05-08T05:29:33.925572+0000","last_fullsized":"2024-05-08T05:29:33.925572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:31.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827128+0000","last_change":"2024-05-08T05:25:37.055548+0000","last_active":"2024-05-08T05:29:33.827128+0000","last_peered":"2024-05-08T05:29:33.827128+0000","last_clean":"2024-05-08T05:29:33.827128+0000","last_became_active":"2024-05-08T05:25:37.055138+0000","last_became_peered":"2024-05-08T05:25:37.055138+0000","last_unstale":"2024-05-08T05:29:33.827128+0000","last_undegraded":"2024-05-08T05:29:33.827128+0000","last_fullsized":"2024-05-08T05:29:33.827128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:45.697831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936365+0000","last_change":"2024-05-08T05:25:22.648072+0000","last_active":"2024-05-08T05:29:33.936365+0000","last_peered":"2024-05-08T05:29:33.936365+0000","last_clean":"2024-05-08T05:29:33.936365+0000","last_became_active":"2024-05-08T05:25:22.647457+0000","last_became_peered":"2024-05-08T05:25:22.647457+0000","last_unstale":"2024-05-08T05:29:33.936365+0000","last_undegraded":"2024-05-08T05:29:33.936365+0000","last_fullsized":"2024-05-08T05:29:33.936365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:17.476479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933534+0000","last_change":"2024-05-08T05:25:25.662698+0000","last_active":"2024-05-08T05:29:33.933534+0000","last_peered":"2024-05-08T05:29:33.933534+0000","last_clean":"2024-05-08T05:29:33.933534+0000","last_became_active":"2024-05-08T05:25:25.662368+0000","last_became_peered":"2024-05-08T05:25:25.662368+0000","last_unstale":"2024-05-08T05:29:33.933534+0000","last_undegraded":"2024-05-08T05:29:33.933534+0000","last_fullsized":"2024-05-08T05:29:33.933534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:38.511037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088201+0000","last_change":"2024-05-08T05:25:37.054407+0000","last_active":"2024-05-08T05:29:34.088201+0000","last_peered":"2024-05-08T05:29:34.088201+0000","last_clean":"2024-05-08T05:29:34.088201+0000","last_became_active":"2024-05-08T05:25:37.053999+0000","last_became_peered":"2024-05-08T05:25:37.053999+0000","last_unstale":"2024-05-08T05:29:34.088201+0000","last_undegraded":"2024-05-08T05:29:34.088201+0000","last_fullsized":"2024-05-08T05:29:34.088201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:16.942601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680002+0000","last_change":"2024-05-08T05:25:22.643654+0000","last_active":"2024-05-08T05:29:34.680002+0000","last_peered":"2024-05-08T05:29:34.680002+0000","last_clean":"2024-05-08T05:29:34.680002+0000","last_became_active":"2024-05-08T05:25:22.643315+0000","last_became_peered":"2024-05-08T05:25:22.643315+0000","last_unstale":"2024-05-08T05:29:34.680002+0000","last_undegraded":"2024-05-08T05:29:34.680002+0000","last_fullsized":"2024-05-08T05:29:34.680002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:53.514629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925576+0000","last_change":"2024-05-08T05:25:25.663148+0000","last_active":"2024-05-08T05:29:33.925576+0000","last_peered":"2024-05-08T05:29:33.925576+0000","last_clean":"2024-05-08T05:29:33.925576+0000","last_became_active":"2024-05-08T05:25:25.662770+0000","last_became_peered":"2024-05-08T05:25:25.662770+0000","last_unstale":"2024-05-08T05:29:33.925576+0000","last_undegraded":"2024-05-08T05:29:33.925576+0000","last_fullsized":"2024-05-08T05:29:33.925576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:46.458233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088500+0000","last_change":"2024-05-08T05:25:37.054265+0000","last_active":"2024-05-08T05:29:34.088500+0000","last_peered":"2024-05-08T05:29:34.088500+0000","last_clean":"2024-05-08T05:29:34.088500+0000","last_became_active":"2024-05-08T05:25:37.053579+0000","last_became_peered":"2024-05-08T05:25:37.053579+0000","last_unstale":"2024-05-08T05:29:34.088500+0000","last_undegraded":"2024-05-08T05:29:34.088500+0000","last_fullsized":"2024-05-08T05:29:34.088500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:21.748715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933115+0000","last_change":"2024-05-08T05:25:22.648083+0000","last_active":"2024-05-08T05:29:33.933115+0000","last_peered":"2024-05-08T05:29:33.933115+0000","last_clean":"2024-05-08T05:29:33.933115+0000","last_became_active":"2024-05-08T05:25:22.647650+0000","last_became_peered":"2024-05-08T05:25:22.647650+0000","last_unstale":"2024-05-08T05:29:33.933115+0000","last_undegraded":"2024-05-08T05:29:33.933115+0000","last_fullsized":"2024-05-08T05:29:33.933115+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:56.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.824690+0000","last_change":"2024-05-08T05:25:25.668290+0000","last_active":"2024-05-08T05:29:33.824690+0000","last_peered":"2024-05-08T05:29:33.824690+0000","last_clean":"2024-05-08T05:29:33.824690+0000","last_became_active":"2024-05-08T05:25:25.667888+0000","last_became_peered":"2024-05-08T05:25:25.667888+0000","last_unstale":"2024-05-08T05:29:33.824690+0000","last_undegraded":"2024-05-08T05:29:33.824690+0000","last_fullsized":"2024-05-08T05:29:33.824690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:19.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925789+0000","last_change":"2024-05-08T05:25:37.057639+0000","last_active":"2024-05-08T05:29:33.925789+0000","last_peered":"2024-05-08T05:29:33.925789+0000","last_clean":"2024-05-08T05:29:33.925789+0000","last_became_active":"2024-05-08T05:25:37.056848+0000","last_became_peered":"2024-05-08T05:25:37.056848+0000","last_unstale":"2024-05-08T05:29:33.925789+0000","last_undegraded":"2024-05-08T05:29:33.925789+0000","last_fullsized":"2024-05-08T05:29:33.925789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:37.137224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":438,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:39.684901+0000","last_change":"2024-05-08T05:25:22.649658+0000","last_active":"2024-05-08T05:29:39.684901+0000","last_peered":"2024-05-08T05:29:39.684901+0000","last_clean":"2024-05-08T05:29:39.684901+0000","last_became_active":"2024-05-08T05:25:22.648771+0000","last_became_peered":"2024-05-08T05:25:22.648771+0000","last_unstale":"2024-05-08T05:29:39.684901+0000","last_undegraded":"2024-05-08T05:29:39.684901+0000","last_fullsized":"2024-05-08T05:29:39.684901+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:54.412710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":384,"num_read_kb":384,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826502+0000","last_change":"2024-05-08T05:25:25.673931+0000","last_active":"2024-05-08T05:29:33.826502+0000","last_peered":"2024-05-08T05:29:33.826502+0000","last_clean":"2024-05-08T05:29:33.826502+0000","last_became_active":"2024-05-08T05:25:25.673585+0000","last_became_peered":"2024-05-08T05:25:25.673585+0000","last_unstale":"2024-05-08T05:29:33.826502+0000","last_undegraded":"2024-05-08T05:29:33.826502+0000","last_fullsized":"2024-05-08T05:29:33.826502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:27.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.824556+0000","last_change":"2024-05-08T05:25:37.047651+0000","last_active":"2024-05-08T05:29:33.824556+0000","last_peered":"2024-05-08T05:29:33.824556+0000","last_clean":"2024-05-08T05:29:33.824556+0000","last_became_active":"2024-05-08T05:25:37.047246+0000","last_became_peered":"2024-05-08T05:25:37.047246+0000","last_unstale":"2024-05-08T05:29:33.824556+0000","last_undegraded":"2024-05-08T05:29:33.824556+0000","last_fullsized":"2024-05-08T05:29:33.824556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:22.300076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085560+0000","last_change":"2024-05-08T05:25:22.646973+0000","last_active":"2024-05-08T05:29:34.085560+0000","last_peered":"2024-05-08T05:29:34.085560+0000","last_clean":"2024-05-08T05:29:34.085560+0000","last_became_active":"2024-05-08T05:25:22.646232+0000","last_became_peered":"2024-05-08T05:25:22.646232+0000","last_unstale":"2024-05-08T05:29:34.085560+0000","last_undegraded":"2024-05-08T05:29:34.085560+0000","last_fullsized":"2024-05-08T05:29:34.085560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:38.147470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088524+0000","last_change":"2024-05-08T05:25:25.661373+0000","last_active":"2024-05-08T05:29:34.088524+0000","last_peered":"2024-05-08T05:29:34.088524+0000","last_clean":"2024-05-08T05:29:34.088524+0000","last_became_active":"2024-05-08T05:25:25.661080+0000","last_became_peered":"2024-05-08T05:25:25.661080+0000","last_unstale":"2024-05-08T05:29:34.088524+0000","last_undegraded":"2024-05-08T05:29:34.088524+0000","last_fullsized":"2024-05-08T05:29:34.088524+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:18.718610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093071+0000","last_change":"2024-05-08T05:25:37.051797+0000","last_active":"2024-05-08T05:29:34.093071+0000","last_peered":"2024-05-08T05:29:34.093071+0000","last_clean":"2024-05-08T05:29:34.093071+0000","last_became_active":"2024-05-08T05:25:37.051381+0000","last_became_peered":"2024-05-08T05:25:37.051381+0000","last_unstale":"2024-05-08T05:29:34.093071+0000","last_undegraded":"2024-05-08T05:29:34.093071+0000","last_fullsized":"2024-05-08T05:29:34.093071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:34.337512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933456+0000","last_change":"2024-05-08T05:25:22.748510+0000","last_active":"2024-05-08T05:29:33.933456+0000","last_peered":"2024-05-08T05:29:33.933456+0000","last_clean":"2024-05-08T05:29:33.933456+0000","last_became_active":"2024-05-08T05:25:22.748140+0000","last_became_peered":"2024-05-08T05:25:22.748140+0000","last_unstale":"2024-05-08T05:29:33.933456+0000","last_undegraded":"2024-05-08T05:29:33.933456+0000","last_fullsized":"2024-05-08T05:29:33.933456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:32.325653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826339+0000","last_change":"2024-05-08T05:25:25.671261+0000","last_active":"2024-05-08T05:29:33.826339+0000","last_peered":"2024-05-08T05:29:33.826339+0000","last_clean":"2024-05-08T05:29:33.826339+0000","last_became_active":"2024-05-08T05:25:25.670937+0000","last_became_peered":"2024-05-08T05:25:25.670937+0000","last_unstale":"2024-05-08T05:29:33.826339+0000","last_undegraded":"2024-05-08T05:29:33.826339+0000","last_fullsized":"2024-05-08T05:29:33.826339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:41.375908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936237+0000","last_change":"2024-05-08T05:26:05.150754+0000","last_active":"2024-05-08T05:29:33.936237+0000","last_peered":"2024-05-08T05:29:33.936237+0000","last_clean":"2024-05-08T05:29:33.936237+0000","last_became_active":"2024-05-08T05:26:05.150331+0000","last_became_peered":"2024-05-08T05:26:05.150331+0000","last_unstale":"2024-05-08T05:29:33.936237+0000","last_undegraded":"2024-05-08T05:29:33.936237+0000","last_fullsized":"2024-05-08T05:29:33.936237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:23.309103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'44","reported_seq":489,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:39.714460+0000","last_change":"2024-05-08T05:25:22.645627+0000","last_active":"2024-05-08T05:29:39.714460+0000","last_peered":"2024-05-08T05:29:39.714460+0000","last_clean":"2024-05-08T05:29:39.714460+0000","last_became_active":"2024-05-08T05:25:22.645243+0000","last_became_peered":"2024-05-08T05:25:22.645243+0000","last_unstale":"2024-05-08T05:29:39.714460+0000","last_undegraded":"2024-05-08T05:29:39.714460+0000","last_fullsized":"2024-05-08T05:29:39.714460+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:23.256903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72818,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":399,"num_read_kb":272,"num_write":46,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925853+0000","last_change":"2024-05-08T05:25:25.661258+0000","last_active":"2024-05-08T05:29:33.925853+0000","last_peered":"2024-05-08T05:29:33.925853+0000","last_clean":"2024-05-08T05:29:33.925853+0000","last_became_active":"2024-05-08T05:25:25.660925+0000","last_became_peered":"2024-05-08T05:25:25.660925+0000","last_unstale":"2024-05-08T05:29:33.925853+0000","last_undegraded":"2024-05-08T05:29:33.925853+0000","last_fullsized":"2024-05-08T05:29:33.925853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:58.764540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933810+0000","last_change":"2024-05-08T05:25:37.053539+0000","last_active":"2024-05-08T05:29:33.933810+0000","last_peered":"2024-05-08T05:29:33.933810+0000","last_clean":"2024-05-08T05:29:33.933810+0000","last_became_active":"2024-05-08T05:25:37.053102+0000","last_became_peered":"2024-05-08T05:25:37.053102+0000","last_unstale":"2024-05-08T05:29:33.933810+0000","last_undegraded":"2024-05-08T05:29:33.933810+0000","last_fullsized":"2024-05-08T05:29:33.933810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:48.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827052+0000","last_change":"2024-05-08T05:25:22.646651+0000","last_active":"2024-05-08T05:29:33.827052+0000","last_peered":"2024-05-08T05:29:33.827052+0000","last_clean":"2024-05-08T05:29:33.827052+0000","last_became_active":"2024-05-08T05:25:22.646285+0000","last_became_peered":"2024-05-08T05:25:22.646285+0000","last_unstale":"2024-05-08T05:29:33.827052+0000","last_undegraded":"2024-05-08T05:29:33.827052+0000","last_fullsized":"2024-05-08T05:29:33.827052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.843858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680138+0000","last_change":"2024-05-08T05:25:25.666018+0000","last_active":"2024-05-08T05:29:34.680138+0000","last_peered":"2024-05-08T05:29:34.680138+0000","last_clean":"2024-05-08T05:29:34.680138+0000","last_became_active":"2024-05-08T05:25:25.663609+0000","last_became_peered":"2024-05-08T05:25:25.663609+0000","last_unstale":"2024-05-08T05:29:34.680138+0000","last_undegraded":"2024-05-08T05:29:34.680138+0000","last_fullsized":"2024-05-08T05:29:34.680138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:36.223121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087978+0000","last_change":"2024-05-08T05:25:37.052907+0000","last_active":"2024-05-08T05:29:34.087978+0000","last_peered":"2024-05-08T05:29:34.087978+0000","last_clean":"2024-05-08T05:29:34.087978+0000","last_became_active":"2024-05-08T05:25:37.052506+0000","last_became_peered":"2024-05-08T05:25:37.052506+0000","last_unstale":"2024-05-08T05:29:34.087978+0000","last_undegraded":"2024-05-08T05:29:34.087978+0000","last_fullsized":"2024-05-08T05:29:34.087978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:41.356996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933355+0000","last_change":"2024-05-08T05:25:22.646671+0000","last_active":"2024-05-08T05:29:33.933355+0000","last_peered":"2024-05-08T05:29:33.933355+0000","last_clean":"2024-05-08T05:29:33.933355+0000","last_became_active":"2024-05-08T05:25:22.646202+0000","last_became_peered":"2024-05-08T05:25:22.646202+0000","last_unstale":"2024-05-08T05:29:33.933355+0000","last_undegraded":"2024-05-08T05:29:33.933355+0000","last_fullsized":"2024-05-08T05:29:33.933355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:59.224960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085357+0000","last_change":"2024-05-08T05:25:25.666969+0000","last_active":"2024-05-08T05:29:34.085357+0000","last_peered":"2024-05-08T05:29:34.085357+0000","last_clean":"2024-05-08T05:29:34.085357+0000","last_became_active":"2024-05-08T05:25:25.666482+0000","last_became_peered":"2024-05-08T05:25:25.666482+0000","last_unstale":"2024-05-08T05:29:34.085357+0000","last_undegraded":"2024-05-08T05:29:34.085357+0000","last_fullsized":"2024-05-08T05:29:34.085357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:07.819116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825127+0000","last_change":"2024-05-08T05:25:37.055807+0000","last_active":"2024-05-08T05:29:33.825127+0000","last_peered":"2024-05-08T05:29:33.825127+0000","last_clean":"2024-05-08T05:29:33.825127+0000","last_became_active":"2024-05-08T05:25:37.054778+0000","last_became_peered":"2024-05-08T05:25:37.054778+0000","last_unstale":"2024-05-08T05:29:33.825127+0000","last_undegraded":"2024-05-08T05:29:33.825127+0000","last_fullsized":"2024-05-08T05:29:33.825127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:33.767885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'9","reported_seq":253,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:38.685427+0000","last_change":"2024-05-08T05:25:22.647093+0000","last_active":"2024-05-08T05:29:38.685427+0000","last_peered":"2024-05-08T05:29:38.685427+0000","last_clean":"2024-05-08T05:29:38.685427+0000","last_became_active":"2024-05-08T05:25:22.646733+0000","last_became_peered":"2024-05-08T05:25:22.646733+0000","last_unstale":"2024-05-08T05:29:38.685427+0000","last_undegraded":"2024-05-08T05:29:38.685427+0000","last_fullsized":"2024-05-08T05:29:38.685427+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:04.346568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1819,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":199,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085169+0000","last_change":"2024-05-08T05:25:25.666565+0000","last_active":"2024-05-08T05:29:34.085169+0000","last_peered":"2024-05-08T05:29:34.085169+0000","last_clean":"2024-05-08T05:29:34.085169+0000","last_became_active":"2024-05-08T05:25:25.666138+0000","last_became_peered":"2024-05-08T05:25:25.666138+0000","last_unstale":"2024-05-08T05:29:34.085169+0000","last_undegraded":"2024-05-08T05:29:34.085169+0000","last_fullsized":"2024-05-08T05:29:34.085169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:05.396530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680587+0000","last_change":"2024-05-08T05:25:37.060933+0000","last_active":"2024-05-08T05:29:34.680587+0000","last_peered":"2024-05-08T05:29:34.680587+0000","last_clean":"2024-05-08T05:29:34.680587+0000","last_became_active":"2024-05-08T05:25:37.060388+0000","last_became_peered":"2024-05-08T05:25:37.060388+0000","last_unstale":"2024-05-08T05:29:34.680587+0000","last_undegraded":"2024-05-08T05:29:34.680587+0000","last_fullsized":"2024-05-08T05:29:34.680587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:09.776051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":81,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925413+0000","last_change":"2024-05-08T05:26:05.156863+0000","last_active":"2024-05-08T05:29:33.925413+0000","last_peered":"2024-05-08T05:29:33.925413+0000","last_clean":"2024-05-08T05:29:33.925413+0000","last_became_active":"2024-05-08T05:26:05.149008+0000","last_became_peered":"2024-05-08T05:26:05.149008+0000","last_unstale":"2024-05-08T05:29:33.925413+0000","last_undegraded":"2024-05-08T05:29:33.925413+0000","last_fullsized":"2024-05-08T05:29:33.925413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:56.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933168+0000","last_change":"2024-05-08T05:25:25.670415+0000","last_active":"2024-05-08T05:29:33.933168+0000","last_peered":"2024-05-08T05:29:33.933168+0000","last_clean":"2024-05-08T05:29:33.933168+0000","last_became_active":"2024-05-08T05:25:25.670095+0000","last_became_peered":"2024-05-08T05:25:25.670095+0000","last_unstale":"2024-05-08T05:29:33.933168+0000","last_undegraded":"2024-05-08T05:29:33.933168+0000","last_fullsized":"2024-05-08T05:29:33.933168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.489203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925444+0000","last_change":"2024-05-08T05:26:05.307721+0000","last_active":"2024-05-08T05:29:33.925444+0000","last_peered":"2024-05-08T05:29:33.925444+0000","last_clean":"2024-05-08T05:29:33.925444+0000","last_became_active":"2024-05-08T05:26:05.307279+0000","last_became_peered":"2024-05-08T05:26:05.307279+0000","last_unstale":"2024-05-08T05:29:33.925444+0000","last_undegraded":"2024-05-08T05:29:33.925444+0000","last_fullsized":"2024-05-08T05:29:33.925444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:08.857770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925842+0000","last_change":"2024-05-08T05:25:22.644279+0000","last_active":"2024-05-08T05:29:33.925842+0000","last_peered":"2024-05-08T05:29:33.925842+0000","last_clean":"2024-05-08T05:29:33.925842+0000","last_became_active":"2024-05-08T05:25:22.643900+0000","last_became_peered":"2024-05-08T05:25:22.643900+0000","last_unstale":"2024-05-08T05:29:33.925842+0000","last_undegraded":"2024-05-08T05:29:33.925842+0000","last_fullsized":"2024-05-08T05:29:33.925842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:32.435843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088135+0000","last_change":"2024-05-08T05:25:25.670130+0000","last_active":"2024-05-08T05:29:34.088135+0000","last_peered":"2024-05-08T05:29:34.088135+0000","last_clean":"2024-05-08T05:29:34.088135+0000","last_became_active":"2024-05-08T05:25:25.669784+0000","last_became_peered":"2024-05-08T05:25:25.669784+0000","last_unstale":"2024-05-08T05:29:34.088135+0000","last_undegraded":"2024-05-08T05:29:34.088135+0000","last_fullsized":"2024-05-08T05:29:34.088135+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:07.188883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925515+0000","last_change":"2024-05-08T05:26:05.308804+0000","last_active":"2024-05-08T05:29:33.925515+0000","last_peered":"2024-05-08T05:29:33.925515+0000","last_clean":"2024-05-08T05:29:33.925515+0000","last_became_active":"2024-05-08T05:26:05.308226+0000","last_became_peered":"2024-05-08T05:26:05.308226+0000","last_unstale":"2024-05-08T05:29:33.925515+0000","last_undegraded":"2024-05-08T05:29:33.925515+0000","last_fullsized":"2024-05-08T05:29:33.925515+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:33.442023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.824979+0000","last_change":"2024-05-08T05:25:22.748409+0000","last_active":"2024-05-08T05:29:33.824979+0000","last_peered":"2024-05-08T05:29:33.824979+0000","last_clean":"2024-05-08T05:29:33.824979+0000","last_became_active":"2024-05-08T05:25:22.748062+0000","last_became_peered":"2024-05-08T05:25:22.748062+0000","last_unstale":"2024-05-08T05:29:33.824979+0000","last_undegraded":"2024-05-08T05:29:33.824979+0000","last_fullsized":"2024-05-08T05:29:33.824979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:42.147084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936551+0000","last_change":"2024-05-08T05:25:25.677057+0000","last_active":"2024-05-08T05:29:33.936551+0000","last_peered":"2024-05-08T05:29:33.936551+0000","last_clean":"2024-05-08T05:29:33.936551+0000","last_became_active":"2024-05-08T05:25:25.676697+0000","last_became_peered":"2024-05-08T05:25:25.676697+0000","last_unstale":"2024-05-08T05:29:33.936551+0000","last_undegraded":"2024-05-08T05:29:33.936551+0000","last_fullsized":"2024-05-08T05:29:33.936551+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:40.786327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680220+0000","last_change":"2024-05-08T05:25:37.053903+0000","last_active":"2024-05-08T05:29:34.680220+0000","last_peered":"2024-05-08T05:29:34.680220+0000","last_clean":"2024-05-08T05:29:34.680220+0000","last_became_active":"2024-05-08T05:25:37.053375+0000","last_became_peered":"2024-05-08T05:25:37.053375+0000","last_unstale":"2024-05-08T05:29:34.680220+0000","last_undegraded":"2024-05-08T05:29:34.680220+0000","last_fullsized":"2024-05-08T05:29:34.680220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:19.374355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'8","reported_seq":252,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:38.683605+0000","last_change":"2024-05-08T05:25:22.646931+0000","last_active":"2024-05-08T05:29:38.683605+0000","last_peered":"2024-05-08T05:29:38.683605+0000","last_clean":"2024-05-08T05:29:38.683605+0000","last_became_active":"2024-05-08T05:25:22.646599+0000","last_became_peered":"2024-05-08T05:25:22.646599+0000","last_unstale":"2024-05-08T05:29:38.683605+0000","last_undegraded":"2024-05-08T05:29:38.683605+0000","last_fullsized":"2024-05-08T05:29:38.683605+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:55.873089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":196,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936384+0000","last_change":"2024-05-08T05:26:05.147977+0000","last_active":"2024-05-08T05:29:33.936384+0000","last_peered":"2024-05-08T05:29:33.936384+0000","last_clean":"2024-05-08T05:29:33.936384+0000","last_became_active":"2024-05-08T05:26:05.147637+0000","last_became_peered":"2024-05-08T05:26:05.147637+0000","last_unstale":"2024-05-08T05:29:33.936384+0000","last_undegraded":"2024-05-08T05:29:33.936384+0000","last_fullsized":"2024-05-08T05:29:33.936384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:59.671306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680429+0000","last_change":"2024-05-08T05:25:37.057913+0000","last_active":"2024-05-08T05:29:34.680429+0000","last_peered":"2024-05-08T05:29:34.680429+0000","last_clean":"2024-05-08T05:29:34.680429+0000","last_became_active":"2024-05-08T05:25:37.057474+0000","last_became_peered":"2024-05-08T05:25:37.057474+0000","last_unstale":"2024-05-08T05:29:34.680429+0000","last_undegraded":"2024-05-08T05:29:34.680429+0000","last_fullsized":"2024-05-08T05:29:34.680429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:19.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933446+0000","last_change":"2024-05-08T05:25:25.663594+0000","last_active":"2024-05-08T05:29:33.933446+0000","last_peered":"2024-05-08T05:29:33.933446+0000","last_clean":"2024-05-08T05:29:33.933446+0000","last_became_active":"2024-05-08T05:25:25.663275+0000","last_became_peered":"2024-05-08T05:25:25.663275+0000","last_unstale":"2024-05-08T05:29:33.933446+0000","last_undegraded":"2024-05-08T05:29:33.933446+0000","last_fullsized":"2024-05-08T05:29:33.933446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:20.581166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":248,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:38.685138+0000","last_change":"2024-05-08T05:25:22.748316+0000","last_active":"2024-05-08T05:29:38.685138+0000","last_peered":"2024-05-08T05:29:38.685138+0000","last_clean":"2024-05-08T05:29:38.685138+0000","last_became_active":"2024-05-08T05:25:22.747819+0000","last_became_peered":"2024-05-08T05:25:22.747819+0000","last_unstale":"2024-05-08T05:29:38.685138+0000","last_undegraded":"2024-05-08T05:29:38.685138+0000","last_fullsized":"2024-05-08T05:29:38.685138+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:50:42.689853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":193,"num_read_kb":193,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085575+0000","last_change":"2024-05-08T05:25:37.058271+0000","last_active":"2024-05-08T05:29:34.085575+0000","last_peered":"2024-05-08T05:29:34.085575+0000","last_clean":"2024-05-08T05:29:34.085575+0000","last_became_active":"2024-05-08T05:25:37.057290+0000","last_became_peered":"2024-05-08T05:25:37.057290+0000","last_unstale":"2024-05-08T05:29:34.085575+0000","last_undegraded":"2024-05-08T05:29:34.085575+0000","last_fullsized":"2024-05-08T05:29:34.085575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:47.369909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925931+0000","last_change":"2024-05-08T05:25:25.666000+0000","last_active":"2024-05-08T05:29:33.925931+0000","last_peered":"2024-05-08T05:29:33.925931+0000","last_clean":"2024-05-08T05:29:33.925931+0000","last_became_active":"2024-05-08T05:25:25.665663+0000","last_became_peered":"2024-05-08T05:25:25.665663+0000","last_unstale":"2024-05-08T05:29:33.925931+0000","last_undegraded":"2024-05-08T05:29:33.925931+0000","last_fullsized":"2024-05-08T05:29:33.925931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:44.328879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093038+0000","last_change":"2024-05-08T05:25:22.648770+0000","last_active":"2024-05-08T05:29:34.093038+0000","last_peered":"2024-05-08T05:29:34.093038+0000","last_clean":"2024-05-08T05:29:34.093038+0000","last_became_active":"2024-05-08T05:25:22.648119+0000","last_became_peered":"2024-05-08T05:25:22.648119+0000","last_unstale":"2024-05-08T05:29:34.093038+0000","last_undegraded":"2024-05-08T05:29:34.093038+0000","last_fullsized":"2024-05-08T05:29:34.093038+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:55.813080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'99","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:35.935788+0000","last_change":"2024-05-08T05:23:55.820436+0000","last_active":"2024-05-08T05:29:35.935788+0000","last_peered":"2024-05-08T05:29:35.935788+0000","last_clean":"2024-05-08T05:29:35.935788+0000","last_became_active":"2024-05-08T05:23:55.814786+0000","last_became_peered":"2024-05-08T05:23:55.814786+0000","last_unstale":"2024-05-08T05:29:35.935788+0000","last_undegraded":"2024-05-08T05:29:35.935788+0000","last_fullsized":"2024-05-08T05:29:35.935788+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_clean_scrub_stamp":"2024-05-08T05:20:32.659283+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:11.169455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933610+0000","last_change":"2024-05-08T05:25:37.057287+0000","last_active":"2024-05-08T05:29:33.933610+0000","last_peered":"2024-05-08T05:29:33.933610+0000","last_clean":"2024-05-08T05:29:33.933610+0000","last_became_active":"2024-05-08T05:25:37.056871+0000","last_became_peered":"2024-05-08T05:25:37.056871+0000","last_unstale":"2024-05-08T05:29:33.933610+0000","last_undegraded":"2024-05-08T05:29:33.933610+0000","last_fullsized":"2024-05-08T05:29:33.933610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:19.004199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.679912+0000","last_change":"2024-05-08T05:25:25.664795+0000","last_active":"2024-05-08T05:29:34.679912+0000","last_peered":"2024-05-08T05:29:34.679912+0000","last_clean":"2024-05-08T05:29:34.679912+0000","last_became_active":"2024-05-08T05:25:25.664444+0000","last_became_peered":"2024-05-08T05:25:25.664444+0000","last_unstale":"2024-05-08T05:29:34.679912+0000","last_undegraded":"2024-05-08T05:29:34.679912+0000","last_fullsized":"2024-05-08T05:29:34.679912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:27.172530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085169+0000","last_change":"2024-05-08T05:25:22.645205+0000","last_active":"2024-05-08T05:29:34.085169+0000","last_peered":"2024-05-08T05:29:34.085169+0000","last_clean":"2024-05-08T05:29:34.085169+0000","last_became_active":"2024-05-08T05:25:22.644720+0000","last_became_peered":"2024-05-08T05:25:22.644720+0000","last_unstale":"2024-05-08T05:29:34.085169+0000","last_undegraded":"2024-05-08T05:29:34.085169+0000","last_fullsized":"2024-05-08T05:29:34.085169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:05.630188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087942+0000","last_change":"2024-05-08T05:25:37.055151+0000","last_active":"2024-05-08T05:29:34.087942+0000","last_peered":"2024-05-08T05:29:34.087942+0000","last_clean":"2024-05-08T05:29:34.087942+0000","last_became_active":"2024-05-08T05:25:37.054737+0000","last_became_peered":"2024-05-08T05:25:37.054737+0000","last_unstale":"2024-05-08T05:29:34.087942+0000","last_undegraded":"2024-05-08T05:29:34.087942+0000","last_fullsized":"2024-05-08T05:29:34.087942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:49.135453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088311+0000","last_change":"2024-05-08T05:26:05.148665+0000","last_active":"2024-05-08T05:29:34.088311+0000","last_peered":"2024-05-08T05:29:34.088311+0000","last_clean":"2024-05-08T05:29:34.088311+0000","last_became_active":"2024-05-08T05:26:05.148283+0000","last_became_peered":"2024-05-08T05:26:05.148283+0000","last_unstale":"2024-05-08T05:29:34.088311+0000","last_undegraded":"2024-05-08T05:29:34.088311+0000","last_fullsized":"2024-05-08T05:29:34.088311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:26.197521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680406+0000","last_change":"2024-05-08T05:25:25.669130+0000","last_active":"2024-05-08T05:29:34.680406+0000","last_peered":"2024-05-08T05:29:34.680406+0000","last_clean":"2024-05-08T05:29:34.680406+0000","last_became_active":"2024-05-08T05:25:25.668812+0000","last_became_peered":"2024-05-08T05:25:25.668812+0000","last_unstale":"2024-05-08T05:29:34.680406+0000","last_undegraded":"2024-05-08T05:29:34.680406+0000","last_fullsized":"2024-05-08T05:29:34.680406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:34.514248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925554+0000","last_change":"2024-05-08T05:25:37.057146+0000","last_active":"2024-05-08T05:29:33.925554+0000","last_peered":"2024-05-08T05:29:33.925554+0000","last_clean":"2024-05-08T05:29:33.925554+0000","last_became_active":"2024-05-08T05:25:37.056706+0000","last_became_peered":"2024-05-08T05:25:37.056706+0000","last_unstale":"2024-05-08T05:29:33.925554+0000","last_undegraded":"2024-05-08T05:29:33.925554+0000","last_fullsized":"2024-05-08T05:29:33.925554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:09.724088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'105","reported_seq":244,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:37.570728+0000","last_change":"2024-05-08T05:25:22.644573+0000","last_active":"2024-05-08T05:29:37.570728+0000","last_peered":"2024-05-08T05:29:37.570728+0000","last_clean":"2024-05-08T05:29:37.570728+0000","last_became_active":"2024-05-08T05:25:22.644231+0000","last_became_peered":"2024-05-08T05:25:22.644231+0000","last_unstale":"2024-05-08T05:29:37.570728+0000","last_undegraded":"2024-05-08T05:29:37.570728+0000","last_fullsized":"2024-05-08T05:29:37.570728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:02.931823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":192,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925815+0000","last_change":"2024-05-08T05:25:25.671036+0000","last_active":"2024-05-08T05:29:33.925815+0000","last_peered":"2024-05-08T05:29:33.925815+0000","last_clean":"2024-05-08T05:29:33.925815+0000","last_became_active":"2024-05-08T05:25:25.669170+0000","last_became_peered":"2024-05-08T05:25:25.669170+0000","last_unstale":"2024-05-08T05:29:33.925815+0000","last_undegraded":"2024-05-08T05:29:33.925815+0000","last_fullsized":"2024-05-08T05:29:33.925815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:18.963232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826699+0000","last_change":"2024-05-08T05:25:37.050656+0000","last_active":"2024-05-08T05:29:33.826699+0000","last_peered":"2024-05-08T05:29:33.826699+0000","last_clean":"2024-05-08T05:29:33.826699+0000","last_became_active":"2024-05-08T05:25:37.050051+0000","last_became_peered":"2024-05-08T05:25:37.050051+0000","last_unstale":"2024-05-08T05:29:33.826699+0000","last_undegraded":"2024-05-08T05:29:33.826699+0000","last_fullsized":"2024-05-08T05:29:33.826699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:56:07.316216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093378+0000","last_change":"2024-05-08T05:25:23.614874+0000","last_active":"2024-05-08T05:29:34.093378+0000","last_peered":"2024-05-08T05:29:34.093378+0000","last_clean":"2024-05-08T05:29:34.093378+0000","last_became_active":"2024-05-08T05:25:23.614550+0000","last_became_peered":"2024-05-08T05:25:23.614550+0000","last_unstale":"2024-05-08T05:29:34.093378+0000","last_undegraded":"2024-05-08T05:29:34.093378+0000","last_fullsized":"2024-05-08T05:29:34.093378+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:10.183477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088730+0000","last_change":"2024-05-08T05:25:25.658710+0000","last_active":"2024-05-08T05:29:34.088730+0000","last_peered":"2024-05-08T05:29:34.088730+0000","last_clean":"2024-05-08T05:29:34.088730+0000","last_became_active":"2024-05-08T05:25:25.658393+0000","last_became_peered":"2024-05-08T05:25:25.658393+0000","last_unstale":"2024-05-08T05:29:34.088730+0000","last_undegraded":"2024-05-08T05:29:34.088730+0000","last_fullsized":"2024-05-08T05:29:34.088730+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:02.629256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092991+0000","last_change":"2024-05-08T05:25:37.059976+0000","last_active":"2024-05-08T05:29:34.092991+0000","last_peered":"2024-05-08T05:29:34.092991+0000","last_clean":"2024-05-08T05:29:34.092991+0000","last_became_active":"2024-05-08T05:25:37.059474+0000","last_became_peered":"2024-05-08T05:25:37.059474+0000","last_unstale":"2024-05-08T05:29:34.092991+0000","last_undegraded":"2024-05-08T05:29:34.092991+0000","last_fullsized":"2024-05-08T05:29:34.092991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:51.946242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825833+0000","last_change":"2024-05-08T05:25:22.748271+0000","last_active":"2024-05-08T05:29:33.825833+0000","last_peered":"2024-05-08T05:29:33.825833+0000","last_clean":"2024-05-08T05:29:33.825833+0000","last_became_active":"2024-05-08T05:25:22.747915+0000","last_became_peered":"2024-05-08T05:25:22.747915+0000","last_unstale":"2024-05-08T05:29:33.825833+0000","last_undegraded":"2024-05-08T05:29:33.825833+0000","last_fullsized":"2024-05-08T05:29:33.825833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:35.262298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925478+0000","last_change":"2024-05-08T05:25:25.668722+0000","last_active":"2024-05-08T05:29:33.925478+0000","last_peered":"2024-05-08T05:29:33.925478+0000","last_clean":"2024-05-08T05:29:33.925478+0000","last_became_active":"2024-05-08T05:25:25.668223+0000","last_became_peered":"2024-05-08T05:25:25.668223+0000","last_unstale":"2024-05-08T05:29:33.925478+0000","last_undegraded":"2024-05-08T05:29:33.925478+0000","last_fullsized":"2024-05-08T05:29:33.925478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:43.511567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825206+0000","last_change":"2024-05-08T05:25:37.056131+0000","last_active":"2024-05-08T05:29:33.825206+0000","last_peered":"2024-05-08T05:29:33.825206+0000","last_clean":"2024-05-08T05:29:33.825206+0000","last_became_active":"2024-05-08T05:25:37.055424+0000","last_became_peered":"2024-05-08T05:25:37.055424+0000","last_unstale":"2024-05-08T05:29:33.825206+0000","last_undegraded":"2024-05-08T05:29:33.825206+0000","last_fullsized":"2024-05-08T05:29:33.825206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:43.279408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925896+0000","last_change":"2024-05-08T05:25:22.643908+0000","last_active":"2024-05-08T05:29:33.925896+0000","last_peered":"2024-05-08T05:29:33.925896+0000","last_clean":"2024-05-08T05:29:33.925896+0000","last_became_active":"2024-05-08T05:25:22.643542+0000","last_became_peered":"2024-05-08T05:25:22.643542+0000","last_unstale":"2024-05-08T05:29:33.925896+0000","last_undegraded":"2024-05-08T05:29:33.925896+0000","last_fullsized":"2024-05-08T05:29:33.925896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:01.671285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087815+0000","last_change":"2024-05-08T05:25:25.667843+0000","last_active":"2024-05-08T05:29:34.087815+0000","last_peered":"2024-05-08T05:29:34.087815+0000","last_clean":"2024-05-08T05:29:34.087815+0000","last_became_active":"2024-05-08T05:25:25.667462+0000","last_became_peered":"2024-05-08T05:25:25.667462+0000","last_unstale":"2024-05-08T05:29:34.087815+0000","last_undegraded":"2024-05-08T05:29:34.087815+0000","last_fullsized":"2024-05-08T05:29:34.087815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:51.325104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092927+0000","last_change":"2024-05-08T05:25:37.053017+0000","last_active":"2024-05-08T05:29:34.092927+0000","last_peered":"2024-05-08T05:29:34.092927+0000","last_clean":"2024-05-08T05:29:34.092927+0000","last_became_active":"2024-05-08T05:25:37.052601+0000","last_became_peered":"2024-05-08T05:25:37.052601+0000","last_unstale":"2024-05-08T05:29:34.092927+0000","last_undegraded":"2024-05-08T05:29:34.092927+0000","last_fullsized":"2024-05-08T05:29:34.092927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:35.393835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826911+0000","last_change":"2024-05-08T05:25:22.644564+0000","last_active":"2024-05-08T05:29:33.826911+0000","last_peered":"2024-05-08T05:29:33.826911+0000","last_clean":"2024-05-08T05:29:33.826911+0000","last_became_active":"2024-05-08T05:25:22.644170+0000","last_became_peered":"2024-05-08T05:25:22.644170+0000","last_unstale":"2024-05-08T05:29:33.826911+0000","last_undegraded":"2024-05-08T05:29:33.826911+0000","last_fullsized":"2024-05-08T05:29:33.826911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:24.757574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926667+0000","last_change":"2024-05-08T05:25:25.676983+0000","last_active":"2024-05-08T05:29:33.926667+0000","last_peered":"2024-05-08T05:29:33.926667+0000","last_clean":"2024-05-08T05:29:33.926667+0000","last_became_active":"2024-05-08T05:25:25.676610+0000","last_became_peered":"2024-05-08T05:25:25.676610+0000","last_unstale":"2024-05-08T05:29:33.926667+0000","last_undegraded":"2024-05-08T05:29:33.926667+0000","last_fullsized":"2024-05-08T05:29:33.926667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:39.137935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092999+0000","last_change":"2024-05-08T05:25:37.059381+0000","last_active":"2024-05-08T05:29:34.092999+0000","last_peered":"2024-05-08T05:29:34.092999+0000","last_clean":"2024-05-08T05:29:34.092999+0000","last_became_active":"2024-05-08T05:25:37.056851+0000","last_became_peered":"2024-05-08T05:25:37.056851+0000","last_unstale":"2024-05-08T05:29:34.092999+0000","last_undegraded":"2024-05-08T05:29:34.092999+0000","last_fullsized":"2024-05-08T05:29:34.092999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:19.339331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925636+0000","last_change":"2024-05-08T05:25:22.648099+0000","last_active":"2024-05-08T05:29:33.925636+0000","last_peered":"2024-05-08T05:29:33.925636+0000","last_clean":"2024-05-08T05:29:33.925636+0000","last_became_active":"2024-05-08T05:25:22.647736+0000","last_became_peered":"2024-05-08T05:25:22.647736+0000","last_unstale":"2024-05-08T05:29:33.925636+0000","last_undegraded":"2024-05-08T05:29:33.925636+0000","last_fullsized":"2024-05-08T05:29:33.925636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:12.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085452+0000","last_change":"2024-05-08T05:25:25.667565+0000","last_active":"2024-05-08T05:29:34.085452+0000","last_peered":"2024-05-08T05:29:34.085452+0000","last_clean":"2024-05-08T05:29:34.085452+0000","last_became_active":"2024-05-08T05:25:25.667217+0000","last_became_peered":"2024-05-08T05:25:25.667217+0000","last_unstale":"2024-05-08T05:29:34.085452+0000","last_undegraded":"2024-05-08T05:29:34.085452+0000","last_fullsized":"2024-05-08T05:29:34.085452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:05.228227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825918+0000","last_change":"2024-05-08T05:25:37.056923+0000","last_active":"2024-05-08T05:29:33.825918+0000","last_peered":"2024-05-08T05:29:33.825918+0000","last_clean":"2024-05-08T05:29:33.825918+0000","last_became_active":"2024-05-08T05:25:37.056496+0000","last_became_peered":"2024-05-08T05:25:37.056496+0000","last_unstale":"2024-05-08T05:29:33.825918+0000","last_undegraded":"2024-05-08T05:29:33.825918+0000","last_fullsized":"2024-05-08T05:29:33.825918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:21.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'9","reported_seq":253,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:38.684318+0000","last_change":"2024-05-08T05:25:23.614900+0000","last_active":"2024-05-08T05:29:38.684318+0000","last_peered":"2024-05-08T05:29:38.684318+0000","last_clean":"2024-05-08T05:29:38.684318+0000","last_became_active":"2024-05-08T05:25:23.614510+0000","last_became_peered":"2024-05-08T05:25:23.614510+0000","last_unstale":"2024-05-08T05:29:38.684318+0000","last_undegraded":"2024-05-08T05:29:38.684318+0000","last_fullsized":"2024-05-08T05:29:38.684318+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:19.539656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2484,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085833+0000","last_change":"2024-05-08T05:25:25.669016+0000","last_active":"2024-05-08T05:29:34.085833+0000","last_peered":"2024-05-08T05:29:34.085833+0000","last_clean":"2024-05-08T05:29:34.085833+0000","last_became_active":"2024-05-08T05:25:25.667809+0000","last_became_peered":"2024-05-08T05:25:25.667809+0000","last_unstale":"2024-05-08T05:29:34.085833+0000","last_undegraded":"2024-05-08T05:29:34.085833+0000","last_fullsized":"2024-05-08T05:29:34.085833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:56.475622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088613+0000","last_change":"2024-05-08T05:25:37.053630+0000","last_active":"2024-05-08T05:29:34.088613+0000","last_peered":"2024-05-08T05:29:34.088613+0000","last_clean":"2024-05-08T05:29:34.088613+0000","last_became_active":"2024-05-08T05:25:37.053069+0000","last_became_peered":"2024-05-08T05:25:37.053069+0000","last_unstale":"2024-05-08T05:29:34.088613+0000","last_undegraded":"2024-05-08T05:29:34.088613+0000","last_fullsized":"2024-05-08T05:29:34.088613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:49:56.693447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:37.200537+0000","last_change":"2024-05-08T05:25:22.645284+0000","last_active":"2024-05-08T05:29:37.200537+0000","last_peered":"2024-05-08T05:29:37.200537+0000","last_clean":"2024-05-08T05:29:37.200537+0000","last_became_active":"2024-05-08T05:25:22.644897+0000","last_became_peered":"2024-05-08T05:25:22.644897+0000","last_unstale":"2024-05-08T05:29:37.200537+0000","last_undegraded":"2024-05-08T05:29:37.200537+0000","last_fullsized":"2024-05-08T05:29:37.200537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:53:31.423412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085358+0000","last_change":"2024-05-08T05:25:25.664930+0000","last_active":"2024-05-08T05:29:34.085358+0000","last_peered":"2024-05-08T05:29:34.085358+0000","last_clean":"2024-05-08T05:29:34.085358+0000","last_became_active":"2024-05-08T05:25:25.664395+0000","last_became_peered":"2024-05-08T05:25:25.664395+0000","last_unstale":"2024-05-08T05:29:34.085358+0000","last_undegraded":"2024-05-08T05:29:34.085358+0000","last_fullsized":"2024-05-08T05:29:34.085358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:37.971474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933025+0000","last_change":"2024-05-08T05:25:37.053258+0000","last_active":"2024-05-08T05:29:33.933025+0000","last_peered":"2024-05-08T05:29:33.933025+0000","last_clean":"2024-05-08T05:29:33.933025+0000","last_became_active":"2024-05-08T05:25:37.052786+0000","last_became_peered":"2024-05-08T05:25:37.052786+0000","last_unstale":"2024-05-08T05:29:33.933025+0000","last_undegraded":"2024-05-08T05:29:33.933025+0000","last_fullsized":"2024-05-08T05:29:33.933025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:07.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680512+0000","last_change":"2024-05-08T05:27:05.770358+0000","last_active":"2024-05-08T05:29:34.680512+0000","last_peered":"2024-05-08T05:29:34.680512+0000","last_clean":"2024-05-08T05:29:34.680512+0000","last_became_active":"2024-05-08T05:27:05.766778+0000","last_became_peered":"2024-05-08T05:27:05.766778+0000","last_unstale":"2024-05-08T05:29:34.680512+0000","last_undegraded":"2024-05-08T05:29:34.680512+0000","last_fullsized":"2024-05-08T05:29:34.680512+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:48:52.941549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085900+0000","last_change":"2024-05-08T05:25:25.663771+0000","last_active":"2024-05-08T05:29:34.085900+0000","last_peered":"2024-05-08T05:29:34.085900+0000","last_clean":"2024-05-08T05:29:34.085900+0000","last_became_active":"2024-05-08T05:25:25.663281+0000","last_became_peered":"2024-05-08T05:25:25.663281+0000","last_unstale":"2024-05-08T05:29:34.085900+0000","last_undegraded":"2024-05-08T05:29:34.085900+0000","last_fullsized":"2024-05-08T05:29:34.085900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:43.892781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087991+0000","last_change":"2024-05-08T05:25:25.665446+0000","last_active":"2024-05-08T05:29:34.087991+0000","last_peered":"2024-05-08T05:29:34.087991+0000","last_clean":"2024-05-08T05:29:34.087991+0000","last_became_active":"2024-05-08T05:25:25.664389+0000","last_became_peered":"2024-05-08T05:25:25.664389+0000","last_unstale":"2024-05-08T05:29:34.087991+0000","last_undegraded":"2024-05-08T05:29:34.087991+0000","last_fullsized":"2024-05-08T05:29:34.087991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826995+0000","last_change":"2024-05-08T05:26:05.149870+0000","last_active":"2024-05-08T05:29:33.826995+0000","last_peered":"2024-05-08T05:29:33.826995+0000","last_clean":"2024-05-08T05:29:33.826995+0000","last_became_active":"2024-05-08T05:26:05.149476+0000","last_became_peered":"2024-05-08T05:26:05.149476+0000","last_unstale":"2024-05-08T05:29:33.826995+0000","last_undegraded":"2024-05-08T05:29:33.826995+0000","last_fullsized":"2024-05-08T05:29:33.826995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:40:43.401832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826865+0000","last_change":"2024-05-08T05:25:25.667836+0000","last_active":"2024-05-08T05:29:33.826865+0000","last_peered":"2024-05-08T05:29:33.826865+0000","last_clean":"2024-05-08T05:29:33.826865+0000","last_became_active":"2024-05-08T05:25:25.667496+0000","last_became_peered":"2024-05-08T05:25:25.667496+0000","last_unstale":"2024-05-08T05:29:33.826865+0000","last_undegraded":"2024-05-08T05:29:33.826865+0000","last_fullsized":"2024-05-08T05:29:33.826865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:01.482837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.089014+0000","last_change":"2024-05-08T05:25:25.667279+0000","last_active":"2024-05-08T05:29:34.089014+0000","last_peered":"2024-05-08T05:29:34.089014+0000","last_clean":"2024-05-08T05:29:34.089014+0000","last_became_active":"2024-05-08T05:25:25.666960+0000","last_became_peered":"2024-05-08T05:25:25.666960+0000","last_unstale":"2024-05-08T05:29:34.089014+0000","last_undegraded":"2024-05-08T05:29:34.089014+0000","last_fullsized":"2024-05-08T05:29:34.089014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:06.335850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680319+0000","last_change":"2024-05-08T05:25:25.667644+0000","last_active":"2024-05-08T05:29:34.680319+0000","last_peered":"2024-05-08T05:29:34.680319+0000","last_clean":"2024-05-08T05:29:34.680319+0000","last_became_active":"2024-05-08T05:25:25.667133+0000","last_became_peered":"2024-05-08T05:25:25.667133+0000","last_unstale":"2024-05-08T05:29:34.680319+0000","last_undegraded":"2024-05-08T05:29:34.680319+0000","last_fullsized":"2024-05-08T05:29:34.680319+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:03.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087979+0000","last_change":"2024-05-08T05:25:25.670761+0000","last_active":"2024-05-08T05:29:34.087979+0000","last_peered":"2024-05-08T05:29:34.087979+0000","last_clean":"2024-05-08T05:29:34.087979+0000","last_became_active":"2024-05-08T05:25:25.670342+0000","last_became_peered":"2024-05-08T05:25:25.670342+0000","last_unstale":"2024-05-08T05:29:34.087979+0000","last_undegraded":"2024-05-08T05:29:34.087979+0000","last_fullsized":"2024-05-08T05:29:34.087979+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:08.069332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827277+0000","last_change":"2024-05-08T05:25:25.664512+0000","last_active":"2024-05-08T05:29:33.827277+0000","last_peered":"2024-05-08T05:29:33.827277+0000","last_clean":"2024-05-08T05:29:33.827277+0000","last_became_active":"2024-05-08T05:25:25.664191+0000","last_became_peered":"2024-05-08T05:25:25.664191+0000","last_unstale":"2024-05-08T05:29:33.827277+0000","last_undegraded":"2024-05-08T05:29:33.827277+0000","last_fullsized":"2024-05-08T05:29:33.827277+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:07.586637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.927122+0000","last_change":"2024-05-08T05:25:25.675711+0000","last_active":"2024-05-08T05:29:33.927122+0000","last_peered":"2024-05-08T05:29:33.927122+0000","last_clean":"2024-05-08T05:29:33.927122+0000","last_became_active":"2024-05-08T05:25:25.671890+0000","last_became_peered":"2024-05-08T05:25:25.671890+0000","last_unstale":"2024-05-08T05:29:33.927122+0000","last_undegraded":"2024-05-08T05:29:33.927122+0000","last_fullsized":"2024-05-08T05:29:33.927122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:57:49.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933145+0000","last_change":"2024-05-08T05:25:25.662603+0000","last_active":"2024-05-08T05:29:33.933145+0000","last_peered":"2024-05-08T05:29:33.933145+0000","last_clean":"2024-05-08T05:29:33.933145+0000","last_became_active":"2024-05-08T05:25:25.662287+0000","last_became_peered":"2024-05-08T05:25:25.662287+0000","last_unstale":"2024-05-08T05:29:33.933145+0000","last_undegraded":"2024-05-08T05:29:33.933145+0000","last_fullsized":"2024-05-08T05:29:33.933145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:02.152557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926090+0000","last_change":"2024-05-08T05:25:25.667309+0000","last_active":"2024-05-08T05:29:33.926090+0000","last_peered":"2024-05-08T05:29:33.926090+0000","last_clean":"2024-05-08T05:29:33.926090+0000","last_became_active":"2024-05-08T05:25:25.666968+0000","last_became_peered":"2024-05-08T05:25:25.666968+0000","last_unstale":"2024-05-08T05:29:33.926090+0000","last_undegraded":"2024-05-08T05:29:33.926090+0000","last_fullsized":"2024-05-08T05:29:33.926090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:28.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826130+0000","last_change":"2024-05-08T05:25:25.673579+0000","last_active":"2024-05-08T05:29:33.826130+0000","last_peered":"2024-05-08T05:29:33.826130+0000","last_clean":"2024-05-08T05:29:33.826130+0000","last_became_active":"2024-05-08T05:25:25.673130+0000","last_became_peered":"2024-05-08T05:25:25.673130+0000","last_unstale":"2024-05-08T05:29:33.826130+0000","last_undegraded":"2024-05-08T05:29:33.826130+0000","last_fullsized":"2024-05-08T05:29:33.826130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:01.291319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680150+0000","last_change":"2024-05-08T05:25:25.667359+0000","last_active":"2024-05-08T05:29:34.680150+0000","last_peered":"2024-05-08T05:29:34.680150+0000","last_clean":"2024-05-08T05:29:34.680150+0000","last_became_active":"2024-05-08T05:25:25.667044+0000","last_became_peered":"2024-05-08T05:25:25.667044+0000","last_unstale":"2024-05-08T05:29:34.680150+0000","last_undegraded":"2024-05-08T05:29:34.680150+0000","last_fullsized":"2024-05-08T05:29:34.680150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:54.973053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088126+0000","last_change":"2024-05-08T05:26:05.150407+0000","last_active":"2024-05-08T05:29:34.088126+0000","last_peered":"2024-05-08T05:29:34.088126+0000","last_clean":"2024-05-08T05:29:34.088126+0000","last_became_active":"2024-05-08T05:26:05.149916+0000","last_became_peered":"2024-05-08T05:26:05.149916+0000","last_unstale":"2024-05-08T05:29:34.088126+0000","last_undegraded":"2024-05-08T05:29:34.088126+0000","last_fullsized":"2024-05-08T05:29:34.088126+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:05:57.075928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107400,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":805470,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2322,"num_read_kb":1391,"num_write":523,"num_write_kb":2404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2416410,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":400,"ondisk_log_size":400,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678082,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":532312,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874311,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1216512,"data_stored":1519983,"data_compressed":86432,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037837,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":928128,"data_compressed":54719,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234066,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":924298,"data_compressed":54719,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462998,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":1130768,"data_compressed":71230,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626524,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":527243,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574965,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":601091,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738489,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":529090,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":602904,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182507,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1247476,"data_compressed":69921,"data_compressed_allocated":94208,"data_compressed_original":798720,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986279,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":531737,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822753,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1138688,"data_stored":1452428,"data_compressed":86432,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":75493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332366,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":404017,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:29:43.777 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-05-08T05:29:43.777 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph pg dump --format=json 2024-05-08T05:29:44.070 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:44.318 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:44 smithi059 ceph-mon[30304]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:29:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:44 smithi143 ceph-mon[34337]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:29:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:44 smithi096 ceph-mon[34253]: from='client.15390 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:29:45.267 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:45 smithi059 ceph-mon[30304]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 103 KiB/s wr, 57 op/s 2024-05-08T05:29:45.394 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:45 smithi143 ceph-mon[34337]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 103 KiB/s wr, 57 op/s 2024-05-08T05:29:45.431 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:45 smithi096 ceph-mon[34253]: pgmap v8: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 103 KiB/s wr, 57 op/s 2024-05-08T05:29:45.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.912+0000 7fec28346700 1 -- 172.21.15.59:0/246880114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec201076c0 msgr2=0x7fec2010f290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.912+0000 7fec28346700 1 --2- 172.21.15.59:0/246880114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec201076c0 0x7fec2010f290 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fec1800b0d0 tx=0x7fec1800b3e0 comp rx=0 tx=0).stop 2024-05-08T05:29:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.912+0000 7fec28346700 1 -- 172.21.15.59:0/246880114 shutdown_connections 2024-05-08T05:29:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.912+0000 7fec28346700 1 --2- 172.21.15.59:0/246880114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec201076c0 0x7fec2010f290 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.912+0000 7fec28346700 1 --2- 172.21.15.59:0/246880114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec20102630 0x7fec20102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.912+0000 7fec28346700 1 --2- 172.21.15.59:0/246880114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec20102fe0 0x7fec20106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.912+0000 7fec28346700 1 -- 172.21.15.59:0/246880114 >> 172.21.15.59:0/246880114 conn(0x7fec20079350 msgr2=0x7fec20079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.912+0000 7fec28346700 1 -- 172.21.15.59:0/246880114 shutdown_connections 2024-05-08T05:29:45.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.912+0000 7fec28346700 1 -- 172.21.15.59:0/246880114 wait complete. 2024-05-08T05:29:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec28346700 1 Processor -- start 2024-05-08T05:29:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec28346700 1 -- start start 2024-05-08T05:29:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec28346700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec20102630 0x7fec2019a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec28346700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec20102fe0 0x7fec2019ae80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec28346700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec201076c0 0x7fec201a59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:45.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec28346700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fec20112560 con 0x7fec20102fe0 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec28346700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fec201123e0 con 0x7fec20102630 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec28346700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fec201126e0 con 0x7fec201076c0 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec258e1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec20102fe0 0x7fec2019ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec258e1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec20102fe0 0x7fec2019ae80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53800/0 (socket says 172.21.15.59:53800) 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec258e1700 1 -- 172.21.15.59:0/653887131 learned_addr learned my addr 172.21.15.59:0/653887131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec260e2700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec20102630 0x7fec2019a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec268e3700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec201076c0 0x7fec201a59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec258e1700 1 -- 172.21.15.59:0/653887131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec201076c0 msgr2=0x7fec201a59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec258e1700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec201076c0 0x7fec201a59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec258e1700 1 -- 172.21.15.59:0/653887131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec20102630 msgr2=0x7fec2019a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec258e1700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec20102630 0x7fec2019a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.913+0000 7fec258e1700 1 -- 172.21.15.59:0/653887131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec201a5f00 con 0x7fec20102fe0 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.914+0000 7fec258e1700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec20102fe0 0x7fec2019ae80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fec1000d980 tx=0x7fec1000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.914+0000 7fec177fe700 1 -- 172.21.15.59:0/653887131 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec10017070 con 0x7fec20102fe0 2024-05-08T05:29:45.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.914+0000 7fec177fe700 1 -- 172.21.15.59:0/653887131 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec1000fd40 con 0x7fec20102fe0 2024-05-08T05:29:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.914+0000 7fec177fe700 1 -- 172.21.15.59:0/653887131 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1000e3b0 con 0x7fec20102fe0 2024-05-08T05:29:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.914+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec201a6130 con 0x7fec20102fe0 2024-05-08T05:29:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.914+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec201a6640 con 0x7fec20102fe0 2024-05-08T05:29:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.915+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec20103ac0 con 0x7fec20102fe0 2024-05-08T05:29:45.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.916+0000 7fec177fe700 1 -- 172.21.15.59:0/653887131 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fec10010610 con 0x7fec20102fe0 2024-05-08T05:29:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.917+0000 7fec177fe700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fec0c0b2b10 0x7fec0c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.917+0000 7fec260e2700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fec0c0b2b10 0x7fec0c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.917+0000 7fec177fe700 1 -- 172.21.15.59:0/653887131 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fec100e2640 con 0x7fec20102fe0 2024-05-08T05:29:45.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.918+0000 7fec260e2700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fec0c0b2b10 0x7fec0c0b4fd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00ac50 tx=0x7fec1c00a380 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:45.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:45.920+0000 7fec177fe700 1 -- 172.21.15.59:0/653887131 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fec100a9a20 con 0x7fec20102fe0 2024-05-08T05:29:46.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.134+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fec200638e0 con 0x7fec0c0b2b10 2024-05-08T05:29:46.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.143+0000 7fec177fe700 1 -- 172.21.15.59:0/653887131 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339751 (secure 0 0 0) 0x7fec200638e0 con 0x7fec0c0b2b10 2024-05-08T05:29:46.148 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:29:46.150 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fec0c0b2b10 msgr2=0x7fec0c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fec0c0b2b10 0x7fec0c0b4fd0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fec1c00ac50 tx=0x7fec1c00a380 comp rx=0 tx=0).stop 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec20102fe0 msgr2=0x7fec2019ae80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec20102fe0 0x7fec2019ae80 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fec1000d980 tx=0x7fec1000dc90 comp rx=0 tx=0).stop 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 shutdown_connections 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fec0c0b2b10 0x7fec0c0b4fd0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fec201076c0 0x7fec201a59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fec20102630 0x7fec2019a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 --2- 172.21.15.59:0/653887131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fec20102fe0 0x7fec2019ae80 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 >> 172.21.15.59:0/653887131 conn(0x7fec20079350 msgr2=0x7fec20100380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 shutdown_connections 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:46.148+0000 7fec28346700 1 -- 172.21.15.59:0/653887131 wait complete. 2024-05-08T05:29:46.151 INFO:teuthology.orchestra.run.smithi059.stderr:dumped all 2024-05-08T05:29:46.962 INFO:teuthology.orchestra.run.smithi059.stdout:{"pg_ready":true,"pg_map":{"version":9,"stamp":"2024-05-08T05:29:45.841852+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1396072,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":2516,"num_read_kb":1664,"num_write":660,"num_write_kb":4556,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":2951840,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1001,"ondisk_log_size":1001,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":361824,"kb_used_data":12632,"kb_used_omap":18,"kb_used_meta":349037,"kb_avail":1124580000,"statfs":{"total":1151940427776,"available":1151569920000,"internally_reserved":0,"allocated":12935168,"data_stored":10527458,"data_compressed":636033,"data_compressed_allocated":860160,"data_compressed_original":5050368,"omap_allocated":19141,"internal_metadata":357414203},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":200325,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":420,"num_read_kb":465,"num_write":145,"num_write_kb":922,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.992597"},"pg_stats":[{"pgid":"3.3f","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088374+0000","last_change":"2024-05-08T05:25:25.667895+0000","last_active":"2024-05-08T05:29:34.088374+0000","last_peered":"2024-05-08T05:29:34.088374+0000","last_clean":"2024-05-08T05:29:34.088374+0000","last_became_active":"2024-05-08T05:25:25.667563+0000","last_became_peered":"2024-05-08T05:25:25.667563+0000","last_unstale":"2024-05-08T05:29:34.088374+0000","last_undegraded":"2024-05-08T05:29:34.088374+0000","last_fullsized":"2024-05-08T05:29:34.088374+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:09:47.677082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088642+0000","last_change":"2024-05-08T05:25:25.673346+0000","last_active":"2024-05-08T05:29:34.088642+0000","last_peered":"2024-05-08T05:29:34.088642+0000","last_clean":"2024-05-08T05:29:34.088642+0000","last_became_active":"2024-05-08T05:25:25.673011+0000","last_became_peered":"2024-05-08T05:25:25.673011+0000","last_unstale":"2024-05-08T05:29:34.088642+0000","last_undegraded":"2024-05-08T05:29:34.088642+0000","last_fullsized":"2024-05-08T05:29:34.088642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:21:05.091177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.089071+0000","last_change":"2024-05-08T05:25:25.670469+0000","last_active":"2024-05-08T05:29:34.089071+0000","last_peered":"2024-05-08T05:29:34.089071+0000","last_clean":"2024-05-08T05:29:34.089071+0000","last_became_active":"2024-05-08T05:25:25.669700+0000","last_became_peered":"2024-05-08T05:25:25.669700+0000","last_unstale":"2024-05-08T05:29:34.089071+0000","last_undegraded":"2024-05-08T05:29:34.089071+0000","last_fullsized":"2024-05-08T05:29:34.089071+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:54:06.046633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093206+0000","last_change":"2024-05-08T05:25:25.659937+0000","last_active":"2024-05-08T05:29:34.093206+0000","last_peered":"2024-05-08T05:29:34.093206+0000","last_clean":"2024-05-08T05:29:34.093206+0000","last_became_active":"2024-05-08T05:25:25.658892+0000","last_became_peered":"2024-05-08T05:25:25.658892+0000","last_unstale":"2024-05-08T05:29:34.093206+0000","last_undegraded":"2024-05-08T05:29:34.093206+0000","last_fullsized":"2024-05-08T05:29:34.093206+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:32:10.210034+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925563+0000","last_change":"2024-05-08T05:25:25.669132+0000","last_active":"2024-05-08T05:29:33.925563+0000","last_peered":"2024-05-08T05:29:33.925563+0000","last_clean":"2024-05-08T05:29:33.925563+0000","last_became_active":"2024-05-08T05:25:25.668556+0000","last_became_peered":"2024-05-08T05:25:25.668556+0000","last_unstale":"2024-05-08T05:29:33.925563+0000","last_undegraded":"2024-05-08T05:29:33.925563+0000","last_fullsized":"2024-05-08T05:29:33.925563+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:03:28.443763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936400+0000","last_change":"2024-05-08T05:25:25.668978+0000","last_active":"2024-05-08T05:29:33.936400+0000","last_peered":"2024-05-08T05:29:33.936400+0000","last_clean":"2024-05-08T05:29:33.936400+0000","last_became_active":"2024-05-08T05:25:25.668647+0000","last_became_peered":"2024-05-08T05:25:25.668647+0000","last_unstale":"2024-05-08T05:29:33.936400+0000","last_undegraded":"2024-05-08T05:29:33.936400+0000","last_fullsized":"2024-05-08T05:29:33.936400+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:07:00.962620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085103+0000","last_change":"2024-05-08T05:25:25.664784+0000","last_active":"2024-05-08T05:29:34.085103+0000","last_peered":"2024-05-08T05:29:34.085103+0000","last_clean":"2024-05-08T05:29:34.085103+0000","last_became_active":"2024-05-08T05:25:25.664390+0000","last_became_peered":"2024-05-08T05:25:25.664390+0000","last_unstale":"2024-05-08T05:29:34.085103+0000","last_undegraded":"2024-05-08T05:29:34.085103+0000","last_fullsized":"2024-05-08T05:29:34.085103+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:08:35.358796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925616+0000","last_change":"2024-05-08T05:25:25.677139+0000","last_active":"2024-05-08T05:29:33.925616+0000","last_peered":"2024-05-08T05:29:33.925616+0000","last_clean":"2024-05-08T05:29:33.925616+0000","last_became_active":"2024-05-08T05:25:25.676444+0000","last_became_peered":"2024-05-08T05:25:25.676444+0000","last_unstale":"2024-05-08T05:29:33.925616+0000","last_undegraded":"2024-05-08T05:29:33.925616+0000","last_fullsized":"2024-05-08T05:29:33.925616+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:56.704152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826525+0000","last_change":"2024-05-08T05:25:25.675593+0000","last_active":"2024-05-08T05:29:33.826525+0000","last_peered":"2024-05-08T05:29:33.826525+0000","last_clean":"2024-05-08T05:29:33.826525+0000","last_became_active":"2024-05-08T05:25:25.675136+0000","last_became_peered":"2024-05-08T05:25:25.675136+0000","last_unstale":"2024-05-08T05:29:33.826525+0000","last_undegraded":"2024-05-08T05:29:33.826525+0000","last_fullsized":"2024-05-08T05:29:33.826525+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:44:49.042956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"90'16","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936724+0000","last_change":"2024-05-08T05:25:25.669384+0000","last_active":"2024-05-08T05:29:33.936724+0000","last_peered":"2024-05-08T05:29:33.936724+0000","last_clean":"2024-05-08T05:29:33.936724+0000","last_became_active":"2024-05-08T05:25:25.668861+0000","last_became_peered":"2024-05-08T05:25:25.668861+0000","last_unstale":"2024-05-08T05:29:33.936724+0000","last_undegraded":"2024-05-08T05:29:33.936724+0000","last_fullsized":"2024-05-08T05:29:33.936724+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:48:57.491242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085723+0000","last_change":"2024-05-08T05:25:25.669463+0000","last_active":"2024-05-08T05:29:34.085723+0000","last_peered":"2024-05-08T05:29:34.085723+0000","last_clean":"2024-05-08T05:29:34.085723+0000","last_became_active":"2024-05-08T05:25:25.668672+0000","last_became_peered":"2024-05-08T05:25:25.668672+0000","last_unstale":"2024-05-08T05:29:34.085723+0000","last_undegraded":"2024-05-08T05:29:34.085723+0000","last_fullsized":"2024-05-08T05:29:34.085723+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:30.779473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093195+0000","last_change":"2024-05-08T05:25:25.670378+0000","last_active":"2024-05-08T05:29:34.093195+0000","last_peered":"2024-05-08T05:29:34.093195+0000","last_clean":"2024-05-08T05:29:34.093195+0000","last_became_active":"2024-05-08T05:25:25.669801+0000","last_became_peered":"2024-05-08T05:25:25.669801+0000","last_unstale":"2024-05-08T05:29:34.093195+0000","last_undegraded":"2024-05-08T05:29:34.093195+0000","last_fullsized":"2024-05-08T05:29:34.093195+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:28:49.614163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680129+0000","last_change":"2024-05-08T05:25:25.670874+0000","last_active":"2024-05-08T05:29:34.680129+0000","last_peered":"2024-05-08T05:29:34.680129+0000","last_clean":"2024-05-08T05:29:34.680129+0000","last_became_active":"2024-05-08T05:25:25.670557+0000","last_became_peered":"2024-05-08T05:25:25.670557+0000","last_unstale":"2024-05-08T05:29:34.680129+0000","last_undegraded":"2024-05-08T05:29:34.680129+0000","last_fullsized":"2024-05-08T05:29:34.680129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:33:58.594223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088286+0000","last_change":"2024-05-08T05:25:25.670015+0000","last_active":"2024-05-08T05:29:34.088286+0000","last_peered":"2024-05-08T05:29:34.088286+0000","last_clean":"2024-05-08T05:29:34.088286+0000","last_became_active":"2024-05-08T05:25:25.669493+0000","last_became_peered":"2024-05-08T05:25:25.669493+0000","last_unstale":"2024-05-08T05:29:34.088286+0000","last_undegraded":"2024-05-08T05:29:34.088286+0000","last_fullsized":"2024-05-08T05:29:34.088286+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:49.247104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088404+0000","last_change":"2024-05-08T05:25:25.659651+0000","last_active":"2024-05-08T05:29:34.088404+0000","last_peered":"2024-05-08T05:29:34.088404+0000","last_clean":"2024-05-08T05:29:34.088404+0000","last_became_active":"2024-05-08T05:25:25.659308+0000","last_became_peered":"2024-05-08T05:25:25.659308+0000","last_unstale":"2024-05-08T05:29:34.088404+0000","last_undegraded":"2024-05-08T05:29:34.088404+0000","last_fullsized":"2024-05-08T05:29:34.088404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:12:49.945474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825750+0000","last_change":"2024-05-08T05:25:25.663696+0000","last_active":"2024-05-08T05:29:33.825750+0000","last_peered":"2024-05-08T05:29:33.825750+0000","last_clean":"2024-05-08T05:29:33.825750+0000","last_became_active":"2024-05-08T05:25:25.662590+0000","last_became_peered":"2024-05-08T05:25:25.662590+0000","last_unstale":"2024-05-08T05:29:33.825750+0000","last_undegraded":"2024-05-08T05:29:33.825750+0000","last_fullsized":"2024-05-08T05:29:33.825750+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:29:21.576589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"90'8","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088209+0000","last_change":"2024-05-08T05:25:25.668552+0000","last_active":"2024-05-08T05:29:34.088209+0000","last_peered":"2024-05-08T05:29:34.088209+0000","last_clean":"2024-05-08T05:29:34.088209+0000","last_became_active":"2024-05-08T05:25:25.668185+0000","last_became_peered":"2024-05-08T05:25:25.668185+0000","last_unstale":"2024-05-08T05:29:34.088209+0000","last_undegraded":"2024-05-08T05:29:34.088209+0000","last_fullsized":"2024-05-08T05:29:34.088209+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:47:25.762909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"90'10","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825129+0000","last_change":"2024-05-08T05:25:25.668259+0000","last_active":"2024-05-08T05:29:33.825129+0000","last_peered":"2024-05-08T05:29:33.825129+0000","last_clean":"2024-05-08T05:29:33.825129+0000","last_became_active":"2024-05-08T05:25:25.667767+0000","last_became_peered":"2024-05-08T05:25:25.667767+0000","last_unstale":"2024-05-08T05:29:33.825129+0000","last_undegraded":"2024-05-08T05:29:33.825129+0000","last_fullsized":"2024-05-08T05:29:33.825129+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:36:34.984139+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"90'7","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827417+0000","last_change":"2024-05-08T05:25:25.663381+0000","last_active":"2024-05-08T05:29:33.827417+0000","last_peered":"2024-05-08T05:29:33.827417+0000","last_clean":"2024-05-08T05:29:33.827417+0000","last_became_active":"2024-05-08T05:25:25.663019+0000","last_became_peered":"2024-05-08T05:25:25.663019+0000","last_unstale":"2024-05-08T05:29:33.827417+0000","last_undegraded":"2024-05-08T05:29:33.827417+0000","last_fullsized":"2024-05-08T05:29:33.827417+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:20:58.440446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826825+0000","last_change":"2024-05-08T05:25:37.052656+0000","last_active":"2024-05-08T05:29:33.826825+0000","last_peered":"2024-05-08T05:29:33.826825+0000","last_clean":"2024-05-08T05:29:33.826825+0000","last_became_active":"2024-05-08T05:25:37.052237+0000","last_became_peered":"2024-05-08T05:25:37.052237+0000","last_unstale":"2024-05-08T05:29:33.826825+0000","last_undegraded":"2024-05-08T05:29:33.826825+0000","last_fullsized":"2024-05-08T05:29:33.826825+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:28:54.354488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"91'15","reported_seq":260,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:43.690069+0000","last_change":"2024-05-08T05:25:22.649492+0000","last_active":"2024-05-08T05:29:43.690069+0000","last_peered":"2024-05-08T05:29:43.690069+0000","last_clean":"2024-05-08T05:29:43.690069+0000","last_became_active":"2024-05-08T05:25:22.649132+0000","last_became_peered":"2024-05-08T05:25:22.649132+0000","last_unstale":"2024-05-08T05:29:43.690069+0000","last_undegraded":"2024-05-08T05:29:43.690069+0000","last_fullsized":"2024-05-08T05:29:43.690069+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:45:24.474848+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926798+0000","last_change":"2024-05-08T05:25:25.667811+0000","last_active":"2024-05-08T05:29:33.926798+0000","last_peered":"2024-05-08T05:29:33.926798+0000","last_clean":"2024-05-08T05:29:33.926798+0000","last_became_active":"2024-05-08T05:25:25.667449+0000","last_became_peered":"2024-05-08T05:25:25.667449+0000","last_unstale":"2024-05-08T05:29:33.926798+0000","last_undegraded":"2024-05-08T05:29:33.926798+0000","last_fullsized":"2024-05-08T05:29:33.926798+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:29.293702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092852+0000","last_change":"2024-05-08T05:26:05.308251+0000","last_active":"2024-05-08T05:29:34.092852+0000","last_peered":"2024-05-08T05:29:34.092852+0000","last_clean":"2024-05-08T05:29:34.092852+0000","last_became_active":"2024-05-08T05:26:05.307622+0000","last_became_peered":"2024-05-08T05:26:05.307622+0000","last_unstale":"2024-05-08T05:29:34.092852+0000","last_undegraded":"2024-05-08T05:29:34.092852+0000","last_fullsized":"2024-05-08T05:29:34.092852+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:36:51.116094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"88'2","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926878+0000","last_change":"2024-05-08T05:25:22.748130+0000","last_active":"2024-05-08T05:29:33.926878+0000","last_peered":"2024-05-08T05:29:33.926878+0000","last_clean":"2024-05-08T05:29:33.926878+0000","last_became_active":"2024-05-08T05:25:22.747769+0000","last_became_peered":"2024-05-08T05:25:22.747769+0000","last_unstale":"2024-05-08T05:29:33.926878+0000","last_undegraded":"2024-05-08T05:29:33.926878+0000","last_fullsized":"2024-05-08T05:29:33.926878+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:20:04.086851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088665+0000","last_change":"2024-05-08T05:25:25.672114+0000","last_active":"2024-05-08T05:29:34.088665+0000","last_peered":"2024-05-08T05:29:34.088665+0000","last_clean":"2024-05-08T05:29:34.088665+0000","last_became_active":"2024-05-08T05:25:25.671761+0000","last_became_peered":"2024-05-08T05:25:25.671761+0000","last_unstale":"2024-05-08T05:29:34.088665+0000","last_undegraded":"2024-05-08T05:29:34.088665+0000","last_fullsized":"2024-05-08T05:29:34.088665+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:40.684565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092958+0000","last_change":"2024-05-08T05:25:37.058375+0000","last_active":"2024-05-08T05:29:34.092958+0000","last_peered":"2024-05-08T05:29:34.092958+0000","last_clean":"2024-05-08T05:29:34.092958+0000","last_became_active":"2024-05-08T05:25:37.057844+0000","last_became_peered":"2024-05-08T05:25:37.057844+0000","last_unstale":"2024-05-08T05:29:34.092958+0000","last_undegraded":"2024-05-08T05:29:34.092958+0000","last_fullsized":"2024-05-08T05:29:34.092958+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:51:42.881830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"88'12","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088188+0000","last_change":"2024-05-08T05:27:05.576781+0000","last_active":"2024-05-08T05:29:34.088188+0000","last_peered":"2024-05-08T05:29:34.088188+0000","last_clean":"2024-05-08T05:29:34.088188+0000","last_became_active":"2024-05-08T05:27:05.566420+0000","last_became_peered":"2024-05-08T05:27:05.566420+0000","last_unstale":"2024-05-08T05:29:34.088188+0000","last_undegraded":"2024-05-08T05:29:34.088188+0000","last_fullsized":"2024-05-08T05:29:34.088188+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:04:09.385135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":7,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,7],"acting":[11,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1c","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826181+0000","last_change":"2024-05-08T05:25:25.673300+0000","last_active":"2024-05-08T05:29:33.826181+0000","last_peered":"2024-05-08T05:29:33.826181+0000","last_clean":"2024-05-08T05:29:33.826181+0000","last_became_active":"2024-05-08T05:25:25.672881+0000","last_became_peered":"2024-05-08T05:25:25.672881+0000","last_unstale":"2024-05-08T05:29:33.826181+0000","last_undegraded":"2024-05-08T05:29:33.826181+0000","last_fullsized":"2024-05-08T05:29:33.826181+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:16.111775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092846+0000","last_change":"2024-05-08T05:25:37.058903+0000","last_active":"2024-05-08T05:29:34.092846+0000","last_peered":"2024-05-08T05:29:34.092846+0000","last_clean":"2024-05-08T05:29:34.092846+0000","last_became_active":"2024-05-08T05:25:37.058508+0000","last_became_peered":"2024-05-08T05:25:37.058508+0000","last_unstale":"2024-05-08T05:29:34.092846+0000","last_undegraded":"2024-05-08T05:29:34.092846+0000","last_fullsized":"2024-05-08T05:29:34.092846+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:15:16.605748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925877+0000","last_change":"2024-05-08T05:25:22.646191+0000","last_active":"2024-05-08T05:29:33.925877+0000","last_peered":"2024-05-08T05:29:33.925877+0000","last_clean":"2024-05-08T05:29:33.925877+0000","last_became_active":"2024-05-08T05:25:22.645187+0000","last_became_peered":"2024-05-08T05:25:22.645187+0000","last_unstale":"2024-05-08T05:29:33.925877+0000","last_undegraded":"2024-05-08T05:29:33.925877+0000","last_fullsized":"2024-05-08T05:29:33.925877+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:27:13.358658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"90'15","reported_seq":62,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933506+0000","last_change":"2024-05-08T05:26:05.308149+0000","last_active":"2024-05-08T05:29:33.933506+0000","last_peered":"2024-05-08T05:29:33.933506+0000","last_clean":"2024-05-08T05:29:33.933506+0000","last_became_active":"2024-05-08T05:26:05.307602+0000","last_became_peered":"2024-05-08T05:26:05.307602+0000","last_unstale":"2024-05-08T05:29:33.933506+0000","last_undegraded":"2024-05-08T05:29:33.933506+0000","last_fullsized":"2024-05-08T05:29:33.933506+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:27:02.791894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,1],"acting":[5,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827181+0000","last_change":"2024-05-08T05:25:37.059052+0000","last_active":"2024-05-08T05:29:33.827181+0000","last_peered":"2024-05-08T05:29:33.827181+0000","last_clean":"2024-05-08T05:29:33.827181+0000","last_became_active":"2024-05-08T05:25:37.058545+0000","last_became_peered":"2024-05-08T05:25:37.058545+0000","last_unstale":"2024-05-08T05:29:33.827181+0000","last_undegraded":"2024-05-08T05:29:33.827181+0000","last_fullsized":"2024-05-08T05:29:33.827181+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:31:31.521284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'9","reported_seq":258,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:43.691097+0000","last_change":"2024-05-08T05:25:22.648874+0000","last_active":"2024-05-08T05:29:43.691097+0000","last_peered":"2024-05-08T05:29:43.691097+0000","last_clean":"2024-05-08T05:29:43.691097+0000","last_became_active":"2024-05-08T05:25:22.648477+0000","last_became_peered":"2024-05-08T05:25:22.648477+0000","last_unstale":"2024-05-08T05:29:43.691097+0000","last_undegraded":"2024-05-08T05:29:43.691097+0000","last_fullsized":"2024-05-08T05:29:43.691097+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:19:08.075904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1936,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":202,"num_read_kb":5,"num_write":6,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925572+0000","last_change":"2024-05-08T05:25:25.662449+0000","last_active":"2024-05-08T05:29:33.925572+0000","last_peered":"2024-05-08T05:29:33.925572+0000","last_clean":"2024-05-08T05:29:33.925572+0000","last_became_active":"2024-05-08T05:25:25.662089+0000","last_became_peered":"2024-05-08T05:25:25.662089+0000","last_unstale":"2024-05-08T05:29:33.925572+0000","last_undegraded":"2024-05-08T05:29:33.925572+0000","last_fullsized":"2024-05-08T05:29:33.925572+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:31.360921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827128+0000","last_change":"2024-05-08T05:25:37.055548+0000","last_active":"2024-05-08T05:29:33.827128+0000","last_peered":"2024-05-08T05:29:33.827128+0000","last_clean":"2024-05-08T05:29:33.827128+0000","last_became_active":"2024-05-08T05:25:37.055138+0000","last_became_peered":"2024-05-08T05:25:37.055138+0000","last_unstale":"2024-05-08T05:29:33.827128+0000","last_undegraded":"2024-05-08T05:29:33.827128+0000","last_fullsized":"2024-05-08T05:29:33.827128+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:37:45.697831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"88'7","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936365+0000","last_change":"2024-05-08T05:25:22.648072+0000","last_active":"2024-05-08T05:29:33.936365+0000","last_peered":"2024-05-08T05:29:33.936365+0000","last_clean":"2024-05-08T05:29:33.936365+0000","last_became_active":"2024-05-08T05:25:22.647457+0000","last_became_peered":"2024-05-08T05:25:22.647457+0000","last_unstale":"2024-05-08T05:29:33.936365+0000","last_undegraded":"2024-05-08T05:29:33.936365+0000","last_fullsized":"2024-05-08T05:29:33.936365+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:52:17.476479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933534+0000","last_change":"2024-05-08T05:25:25.662698+0000","last_active":"2024-05-08T05:29:33.933534+0000","last_peered":"2024-05-08T05:29:33.933534+0000","last_clean":"2024-05-08T05:29:33.933534+0000","last_became_active":"2024-05-08T05:25:25.662368+0000","last_became_peered":"2024-05-08T05:25:25.662368+0000","last_unstale":"2024-05-08T05:29:33.933534+0000","last_undegraded":"2024-05-08T05:29:33.933534+0000","last_fullsized":"2024-05-08T05:29:33.933534+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:13:38.511037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088201+0000","last_change":"2024-05-08T05:25:37.054407+0000","last_active":"2024-05-08T05:29:34.088201+0000","last_peered":"2024-05-08T05:29:34.088201+0000","last_clean":"2024-05-08T05:29:34.088201+0000","last_became_active":"2024-05-08T05:25:37.053999+0000","last_became_peered":"2024-05-08T05:25:37.053999+0000","last_unstale":"2024-05-08T05:29:34.088201+0000","last_undegraded":"2024-05-08T05:29:34.088201+0000","last_fullsized":"2024-05-08T05:29:34.088201+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:25:16.942601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"88'3","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680002+0000","last_change":"2024-05-08T05:25:22.643654+0000","last_active":"2024-05-08T05:29:34.680002+0000","last_peered":"2024-05-08T05:29:34.680002+0000","last_clean":"2024-05-08T05:29:34.680002+0000","last_became_active":"2024-05-08T05:25:22.643315+0000","last_became_peered":"2024-05-08T05:25:22.643315+0000","last_unstale":"2024-05-08T05:29:34.680002+0000","last_undegraded":"2024-05-08T05:29:34.680002+0000","last_fullsized":"2024-05-08T05:29:34.680002+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:37:53.514629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925576+0000","last_change":"2024-05-08T05:25:25.663148+0000","last_active":"2024-05-08T05:29:33.925576+0000","last_peered":"2024-05-08T05:29:33.925576+0000","last_clean":"2024-05-08T05:29:33.925576+0000","last_became_active":"2024-05-08T05:25:25.662770+0000","last_became_peered":"2024-05-08T05:25:25.662770+0000","last_unstale":"2024-05-08T05:29:33.925576+0000","last_undegraded":"2024-05-08T05:29:33.925576+0000","last_fullsized":"2024-05-08T05:29:33.925576+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:15:46.458233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088500+0000","last_change":"2024-05-08T05:25:37.054265+0000","last_active":"2024-05-08T05:29:34.088500+0000","last_peered":"2024-05-08T05:29:34.088500+0000","last_clean":"2024-05-08T05:29:34.088500+0000","last_became_active":"2024-05-08T05:25:37.053579+0000","last_became_peered":"2024-05-08T05:25:37.053579+0000","last_unstale":"2024-05-08T05:29:34.088500+0000","last_undegraded":"2024-05-08T05:29:34.088500+0000","last_fullsized":"2024-05-08T05:29:34.088500+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:22:21.748715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"88'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933115+0000","last_change":"2024-05-08T05:25:22.648083+0000","last_active":"2024-05-08T05:29:33.933115+0000","last_peered":"2024-05-08T05:29:33.933115+0000","last_clean":"2024-05-08T05:29:33.933115+0000","last_became_active":"2024-05-08T05:25:22.647650+0000","last_became_peered":"2024-05-08T05:25:22.647650+0000","last_unstale":"2024-05-08T05:29:33.933115+0000","last_undegraded":"2024-05-08T05:29:33.933115+0000","last_fullsized":"2024-05-08T05:29:33.933115+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:27:56.784190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"90'11","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.824690+0000","last_change":"2024-05-08T05:25:25.668290+0000","last_active":"2024-05-08T05:29:33.824690+0000","last_peered":"2024-05-08T05:29:33.824690+0000","last_clean":"2024-05-08T05:29:33.824690+0000","last_became_active":"2024-05-08T05:25:25.667888+0000","last_became_peered":"2024-05-08T05:25:25.667888+0000","last_unstale":"2024-05-08T05:29:33.824690+0000","last_undegraded":"2024-05-08T05:29:33.824690+0000","last_fullsized":"2024-05-08T05:29:33.824690+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:55:19.983471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925789+0000","last_change":"2024-05-08T05:25:37.057639+0000","last_active":"2024-05-08T05:29:33.925789+0000","last_peered":"2024-05-08T05:29:33.925789+0000","last_clean":"2024-05-08T05:29:33.925789+0000","last_became_active":"2024-05-08T05:25:37.056848+0000","last_became_peered":"2024-05-08T05:25:37.056848+0000","last_unstale":"2024-05-08T05:29:33.925789+0000","last_undegraded":"2024-05-08T05:29:33.925789+0000","last_fullsized":"2024-05-08T05:29:33.925789+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:18:37.137224+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'8","reported_seq":448,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:44.691871+0000","last_change":"2024-05-08T05:25:22.649658+0000","last_active":"2024-05-08T05:29:44.691871+0000","last_peered":"2024-05-08T05:29:44.691871+0000","last_clean":"2024-05-08T05:29:44.691871+0000","last_became_active":"2024-05-08T05:25:22.648771+0000","last_became_peered":"2024-05-08T05:25:22.648771+0000","last_unstale":"2024-05-08T05:29:44.691871+0000","last_undegraded":"2024-05-08T05:29:44.691871+0000","last_fullsized":"2024-05-08T05:29:44.691871+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:07:54.412710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":394,"num_read_kb":394,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826502+0000","last_change":"2024-05-08T05:25:25.673931+0000","last_active":"2024-05-08T05:29:33.826502+0000","last_peered":"2024-05-08T05:29:33.826502+0000","last_clean":"2024-05-08T05:29:33.826502+0000","last_became_active":"2024-05-08T05:25:25.673585+0000","last_became_peered":"2024-05-08T05:25:25.673585+0000","last_unstale":"2024-05-08T05:29:33.826502+0000","last_undegraded":"2024-05-08T05:29:33.826502+0000","last_fullsized":"2024-05-08T05:29:33.826502+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:58:27.569247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.824556+0000","last_change":"2024-05-08T05:25:37.047651+0000","last_active":"2024-05-08T05:29:33.824556+0000","last_peered":"2024-05-08T05:29:33.824556+0000","last_clean":"2024-05-08T05:29:33.824556+0000","last_became_active":"2024-05-08T05:25:37.047246+0000","last_became_peered":"2024-05-08T05:25:37.047246+0000","last_unstale":"2024-05-08T05:29:33.824556+0000","last_undegraded":"2024-05-08T05:29:33.824556+0000","last_fullsized":"2024-05-08T05:29:33.824556+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:57:22.300076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"88'4","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085560+0000","last_change":"2024-05-08T05:25:22.646973+0000","last_active":"2024-05-08T05:29:34.085560+0000","last_peered":"2024-05-08T05:29:34.085560+0000","last_clean":"2024-05-08T05:29:34.085560+0000","last_became_active":"2024-05-08T05:25:22.646232+0000","last_became_peered":"2024-05-08T05:25:22.646232+0000","last_unstale":"2024-05-08T05:29:34.085560+0000","last_undegraded":"2024-05-08T05:29:34.085560+0000","last_fullsized":"2024-05-08T05:29:34.085560+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:14:38.147470+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088524+0000","last_change":"2024-05-08T05:25:25.661373+0000","last_active":"2024-05-08T05:29:34.088524+0000","last_peered":"2024-05-08T05:29:34.088524+0000","last_clean":"2024-05-08T05:29:34.088524+0000","last_became_active":"2024-05-08T05:25:25.661080+0000","last_became_peered":"2024-05-08T05:25:25.661080+0000","last_unstale":"2024-05-08T05:29:34.088524+0000","last_undegraded":"2024-05-08T05:29:34.088524+0000","last_fullsized":"2024-05-08T05:29:34.088524+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:10:18.718610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093071+0000","last_change":"2024-05-08T05:25:37.051797+0000","last_active":"2024-05-08T05:29:34.093071+0000","last_peered":"2024-05-08T05:29:34.093071+0000","last_clean":"2024-05-08T05:29:34.093071+0000","last_became_active":"2024-05-08T05:25:37.051381+0000","last_became_peered":"2024-05-08T05:25:37.051381+0000","last_unstale":"2024-05-08T05:29:34.093071+0000","last_undegraded":"2024-05-08T05:29:34.093071+0000","last_fullsized":"2024-05-08T05:29:34.093071+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:40:34.337512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933456+0000","last_change":"2024-05-08T05:25:22.748510+0000","last_active":"2024-05-08T05:29:33.933456+0000","last_peered":"2024-05-08T05:29:33.933456+0000","last_clean":"2024-05-08T05:29:33.933456+0000","last_became_active":"2024-05-08T05:25:22.748140+0000","last_became_peered":"2024-05-08T05:25:22.748140+0000","last_unstale":"2024-05-08T05:29:33.933456+0000","last_undegraded":"2024-05-08T05:29:33.933456+0000","last_fullsized":"2024-05-08T05:29:33.933456+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:21:32.325653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826339+0000","last_change":"2024-05-08T05:25:25.671261+0000","last_active":"2024-05-08T05:29:33.826339+0000","last_peered":"2024-05-08T05:29:33.826339+0000","last_clean":"2024-05-08T05:29:33.826339+0000","last_became_active":"2024-05-08T05:25:25.670937+0000","last_became_peered":"2024-05-08T05:25:25.670937+0000","last_unstale":"2024-05-08T05:29:33.826339+0000","last_undegraded":"2024-05-08T05:29:33.826339+0000","last_fullsized":"2024-05-08T05:29:33.826339+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:41.375908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936237+0000","last_change":"2024-05-08T05:26:05.150754+0000","last_active":"2024-05-08T05:29:33.936237+0000","last_peered":"2024-05-08T05:29:33.936237+0000","last_clean":"2024-05-08T05:29:33.936237+0000","last_became_active":"2024-05-08T05:26:05.150331+0000","last_became_peered":"2024-05-08T05:26:05.150331+0000","last_unstale":"2024-05-08T05:29:33.936237+0000","last_undegraded":"2024-05-08T05:29:33.936237+0000","last_fullsized":"2024-05-08T05:29:33.936237+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:30:23.309103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,11,5],"acting":[10,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.14","version":"91'44","reported_seq":499,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:44.722233+0000","last_change":"2024-05-08T05:25:22.645627+0000","last_active":"2024-05-08T05:29:44.722233+0000","last_peered":"2024-05-08T05:29:44.722233+0000","last_clean":"2024-05-08T05:29:44.722233+0000","last_became_active":"2024-05-08T05:25:22.645243+0000","last_became_peered":"2024-05-08T05:25:22.645243+0000","last_unstale":"2024-05-08T05:29:44.722233+0000","last_undegraded":"2024-05-08T05:29:44.722233+0000","last_fullsized":"2024-05-08T05:29:44.722233+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:56:23.256903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":72818,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":409,"num_read_kb":277,"num_write":46,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925853+0000","last_change":"2024-05-08T05:25:25.661258+0000","last_active":"2024-05-08T05:29:33.925853+0000","last_peered":"2024-05-08T05:29:33.925853+0000","last_clean":"2024-05-08T05:29:33.925853+0000","last_became_active":"2024-05-08T05:25:25.660925+0000","last_became_peered":"2024-05-08T05:25:25.660925+0000","last_unstale":"2024-05-08T05:29:33.925853+0000","last_undegraded":"2024-05-08T05:29:33.925853+0000","last_fullsized":"2024-05-08T05:29:33.925853+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:41:58.764540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933810+0000","last_change":"2024-05-08T05:25:37.053539+0000","last_active":"2024-05-08T05:29:33.933810+0000","last_peered":"2024-05-08T05:29:33.933810+0000","last_clean":"2024-05-08T05:29:33.933810+0000","last_became_active":"2024-05-08T05:25:37.053102+0000","last_became_peered":"2024-05-08T05:25:37.053102+0000","last_unstale":"2024-05-08T05:29:33.933810+0000","last_undegraded":"2024-05-08T05:29:33.933810+0000","last_fullsized":"2024-05-08T05:29:33.933810+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:48.182992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"88'9","reported_seq":57,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827052+0000","last_change":"2024-05-08T05:25:22.646651+0000","last_active":"2024-05-08T05:29:33.827052+0000","last_peered":"2024-05-08T05:29:33.827052+0000","last_clean":"2024-05-08T05:29:33.827052+0000","last_became_active":"2024-05-08T05:25:22.646285+0000","last_became_peered":"2024-05-08T05:25:22.646285+0000","last_unstale":"2024-05-08T05:29:33.827052+0000","last_undegraded":"2024-05-08T05:29:33.827052+0000","last_fullsized":"2024-05-08T05:29:33.827052+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:42:08.843858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680138+0000","last_change":"2024-05-08T05:25:25.666018+0000","last_active":"2024-05-08T05:29:34.680138+0000","last_peered":"2024-05-08T05:29:34.680138+0000","last_clean":"2024-05-08T05:29:34.680138+0000","last_became_active":"2024-05-08T05:25:25.663609+0000","last_became_peered":"2024-05-08T05:25:25.663609+0000","last_unstale":"2024-05-08T05:29:34.680138+0000","last_undegraded":"2024-05-08T05:29:34.680138+0000","last_fullsized":"2024-05-08T05:29:34.680138+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:17:36.223121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087978+0000","last_change":"2024-05-08T05:25:37.052907+0000","last_active":"2024-05-08T05:29:34.087978+0000","last_peered":"2024-05-08T05:29:34.087978+0000","last_clean":"2024-05-08T05:29:34.087978+0000","last_became_active":"2024-05-08T05:25:37.052506+0000","last_became_peered":"2024-05-08T05:25:37.052506+0000","last_unstale":"2024-05-08T05:29:34.087978+0000","last_undegraded":"2024-05-08T05:29:34.087978+0000","last_fullsized":"2024-05-08T05:29:34.087978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:41:41.356996+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"88'5","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933355+0000","last_change":"2024-05-08T05:25:22.646671+0000","last_active":"2024-05-08T05:29:33.933355+0000","last_peered":"2024-05-08T05:29:33.933355+0000","last_clean":"2024-05-08T05:29:33.933355+0000","last_became_active":"2024-05-08T05:25:22.646202+0000","last_became_peered":"2024-05-08T05:25:22.646202+0000","last_unstale":"2024-05-08T05:29:33.933355+0000","last_undegraded":"2024-05-08T05:29:33.933355+0000","last_fullsized":"2024-05-08T05:29:33.933355+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:37:59.224960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085357+0000","last_change":"2024-05-08T05:25:25.666969+0000","last_active":"2024-05-08T05:29:34.085357+0000","last_peered":"2024-05-08T05:29:34.085357+0000","last_clean":"2024-05-08T05:29:34.085357+0000","last_became_active":"2024-05-08T05:25:25.666482+0000","last_became_peered":"2024-05-08T05:25:25.666482+0000","last_unstale":"2024-05-08T05:29:34.085357+0000","last_undegraded":"2024-05-08T05:29:34.085357+0000","last_fullsized":"2024-05-08T05:29:34.085357+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:55:07.819116+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825127+0000","last_change":"2024-05-08T05:25:37.055807+0000","last_active":"2024-05-08T05:29:33.825127+0000","last_peered":"2024-05-08T05:29:33.825127+0000","last_clean":"2024-05-08T05:29:33.825127+0000","last_became_active":"2024-05-08T05:25:37.054778+0000","last_became_peered":"2024-05-08T05:25:37.054778+0000","last_unstale":"2024-05-08T05:29:33.825127+0000","last_undegraded":"2024-05-08T05:29:33.825127+0000","last_fullsized":"2024-05-08T05:29:33.825127+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:51:33.767885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'9","reported_seq":258,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:43.691164+0000","last_change":"2024-05-08T05:25:22.647093+0000","last_active":"2024-05-08T05:29:43.691164+0000","last_peered":"2024-05-08T05:29:43.691164+0000","last_clean":"2024-05-08T05:29:43.691164+0000","last_became_active":"2024-05-08T05:25:22.646733+0000","last_became_peered":"2024-05-08T05:25:22.646733+0000","last_unstale":"2024-05-08T05:29:43.691164+0000","last_undegraded":"2024-05-08T05:29:43.691164+0000","last_fullsized":"2024-05-08T05:29:43.691164+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:04.346568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1819,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":204,"num_read_kb":7,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085169+0000","last_change":"2024-05-08T05:25:25.666565+0000","last_active":"2024-05-08T05:29:34.085169+0000","last_peered":"2024-05-08T05:29:34.085169+0000","last_clean":"2024-05-08T05:29:34.085169+0000","last_became_active":"2024-05-08T05:25:25.666138+0000","last_became_peered":"2024-05-08T05:25:25.666138+0000","last_unstale":"2024-05-08T05:29:34.085169+0000","last_undegraded":"2024-05-08T05:29:34.085169+0000","last_fullsized":"2024-05-08T05:29:34.085169+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:40:05.396530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680587+0000","last_change":"2024-05-08T05:25:37.060933+0000","last_active":"2024-05-08T05:29:34.680587+0000","last_peered":"2024-05-08T05:29:34.680587+0000","last_clean":"2024-05-08T05:29:34.680587+0000","last_became_active":"2024-05-08T05:25:37.060388+0000","last_became_peered":"2024-05-08T05:25:37.060388+0000","last_unstale":"2024-05-08T05:29:34.680587+0000","last_undegraded":"2024-05-08T05:29:34.680587+0000","last_fullsized":"2024-05-08T05:29:34.680587+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:23:09.776051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"88'17","reported_seq":81,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925413+0000","last_change":"2024-05-08T05:26:05.156863+0000","last_active":"2024-05-08T05:29:33.925413+0000","last_peered":"2024-05-08T05:29:33.925413+0000","last_clean":"2024-05-08T05:29:33.925413+0000","last_became_active":"2024-05-08T05:26:05.149008+0000","last_became_peered":"2024-05-08T05:26:05.149008+0000","last_unstale":"2024-05-08T05:29:33.925413+0000","last_undegraded":"2024-05-08T05:29:33.925413+0000","last_fullsized":"2024-05-08T05:29:33.925413+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:57:56.969478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,4],"acting":[8,0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933168+0000","last_change":"2024-05-08T05:25:25.670415+0000","last_active":"2024-05-08T05:29:33.933168+0000","last_peered":"2024-05-08T05:29:33.933168+0000","last_clean":"2024-05-08T05:29:33.933168+0000","last_became_active":"2024-05-08T05:25:25.670095+0000","last_became_peered":"2024-05-08T05:25:25.670095+0000","last_unstale":"2024-05-08T05:29:33.933168+0000","last_undegraded":"2024-05-08T05:29:33.933168+0000","last_fullsized":"2024-05-08T05:29:33.933168+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:53:29.489203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925444+0000","last_change":"2024-05-08T05:26:05.307721+0000","last_active":"2024-05-08T05:29:33.925444+0000","last_peered":"2024-05-08T05:29:33.925444+0000","last_clean":"2024-05-08T05:29:33.925444+0000","last_became_active":"2024-05-08T05:26:05.307279+0000","last_became_peered":"2024-05-08T05:26:05.307279+0000","last_unstale":"2024-05-08T05:29:33.925444+0000","last_undegraded":"2024-05-08T05:29:33.925444+0000","last_fullsized":"2024-05-08T05:29:33.925444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:47:08.857770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,3,1],"acting":[8,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.f","version":"90'3","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925842+0000","last_change":"2024-05-08T05:25:22.644279+0000","last_active":"2024-05-08T05:29:33.925842+0000","last_peered":"2024-05-08T05:29:33.925842+0000","last_clean":"2024-05-08T05:29:33.925842+0000","last_became_active":"2024-05-08T05:25:22.643900+0000","last_became_peered":"2024-05-08T05:25:22.643900+0000","last_unstale":"2024-05-08T05:29:33.925842+0000","last_undegraded":"2024-05-08T05:29:33.925842+0000","last_fullsized":"2024-05-08T05:29:33.925842+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:39:32.435843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"90'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088135+0000","last_change":"2024-05-08T05:25:25.670130+0000","last_active":"2024-05-08T05:29:34.088135+0000","last_peered":"2024-05-08T05:29:34.088135+0000","last_clean":"2024-05-08T05:29:34.088135+0000","last_became_active":"2024-05-08T05:25:25.669784+0000","last_became_peered":"2024-05-08T05:25:25.669784+0000","last_unstale":"2024-05-08T05:29:34.088135+0000","last_undegraded":"2024-05-08T05:29:34.088135+0000","last_fullsized":"2024-05-08T05:29:34.088135+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:45:07.188883+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925569+0000","last_change":"2024-05-08T05:26:05.308804+0000","last_active":"2024-05-08T05:29:33.925569+0000","last_peered":"2024-05-08T05:29:33.925569+0000","last_clean":"2024-05-08T05:29:33.925569+0000","last_became_active":"2024-05-08T05:26:05.308226+0000","last_became_peered":"2024-05-08T05:26:05.308226+0000","last_unstale":"2024-05-08T05:29:33.925569+0000","last_undegraded":"2024-05-08T05:29:33.925569+0000","last_fullsized":"2024-05-08T05:29:33.925569+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:33.442023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,1],"acting":[8,10,11,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"88'1","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.824979+0000","last_change":"2024-05-08T05:25:22.748409+0000","last_active":"2024-05-08T05:29:33.824979+0000","last_peered":"2024-05-08T05:29:33.824979+0000","last_clean":"2024-05-08T05:29:33.824979+0000","last_became_active":"2024-05-08T05:25:22.748062+0000","last_became_peered":"2024-05-08T05:25:22.748062+0000","last_unstale":"2024-05-08T05:29:33.824979+0000","last_undegraded":"2024-05-08T05:29:33.824979+0000","last_fullsized":"2024-05-08T05:29:33.824979+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:07:42.147084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"90'4","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936551+0000","last_change":"2024-05-08T05:25:25.677057+0000","last_active":"2024-05-08T05:29:33.936551+0000","last_peered":"2024-05-08T05:29:33.936551+0000","last_clean":"2024-05-08T05:29:33.936551+0000","last_became_active":"2024-05-08T05:25:25.676697+0000","last_became_peered":"2024-05-08T05:25:25.676697+0000","last_unstale":"2024-05-08T05:29:33.936551+0000","last_undegraded":"2024-05-08T05:29:33.936551+0000","last_fullsized":"2024-05-08T05:29:33.936551+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:30:40.786327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680220+0000","last_change":"2024-05-08T05:25:37.053903+0000","last_active":"2024-05-08T05:29:34.680220+0000","last_peered":"2024-05-08T05:29:34.680220+0000","last_clean":"2024-05-08T05:29:34.680220+0000","last_became_active":"2024-05-08T05:25:37.053375+0000","last_became_peered":"2024-05-08T05:25:37.053375+0000","last_unstale":"2024-05-08T05:29:34.680220+0000","last_undegraded":"2024-05-08T05:29:34.680220+0000","last_fullsized":"2024-05-08T05:29:34.680220+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:20:19.374355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"91'8","reported_seq":257,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:43.690521+0000","last_change":"2024-05-08T05:25:22.646931+0000","last_active":"2024-05-08T05:29:43.690521+0000","last_peered":"2024-05-08T05:29:43.690521+0000","last_clean":"2024-05-08T05:29:43.690521+0000","last_became_active":"2024-05-08T05:25:22.646599+0000","last_became_peered":"2024-05-08T05:25:22.646599+0000","last_unstale":"2024-05-08T05:29:43.690521+0000","last_undegraded":"2024-05-08T05:29:43.690521+0000","last_fullsized":"2024-05-08T05:29:43.690521+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:16:55.873089+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1511,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":201,"num_read_kb":4,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"90'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.936384+0000","last_change":"2024-05-08T05:26:05.147977+0000","last_active":"2024-05-08T05:29:33.936384+0000","last_peered":"2024-05-08T05:29:33.936384+0000","last_clean":"2024-05-08T05:29:33.936384+0000","last_became_active":"2024-05-08T05:26:05.147637+0000","last_became_peered":"2024-05-08T05:26:05.147637+0000","last_unstale":"2024-05-08T05:29:33.936384+0000","last_undegraded":"2024-05-08T05:29:33.936384+0000","last_fullsized":"2024-05-08T05:29:33.936384+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:31:59.671306+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680429+0000","last_change":"2024-05-08T05:25:37.057913+0000","last_active":"2024-05-08T05:29:34.680429+0000","last_peered":"2024-05-08T05:29:34.680429+0000","last_clean":"2024-05-08T05:29:34.680429+0000","last_became_active":"2024-05-08T05:25:37.057474+0000","last_became_peered":"2024-05-08T05:25:37.057474+0000","last_unstale":"2024-05-08T05:29:34.680429+0000","last_undegraded":"2024-05-08T05:29:34.680429+0000","last_fullsized":"2024-05-08T05:29:34.680429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:42:19.381899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"90'8","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933446+0000","last_change":"2024-05-08T05:25:25.663594+0000","last_active":"2024-05-08T05:29:33.933446+0000","last_peered":"2024-05-08T05:29:33.933446+0000","last_clean":"2024-05-08T05:29:33.933446+0000","last_became_active":"2024-05-08T05:25:25.663275+0000","last_became_peered":"2024-05-08T05:25:25.663275+0000","last_unstale":"2024-05-08T05:29:33.933446+0000","last_undegraded":"2024-05-08T05:29:33.933446+0000","last_fullsized":"2024-05-08T05:29:33.933446+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:36:20.581166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"90'7","reported_seq":253,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:43.690690+0000","last_change":"2024-05-08T05:25:22.748316+0000","last_active":"2024-05-08T05:29:43.690690+0000","last_peered":"2024-05-08T05:29:43.690690+0000","last_clean":"2024-05-08T05:29:43.690690+0000","last_became_active":"2024-05-08T05:25:22.747819+0000","last_became_peered":"2024-05-08T05:25:22.747819+0000","last_unstale":"2024-05-08T05:29:43.690690+0000","last_undegraded":"2024-05-08T05:29:43.690690+0000","last_fullsized":"2024-05-08T05:29:43.690690+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:50:42.689853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":198,"num_read_kb":198,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"88'1","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085575+0000","last_change":"2024-05-08T05:25:37.058271+0000","last_active":"2024-05-08T05:29:34.085575+0000","last_peered":"2024-05-08T05:29:34.085575+0000","last_clean":"2024-05-08T05:29:34.085575+0000","last_became_active":"2024-05-08T05:25:37.057290+0000","last_became_peered":"2024-05-08T05:25:37.057290+0000","last_unstale":"2024-05-08T05:29:34.085575+0000","last_undegraded":"2024-05-08T05:29:34.085575+0000","last_fullsized":"2024-05-08T05:29:34.085575+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:31:47.369909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925931+0000","last_change":"2024-05-08T05:25:25.666000+0000","last_active":"2024-05-08T05:29:33.925931+0000","last_peered":"2024-05-08T05:29:33.925931+0000","last_clean":"2024-05-08T05:29:33.925931+0000","last_became_active":"2024-05-08T05:25:25.665663+0000","last_became_peered":"2024-05-08T05:25:25.665663+0000","last_unstale":"2024-05-08T05:29:33.925931+0000","last_undegraded":"2024-05-08T05:29:33.925931+0000","last_fullsized":"2024-05-08T05:29:33.925931+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:50:44.328879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"88'6","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093038+0000","last_change":"2024-05-08T05:25:22.648770+0000","last_active":"2024-05-08T05:29:34.093038+0000","last_peered":"2024-05-08T05:29:34.093038+0000","last_clean":"2024-05-08T05:29:34.093038+0000","last_became_active":"2024-05-08T05:25:22.648119+0000","last_became_peered":"2024-05-08T05:25:22.648119+0000","last_unstale":"2024-05-08T05:29:34.093038+0000","last_undegraded":"2024-05-08T05:29:34.093038+0000","last_fullsized":"2024-05-08T05:29:34.093038+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:55.813080+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"91'99","reported_seq":150,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:35.935788+0000","last_change":"2024-05-08T05:23:55.820436+0000","last_active":"2024-05-08T05:29:35.935788+0000","last_peered":"2024-05-08T05:29:35.935788+0000","last_clean":"2024-05-08T05:29:35.935788+0000","last_became_active":"2024-05-08T05:23:55.814786+0000","last_became_peered":"2024-05-08T05:23:55.814786+0000","last_unstale":"2024-05-08T05:29:35.935788+0000","last_undegraded":"2024-05-08T05:29:35.935788+0000","last_fullsized":"2024-05-08T05:29:35.935788+0000","mapping_epoch":66,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":67,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:20:32.659283+0000","last_clean_scrub_stamp":"2024-05-08T05:20:32.659283+0000","objects_scrubbed":0,"log_size":99,"log_dups_size":0,"ondisk_log_size":99,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:20:11.169455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933610+0000","last_change":"2024-05-08T05:25:37.057287+0000","last_active":"2024-05-08T05:29:33.933610+0000","last_peered":"2024-05-08T05:29:33.933610+0000","last_clean":"2024-05-08T05:29:33.933610+0000","last_became_active":"2024-05-08T05:25:37.056871+0000","last_became_peered":"2024-05-08T05:25:37.056871+0000","last_unstale":"2024-05-08T05:29:33.933610+0000","last_undegraded":"2024-05-08T05:29:33.933610+0000","last_fullsized":"2024-05-08T05:29:33.933610+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:26:19.004199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"90'5","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.679912+0000","last_change":"2024-05-08T05:25:25.664795+0000","last_active":"2024-05-08T05:29:34.679912+0000","last_peered":"2024-05-08T05:29:34.679912+0000","last_clean":"2024-05-08T05:29:34.679912+0000","last_became_active":"2024-05-08T05:25:25.664444+0000","last_became_peered":"2024-05-08T05:25:25.664444+0000","last_unstale":"2024-05-08T05:29:34.679912+0000","last_undegraded":"2024-05-08T05:29:34.679912+0000","last_fullsized":"2024-05-08T05:29:34.679912+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T11:54:27.172530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085169+0000","last_change":"2024-05-08T05:25:22.645205+0000","last_active":"2024-05-08T05:29:34.085169+0000","last_peered":"2024-05-08T05:29:34.085169+0000","last_clean":"2024-05-08T05:29:34.085169+0000","last_became_active":"2024-05-08T05:25:22.644720+0000","last_became_peered":"2024-05-08T05:25:22.644720+0000","last_unstale":"2024-05-08T05:29:34.085169+0000","last_undegraded":"2024-05-08T05:29:34.085169+0000","last_fullsized":"2024-05-08T05:29:34.085169+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:24:05.630188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087942+0000","last_change":"2024-05-08T05:25:37.055151+0000","last_active":"2024-05-08T05:29:34.087942+0000","last_peered":"2024-05-08T05:29:34.087942+0000","last_clean":"2024-05-08T05:29:34.087942+0000","last_became_active":"2024-05-08T05:25:37.054737+0000","last_became_peered":"2024-05-08T05:25:37.054737+0000","last_unstale":"2024-05-08T05:29:34.087942+0000","last_undegraded":"2024-05-08T05:29:34.087942+0000","last_fullsized":"2024-05-08T05:29:34.087942+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:15:49.135453+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"88'4","reported_seq":23,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088311+0000","last_change":"2024-05-08T05:26:05.148665+0000","last_active":"2024-05-08T05:29:34.088311+0000","last_peered":"2024-05-08T05:29:34.088311+0000","last_clean":"2024-05-08T05:29:34.088311+0000","last_became_active":"2024-05-08T05:26:05.148283+0000","last_became_peered":"2024-05-08T05:26:05.148283+0000","last_unstale":"2024-05-08T05:29:34.088311+0000","last_undegraded":"2024-05-08T05:29:34.088311+0000","last_fullsized":"2024-05-08T05:29:34.088311+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:15:26.197521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,9],"acting":[11,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.4","version":"90'9","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680406+0000","last_change":"2024-05-08T05:25:25.669130+0000","last_active":"2024-05-08T05:29:34.680406+0000","last_peered":"2024-05-08T05:29:34.680406+0000","last_clean":"2024-05-08T05:29:34.680406+0000","last_became_active":"2024-05-08T05:25:25.668812+0000","last_became_peered":"2024-05-08T05:25:25.668812+0000","last_unstale":"2024-05-08T05:29:34.680406+0000","last_undegraded":"2024-05-08T05:29:34.680406+0000","last_fullsized":"2024-05-08T05:29:34.680406+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:47:34.514248+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925554+0000","last_change":"2024-05-08T05:25:37.057146+0000","last_active":"2024-05-08T05:29:33.925554+0000","last_peered":"2024-05-08T05:29:33.925554+0000","last_clean":"2024-05-08T05:29:33.925554+0000","last_became_active":"2024-05-08T05:25:37.056706+0000","last_became_peered":"2024-05-08T05:25:37.056706+0000","last_unstale":"2024-05-08T05:29:33.925554+0000","last_undegraded":"2024-05-08T05:29:33.925554+0000","last_fullsized":"2024-05-08T05:29:33.925554+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:08:09.724088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'105","reported_seq":244,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:37.570728+0000","last_change":"2024-05-08T05:25:22.644573+0000","last_active":"2024-05-08T05:29:37.570728+0000","last_peered":"2024-05-08T05:29:37.570728+0000","last_clean":"2024-05-08T05:29:37.570728+0000","last_became_active":"2024-05-08T05:25:22.644231+0000","last_became_peered":"2024-05-08T05:25:22.644231+0000","last_unstale":"2024-05-08T05:29:37.570728+0000","last_undegraded":"2024-05-08T05:29:37.570728+0000","last_fullsized":"2024-05-08T05:29:37.570728+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":105,"log_dups_size":0,"ondisk_log_size":105,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:02:02.931823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":312,"num_read_kb":277,"num_write":192,"num_write_kb":1287,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"90'5","reported_seq":45,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925815+0000","last_change":"2024-05-08T05:25:25.671036+0000","last_active":"2024-05-08T05:29:33.925815+0000","last_peered":"2024-05-08T05:29:33.925815+0000","last_clean":"2024-05-08T05:29:33.925815+0000","last_became_active":"2024-05-08T05:25:25.669170+0000","last_became_peered":"2024-05-08T05:25:25.669170+0000","last_unstale":"2024-05-08T05:29:33.925815+0000","last_undegraded":"2024-05-08T05:29:33.925815+0000","last_fullsized":"2024-05-08T05:29:33.925815+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:50:18.963232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826699+0000","last_change":"2024-05-08T05:25:37.050656+0000","last_active":"2024-05-08T05:29:33.826699+0000","last_peered":"2024-05-08T05:29:33.826699+0000","last_clean":"2024-05-08T05:29:33.826699+0000","last_became_active":"2024-05-08T05:25:37.050051+0000","last_became_peered":"2024-05-08T05:25:37.050051+0000","last_unstale":"2024-05-08T05:29:33.826699+0000","last_undegraded":"2024-05-08T05:29:33.826699+0000","last_fullsized":"2024-05-08T05:29:33.826699+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:56:07.316216+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"88'7","reported_seq":55,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.093406+0000","last_change":"2024-05-08T05:25:23.614874+0000","last_active":"2024-05-08T05:29:34.093406+0000","last_peered":"2024-05-08T05:29:34.093406+0000","last_clean":"2024-05-08T05:29:34.093406+0000","last_became_active":"2024-05-08T05:25:23.614550+0000","last_became_peered":"2024-05-08T05:25:23.614550+0000","last_unstale":"2024-05-08T05:29:34.093406+0000","last_undegraded":"2024-05-08T05:29:34.093406+0000","last_fullsized":"2024-05-08T05:29:34.093406+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:53:10.183477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088789+0000","last_change":"2024-05-08T05:25:25.658710+0000","last_active":"2024-05-08T05:29:34.088789+0000","last_peered":"2024-05-08T05:29:34.088789+0000","last_clean":"2024-05-08T05:29:34.088789+0000","last_became_active":"2024-05-08T05:25:25.658393+0000","last_became_peered":"2024-05-08T05:25:25.658393+0000","last_unstale":"2024-05-08T05:29:34.088789+0000","last_undegraded":"2024-05-08T05:29:34.088789+0000","last_fullsized":"2024-05-08T05:29:34.088789+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:08:02.629256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092991+0000","last_change":"2024-05-08T05:25:37.059976+0000","last_active":"2024-05-08T05:29:34.092991+0000","last_peered":"2024-05-08T05:29:34.092991+0000","last_clean":"2024-05-08T05:29:34.092991+0000","last_became_active":"2024-05-08T05:25:37.059474+0000","last_became_peered":"2024-05-08T05:25:37.059474+0000","last_unstale":"2024-05-08T05:29:34.092991+0000","last_undegraded":"2024-05-08T05:29:34.092991+0000","last_fullsized":"2024-05-08T05:29:34.092991+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:32:51.946242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"88'2","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825889+0000","last_change":"2024-05-08T05:25:22.748271+0000","last_active":"2024-05-08T05:29:33.825889+0000","last_peered":"2024-05-08T05:29:33.825889+0000","last_clean":"2024-05-08T05:29:33.825889+0000","last_became_active":"2024-05-08T05:25:22.747915+0000","last_became_peered":"2024-05-08T05:25:22.747915+0000","last_unstale":"2024-05-08T05:29:33.825889+0000","last_undegraded":"2024-05-08T05:29:33.825889+0000","last_fullsized":"2024-05-08T05:29:33.825889+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:22:35.262298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"90'2","reported_seq":43,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925478+0000","last_change":"2024-05-08T05:25:25.668722+0000","last_active":"2024-05-08T05:29:33.925478+0000","last_peered":"2024-05-08T05:29:33.925478+0000","last_clean":"2024-05-08T05:29:33.925478+0000","last_became_active":"2024-05-08T05:25:25.668223+0000","last_became_peered":"2024-05-08T05:25:25.668223+0000","last_unstale":"2024-05-08T05:29:33.925478+0000","last_undegraded":"2024-05-08T05:29:33.925478+0000","last_fullsized":"2024-05-08T05:29:33.925478+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:21:43.511567+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825206+0000","last_change":"2024-05-08T05:25:37.056131+0000","last_active":"2024-05-08T05:29:33.825206+0000","last_peered":"2024-05-08T05:29:33.825206+0000","last_clean":"2024-05-08T05:29:33.825206+0000","last_became_active":"2024-05-08T05:25:37.055424+0000","last_became_peered":"2024-05-08T05:25:37.055424+0000","last_unstale":"2024-05-08T05:29:33.825206+0000","last_undegraded":"2024-05-08T05:29:33.825206+0000","last_fullsized":"2024-05-08T05:29:33.825206+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T17:14:43.279408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"88'7","reported_seq":54,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925944+0000","last_change":"2024-05-08T05:25:22.643908+0000","last_active":"2024-05-08T05:29:33.925944+0000","last_peered":"2024-05-08T05:29:33.925944+0000","last_clean":"2024-05-08T05:29:33.925944+0000","last_became_active":"2024-05-08T05:25:22.643542+0000","last_became_peered":"2024-05-08T05:25:22.643542+0000","last_unstale":"2024-05-08T05:29:33.925944+0000","last_undegraded":"2024-05-08T05:29:33.925944+0000","last_fullsized":"2024-05-08T05:29:33.925944+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:19:01.671285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087874+0000","last_change":"2024-05-08T05:25:25.667843+0000","last_active":"2024-05-08T05:29:34.087874+0000","last_peered":"2024-05-08T05:29:34.087874+0000","last_clean":"2024-05-08T05:29:34.087874+0000","last_became_active":"2024-05-08T05:25:25.667462+0000","last_became_peered":"2024-05-08T05:25:25.667462+0000","last_unstale":"2024-05-08T05:29:34.087874+0000","last_undegraded":"2024-05-08T05:29:34.087874+0000","last_fullsized":"2024-05-08T05:29:34.087874+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:15:51.325104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"88'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092927+0000","last_change":"2024-05-08T05:25:37.053017+0000","last_active":"2024-05-08T05:29:34.092927+0000","last_peered":"2024-05-08T05:29:34.092927+0000","last_clean":"2024-05-08T05:29:34.092927+0000","last_became_active":"2024-05-08T05:25:37.052601+0000","last_became_peered":"2024-05-08T05:25:37.052601+0000","last_unstale":"2024-05-08T05:29:34.092927+0000","last_undegraded":"2024-05-08T05:29:34.092927+0000","last_fullsized":"2024-05-08T05:29:34.092927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:00:35.393835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"91'4","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826911+0000","last_change":"2024-05-08T05:25:22.644564+0000","last_active":"2024-05-08T05:29:33.826911+0000","last_peered":"2024-05-08T05:29:33.826911+0000","last_clean":"2024-05-08T05:29:33.826911+0000","last_became_active":"2024-05-08T05:25:22.644170+0000","last_became_peered":"2024-05-08T05:25:22.644170+0000","last_unstale":"2024-05-08T05:29:33.826911+0000","last_undegraded":"2024-05-08T05:29:33.826911+0000","last_fullsized":"2024-05-08T05:29:33.826911+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:49:24.757574+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926667+0000","last_change":"2024-05-08T05:25:25.676983+0000","last_active":"2024-05-08T05:29:33.926667+0000","last_peered":"2024-05-08T05:29:33.926667+0000","last_clean":"2024-05-08T05:29:33.926667+0000","last_became_active":"2024-05-08T05:25:25.676610+0000","last_became_peered":"2024-05-08T05:25:25.676610+0000","last_unstale":"2024-05-08T05:29:33.926667+0000","last_undegraded":"2024-05-08T05:29:33.926667+0000","last_fullsized":"2024-05-08T05:29:33.926667+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:54:39.137935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.092999+0000","last_change":"2024-05-08T05:25:37.059381+0000","last_active":"2024-05-08T05:29:34.092999+0000","last_peered":"2024-05-08T05:29:34.092999+0000","last_clean":"2024-05-08T05:29:34.092999+0000","last_became_active":"2024-05-08T05:25:37.056851+0000","last_became_peered":"2024-05-08T05:25:37.056851+0000","last_unstale":"2024-05-08T05:29:34.092999+0000","last_undegraded":"2024-05-08T05:29:34.092999+0000","last_fullsized":"2024-05-08T05:29:34.092999+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:39:19.339331+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"88'4","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.925636+0000","last_change":"2024-05-08T05:25:22.648099+0000","last_active":"2024-05-08T05:29:33.925636+0000","last_peered":"2024-05-08T05:29:33.925636+0000","last_clean":"2024-05-08T05:29:33.925636+0000","last_became_active":"2024-05-08T05:25:22.647736+0000","last_became_peered":"2024-05-08T05:25:22.647736+0000","last_unstale":"2024-05-08T05:29:33.925636+0000","last_undegraded":"2024-05-08T05:29:33.925636+0000","last_fullsized":"2024-05-08T05:29:33.925636+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:56:12.455706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"90'5","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085452+0000","last_change":"2024-05-08T05:25:25.667565+0000","last_active":"2024-05-08T05:29:34.085452+0000","last_peered":"2024-05-08T05:29:34.085452+0000","last_clean":"2024-05-08T05:29:34.085452+0000","last_became_active":"2024-05-08T05:25:25.667217+0000","last_became_peered":"2024-05-08T05:25:25.667217+0000","last_unstale":"2024-05-08T05:29:34.085452+0000","last_undegraded":"2024-05-08T05:29:34.085452+0000","last_fullsized":"2024-05-08T05:29:34.085452+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:12:05.228227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.825918+0000","last_change":"2024-05-08T05:25:37.056923+0000","last_active":"2024-05-08T05:29:33.825918+0000","last_peered":"2024-05-08T05:29:33.825918+0000","last_clean":"2024-05-08T05:29:33.825918+0000","last_became_active":"2024-05-08T05:25:37.056496+0000","last_became_peered":"2024-05-08T05:25:37.056496+0000","last_unstale":"2024-05-08T05:29:33.825918+0000","last_undegraded":"2024-05-08T05:29:33.825918+0000","last_fullsized":"2024-05-08T05:29:33.825918+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:33:21.998108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'9","reported_seq":258,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:43.690922+0000","last_change":"2024-05-08T05:25:23.614900+0000","last_active":"2024-05-08T05:29:43.690922+0000","last_peered":"2024-05-08T05:29:43.690922+0000","last_clean":"2024-05-08T05:29:43.690922+0000","last_became_active":"2024-05-08T05:25:23.614510+0000","last_became_peered":"2024-05-08T05:25:23.614510+0000","last_unstale":"2024-05-08T05:29:43.690922+0000","last_undegraded":"2024-05-08T05:29:43.690922+0000","last_fullsized":"2024-05-08T05:29:43.690922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:59:19.539656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2484,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":203,"num_read_kb":6,"num_write":9,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085833+0000","last_change":"2024-05-08T05:25:25.669016+0000","last_active":"2024-05-08T05:29:34.085833+0000","last_peered":"2024-05-08T05:29:34.085833+0000","last_clean":"2024-05-08T05:29:34.085833+0000","last_became_active":"2024-05-08T05:25:25.667809+0000","last_became_peered":"2024-05-08T05:25:25.667809+0000","last_unstale":"2024-05-08T05:29:34.085833+0000","last_undegraded":"2024-05-08T05:29:34.085833+0000","last_fullsized":"2024-05-08T05:29:34.085833+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T06:39:56.475622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088613+0000","last_change":"2024-05-08T05:25:37.053630+0000","last_active":"2024-05-08T05:29:34.088613+0000","last_peered":"2024-05-08T05:29:34.088613+0000","last_clean":"2024-05-08T05:29:34.088613+0000","last_became_active":"2024-05-08T05:25:37.053069+0000","last_became_peered":"2024-05-08T05:25:37.053069+0000","last_unstale":"2024-05-08T05:29:34.088613+0000","last_undegraded":"2024-05-08T05:29:34.088613+0000","last_fullsized":"2024-05-08T05:29:34.088613+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:49:56.693447+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"91'67","reported_seq":145,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:37.200537+0000","last_change":"2024-05-08T05:25:22.645284+0000","last_active":"2024-05-08T05:29:37.200537+0000","last_peered":"2024-05-08T05:29:37.200537+0000","last_clean":"2024-05-08T05:29:37.200537+0000","last_became_active":"2024-05-08T05:25:22.644897+0000","last_became_peered":"2024-05-08T05:25:22.644897+0000","last_unstale":"2024-05-08T05:29:37.200537+0000","last_undegraded":"2024-05-08T05:29:37.200537+0000","last_fullsized":"2024-05-08T05:29:37.200537+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":67,"log_dups_size":0,"ondisk_log_size":67,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:53:31.423412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":328302,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":34,"num_read_kb":33,"num_write":76,"num_write_kb":941,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085358+0000","last_change":"2024-05-08T05:25:25.664930+0000","last_active":"2024-05-08T05:29:34.085358+0000","last_peered":"2024-05-08T05:29:34.085358+0000","last_clean":"2024-05-08T05:29:34.085358+0000","last_became_active":"2024-05-08T05:25:25.664395+0000","last_became_peered":"2024-05-08T05:25:25.664395+0000","last_unstale":"2024-05-08T05:29:34.085358+0000","last_undegraded":"2024-05-08T05:29:34.085358+0000","last_fullsized":"2024-05-08T05:29:34.085358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:37.971474+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933025+0000","last_change":"2024-05-08T05:25:37.053258+0000","last_active":"2024-05-08T05:29:33.933025+0000","last_peered":"2024-05-08T05:29:33.933025+0000","last_clean":"2024-05-08T05:29:33.933025+0000","last_became_active":"2024-05-08T05:25:37.052786+0000","last_became_peered":"2024-05-08T05:25:37.052786+0000","last_unstale":"2024-05-08T05:29:33.933025+0000","last_undegraded":"2024-05-08T05:29:33.933025+0000","last_fullsized":"2024-05-08T05:29:33.933025+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:36.032999+0000","last_clean_scrub_stamp":"2024-05-08T05:25:36.032999+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:20:07.384213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"88'5","reported_seq":64,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680512+0000","last_change":"2024-05-08T05:27:05.770358+0000","last_active":"2024-05-08T05:29:34.680512+0000","last_peered":"2024-05-08T05:29:34.680512+0000","last_clean":"2024-05-08T05:29:34.680512+0000","last_became_active":"2024-05-08T05:27:05.766778+0000","last_became_peered":"2024-05-08T05:27:05.766778+0000","last_unstale":"2024-05-08T05:29:34.680512+0000","last_undegraded":"2024-05-08T05:29:34.680512+0000","last_fullsized":"2024-05-08T05:29:34.680512+0000","mapping_epoch":89,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":90,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:21.619776+0000","last_clean_scrub_stamp":"2024-05-08T05:25:21.619776+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:48:52.941549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,1],"acting":[6,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"90'9","reported_seq":50,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.085900+0000","last_change":"2024-05-08T05:25:25.663771+0000","last_active":"2024-05-08T05:29:34.085900+0000","last_peered":"2024-05-08T05:29:34.085900+0000","last_clean":"2024-05-08T05:29:34.085900+0000","last_became_active":"2024-05-08T05:25:25.663281+0000","last_became_peered":"2024-05-08T05:25:25.663281+0000","last_unstale":"2024-05-08T05:29:34.085900+0000","last_undegraded":"2024-05-08T05:29:34.085900+0000","last_fullsized":"2024-05-08T05:29:34.085900+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:04:43.892781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"90'9","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.087991+0000","last_change":"2024-05-08T05:25:25.665446+0000","last_active":"2024-05-08T05:29:34.087991+0000","last_peered":"2024-05-08T05:29:34.087991+0000","last_clean":"2024-05-08T05:29:34.087991+0000","last_became_active":"2024-05-08T05:25:25.664389+0000","last_became_peered":"2024-05-08T05:25:25.664389+0000","last_unstale":"2024-05-08T05:29:34.087991+0000","last_undegraded":"2024-05-08T05:29:34.087991+0000","last_fullsized":"2024-05-08T05:29:34.087991+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:18:00.228100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"90'12","reported_seq":59,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826995+0000","last_change":"2024-05-08T05:26:05.149870+0000","last_active":"2024-05-08T05:29:33.826995+0000","last_peered":"2024-05-08T05:29:33.826995+0000","last_clean":"2024-05-08T05:29:33.826995+0000","last_became_active":"2024-05-08T05:26:05.149476+0000","last_became_peered":"2024-05-08T05:26:05.149476+0000","last_unstale":"2024-05-08T05:29:33.826995+0000","last_undegraded":"2024-05-08T05:29:33.826995+0000","last_fullsized":"2024-05-08T05:29:33.826995+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T05:40:43.401832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,9,4],"acting":[2,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"90'8","reported_seq":49,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826865+0000","last_change":"2024-05-08T05:25:25.667836+0000","last_active":"2024-05-08T05:29:33.826865+0000","last_peered":"2024-05-08T05:29:33.826865+0000","last_clean":"2024-05-08T05:29:33.826865+0000","last_became_active":"2024-05-08T05:25:25.667496+0000","last_became_peered":"2024-05-08T05:25:25.667496+0000","last_unstale":"2024-05-08T05:29:33.826865+0000","last_undegraded":"2024-05-08T05:29:33.826865+0000","last_fullsized":"2024-05-08T05:29:33.826865+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:54:01.482837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"90'7","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.089014+0000","last_change":"2024-05-08T05:25:25.667279+0000","last_active":"2024-05-08T05:29:34.089014+0000","last_peered":"2024-05-08T05:29:34.089014+0000","last_clean":"2024-05-08T05:29:34.089014+0000","last_became_active":"2024-05-08T05:25:25.666960+0000","last_became_peered":"2024-05-08T05:25:25.666960+0000","last_unstale":"2024-05-08T05:29:34.089014+0000","last_undegraded":"2024-05-08T05:29:34.089014+0000","last_fullsized":"2024-05-08T05:29:34.089014+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T12:28:06.335850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"90'4","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680372+0000","last_change":"2024-05-08T05:25:25.667644+0000","last_active":"2024-05-08T05:29:34.680372+0000","last_peered":"2024-05-08T05:29:34.680372+0000","last_clean":"2024-05-08T05:29:34.680372+0000","last_became_active":"2024-05-08T05:25:25.667133+0000","last_became_peered":"2024-05-08T05:25:25.667133+0000","last_unstale":"2024-05-08T05:29:34.680372+0000","last_undegraded":"2024-05-08T05:29:34.680372+0000","last_fullsized":"2024-05-08T05:29:34.680372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T14:00:03.267647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"90'6","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088046+0000","last_change":"2024-05-08T05:25:25.670761+0000","last_active":"2024-05-08T05:29:34.088046+0000","last_peered":"2024-05-08T05:29:34.088046+0000","last_clean":"2024-05-08T05:29:34.088046+0000","last_became_active":"2024-05-08T05:25:25.670342+0000","last_became_peered":"2024-05-08T05:25:25.670342+0000","last_unstale":"2024-05-08T05:29:34.088046+0000","last_undegraded":"2024-05-08T05:29:34.088046+0000","last_fullsized":"2024-05-08T05:29:34.088046+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T16:27:08.069332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"90'15","reported_seq":56,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.827332+0000","last_change":"2024-05-08T05:25:25.664512+0000","last_active":"2024-05-08T05:29:33.827332+0000","last_peered":"2024-05-08T05:29:33.827332+0000","last_clean":"2024-05-08T05:29:33.827332+0000","last_became_active":"2024-05-08T05:25:25.664191+0000","last_became_peered":"2024-05-08T05:25:25.664191+0000","last_unstale":"2024-05-08T05:29:33.827332+0000","last_undegraded":"2024-05-08T05:29:33.827332+0000","last_fullsized":"2024-05-08T05:29:33.827332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T13:25:07.586637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"90'2","reported_seq":42,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.927122+0000","last_change":"2024-05-08T05:25:25.675711+0000","last_active":"2024-05-08T05:29:33.927122+0000","last_peered":"2024-05-08T05:29:33.927122+0000","last_clean":"2024-05-08T05:29:33.927122+0000","last_became_active":"2024-05-08T05:25:25.671890+0000","last_became_peered":"2024-05-08T05:25:25.671890+0000","last_unstale":"2024-05-08T05:29:33.927122+0000","last_undegraded":"2024-05-08T05:29:33.927122+0000","last_fullsized":"2024-05-08T05:29:33.927122+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T09:57:49.026056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"90'6","reported_seq":47,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.933145+0000","last_change":"2024-05-08T05:25:25.662603+0000","last_active":"2024-05-08T05:29:33.933145+0000","last_peered":"2024-05-08T05:29:33.933145+0000","last_clean":"2024-05-08T05:29:33.933145+0000","last_became_active":"2024-05-08T05:25:25.662287+0000","last_became_peered":"2024-05-08T05:25:25.662287+0000","last_unstale":"2024-05-08T05:29:33.933145+0000","last_undegraded":"2024-05-08T05:29:33.933145+0000","last_fullsized":"2024-05-08T05:29:33.933145+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:31:02.152557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"90'8","reported_seq":48,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.926090+0000","last_change":"2024-05-08T05:25:25.667309+0000","last_active":"2024-05-08T05:29:33.926090+0000","last_peered":"2024-05-08T05:29:33.926090+0000","last_clean":"2024-05-08T05:29:33.926090+0000","last_became_active":"2024-05-08T05:25:25.666968+0000","last_became_peered":"2024-05-08T05:25:25.666968+0000","last_unstale":"2024-05-08T05:29:33.926090+0000","last_undegraded":"2024-05-08T05:29:33.926090+0000","last_fullsized":"2024-05-08T05:29:33.926090+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T15:48:28.825242+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"90'10","reported_seq":51,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:33.826130+0000","last_change":"2024-05-08T05:25:25.673579+0000","last_active":"2024-05-08T05:29:33.826130+0000","last_peered":"2024-05-08T05:29:33.826130+0000","last_clean":"2024-05-08T05:29:33.826130+0000","last_became_active":"2024-05-08T05:25:25.673130+0000","last_became_peered":"2024-05-08T05:25:25.673130+0000","last_unstale":"2024-05-08T05:29:33.826130+0000","last_undegraded":"2024-05-08T05:29:33.826130+0000","last_fullsized":"2024-05-08T05:29:33.826130+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T10:46:01.291319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"90'11","reported_seq":52,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.680150+0000","last_change":"2024-05-08T05:25:25.667359+0000","last_active":"2024-05-08T05:29:34.680150+0000","last_peered":"2024-05-08T05:29:34.680150+0000","last_clean":"2024-05-08T05:29:34.680150+0000","last_became_active":"2024-05-08T05:25:25.667044+0000","last_became_peered":"2024-05-08T05:25:25.667044+0000","last_unstale":"2024-05-08T05:29:34.680150+0000","last_undegraded":"2024-05-08T05:29:34.680150+0000","last_fullsized":"2024-05-08T05:29:34.680150+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T08:50:54.973053+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"90'8","reported_seq":53,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-05-08T05:29:34.088126+0000","last_change":"2024-05-08T05:26:05.150407+0000","last_active":"2024-05-08T05:29:34.088126+0000","last_peered":"2024-05-08T05:29:34.088126+0000","last_clean":"2024-05-08T05:29:34.088126+0000","last_became_active":"2024-05-08T05:26:05.149916+0000","last_became_peered":"2024-05-08T05:26:05.149916+0000","last_unstale":"2024-05-08T05:29:34.088126+0000","last_undegraded":"2024-05-08T05:29:34.088126+0000","last_fullsized":"2024-05-08T05:29:34.088126+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-05-08T05:25:24.637594+0000","last_clean_scrub_stamp":"2024-05-08T05:25:24.637594+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-05-09T07:05:57.075928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,8],"acting":[11,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":144,"num_read_kb":248,"num_write":135,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":147456,"data_stored":1771104,"data_compressed":107400,"data_compressed_allocated":135168,"data_compressed_original":1769472,"omap_allocated":0,"internal_metadata":0},"log_size":99,"ondisk_log_size":99,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":805470,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":2372,"num_read_kb":1416,"num_write":523,"num_write_kb":2404,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":13,"num_bytes_recovered":0,"num_keys_recovered":10,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":811008,"data_stored":2416410,"data_compressed":103473,"data_compressed_allocated":135168,"data_compressed_original":2101248,"omap_allocated":0,"internal_metadata":0},"log_size":400,"ondisk_log_size":400,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":71,"seq":304942678083,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":996,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1019904,"data_stored":532312,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":65,"seq":279172874312,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28212,"kb_used_data":1188,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716940,"statfs":{"total":95995035648,"available":95966146560,"internally_reserved":0,"allocated":1216512,"data_stored":1519983,"data_compressed":86432,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":60,"seq":257698037838,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28056,"kb_used_data":1040,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717096,"statfs":{"total":95995035648,"available":95966306304,"internally_reserved":0,"allocated":1064960,"data_stored":928128,"data_compressed":54719,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234067,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32200,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712952,"statfs":{"total":95995035648,"available":95962062848,"internally_reserved":0,"allocated":1048576,"data_stored":924298,"data_compressed":54719,"data_compressed_allocated":73728,"data_compressed_original":491520,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":47,"seq":201863462999,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1060,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1085440,"data_stored":1130768,"data_compressed":71230,"data_compressed_allocated":94208,"data_compressed_original":688128,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":42,"seq":180388626525,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":988,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1011712,"data_stored":527243,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574966,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28080,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717072,"statfs":{"total":95995035648,"available":95966281728,"internally_reserved":0,"allocated":1089536,"data_stored":601091,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738490,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28032,"kb_used_data":1012,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717120,"statfs":{"total":95995035648,"available":95966330880,"internally_reserved":0,"allocated":1036288,"data_stored":529090,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411442,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36412,"kb_used_data":1076,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708740,"statfs":{"total":95995035648,"available":95957749760,"internally_reserved":0,"allocated":1101824,"data_stored":602904,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182508,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32252,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712900,"statfs":{"total":95995035648,"available":95962009600,"internally_reserved":0,"allocated":1093632,"data_stored":1247476,"data_compressed":69921,"data_compressed_allocated":94208,"data_compressed_original":798720,"omap_allocated":1593,"internal_metadata":31914439},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":31,"seq":133143986280,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32184,"kb_used_data":1004,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712968,"statfs":{"total":95995035648,"available":95962079232,"internally_reserved":0,"allocated":1028096,"data_stored":531737,"data_compressed":35430,"data_compressed_allocated":49152,"data_compressed_original":98304,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":36,"seq":154618822754,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32296,"kb_used_data":1112,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712856,"statfs":{"total":95995035648,"available":95961964544,"internally_reserved":0,"allocated":1138688,"data_stored":1452428,"data_compressed":86432,"data_compressed_allocated":114688,"data_compressed_original":995328,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":590368,"data_compressed":35800,"data_compressed_allocated":45056,"data_compressed_original":589824,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":3492,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":75493,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":106496,"data_stored":77306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":721878,"data_compressed":34491,"data_compressed_allocated":45056,"data_compressed_original":700416,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":2043,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":332366,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1645,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":6610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":394604,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":394338,"data_compressed":19289,"data_compressed_allocated":24576,"data_compressed_original":393216,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":172032,"data_stored":404017,"data_compressed":15202,"data_compressed_allocated":20480,"data_compressed_original":307200,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2618,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-05-08T05:29:46.967 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:29:47.226 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:47 smithi059 ceph-mon[30304]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-05-08T05:29:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:47 smithi059 ceph-mon[30304]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:29:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:47 smithi143 ceph-mon[34337]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-05-08T05:29:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:47 smithi143 ceph-mon[34337]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:29:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:47 smithi096 ceph-mon[34253]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 84 KiB/s wr, 51 op/s 2024-05-08T05:29:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:47 smithi096 ceph-mon[34253]: from='client.15396 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-05-08T05:29:48.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.991+0000 7f5b7e038700 1 -- 172.21.15.59:0/1848414436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102630 msgr2=0x7f5b78102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:48.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.991+0000 7f5b7e038700 1 --2- 172.21.15.59:0/1848414436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102630 0x7f5b78102a10 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f5b680099c0 tx=0x7f5b68009cd0 comp rx=0 tx=0).stop 2024-05-08T05:29:48.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.991+0000 7f5b7e038700 1 -- 172.21.15.59:0/1848414436 shutdown_connections 2024-05-08T05:29:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.991+0000 7f5b7e038700 1 --2- 172.21.15.59:0/1848414436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b781076c0 0x7f5b7810f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.991+0000 7f5b7e038700 1 --2- 172.21.15.59:0/1848414436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102630 0x7f5b78102a10 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.991+0000 7f5b7e038700 1 --2- 172.21.15.59:0/1848414436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b78102fe0 0x7f5b78106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.991+0000 7f5b7e038700 1 -- 172.21.15.59:0/1848414436 >> 172.21.15.59:0/1848414436 conn(0x7f5b78079350 msgr2=0x7f5b78079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.991+0000 7f5b7e038700 1 -- 172.21.15.59:0/1848414436 shutdown_connections 2024-05-08T05:29:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.991+0000 7f5b7e038700 1 -- 172.21.15.59:0/1848414436 wait complete. 2024-05-08T05:29:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.992+0000 7f5b7e038700 1 Processor -- start 2024-05-08T05:29:48.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.992+0000 7f5b7e038700 1 -- start start 2024-05-08T05:29:48.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.992+0000 7f5b7e038700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b78102630 0x7f5b78104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:48.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.992+0000 7f5b7e038700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102fe0 0x7f5b781045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:48.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.992+0000 7f5b7e038700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b781076c0 0x7f5b78104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:48.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.992+0000 7f5b7e038700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5b781125a0 con 0x7f5b78102fe0 2024-05-08T05:29:48.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.992+0000 7f5b7e038700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5b78112420 con 0x7f5b781076c0 2024-05-08T05:29:48.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.992+0000 7f5b7e038700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5b78112720 con 0x7f5b78102630 2024-05-08T05:29:48.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b6effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102fe0 0x7f5b781045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:48.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b6effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102fe0 0x7f5b781045c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56264/0 (socket says 172.21.15.59:56264) 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b6effd700 1 -- 172.21.15.59:0/4193004739 learned_addr learned my addr 172.21.15.59:0/4193004739 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b777fe700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b78102630 0x7f5b78104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b77fff700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b781076c0 0x7f5b78104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b6effd700 1 -- 172.21.15.59:0/4193004739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b78102630 msgr2=0x7f5b78104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b6effd700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b78102630 0x7f5b78104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b6effd700 1 -- 172.21.15.59:0/4193004739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b781076c0 msgr2=0x7f5b78104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b6effd700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b781076c0 0x7f5b78104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b6effd700 1 -- 172.21.15.59:0/4193004739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b781a80d0 con 0x7f5b78102fe0 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b6effd700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102fe0 0x7f5b781045c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c9e0 tx=0x7f5b6000ccf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b757fa700 1 -- 172.21.15.59:0/4193004739 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b60007aa0 con 0x7f5b78102fe0 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b781a83c0 con 0x7f5b78102fe0 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b757fa700 1 -- 172.21.15.59:0/4193004739 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b60007c40 con 0x7f5b78102fe0 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.993+0000 7f5b757fa700 1 -- 172.21.15.59:0/4193004739 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000fe60 con 0x7f5b78102fe0 2024-05-08T05:29:48.996 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.994+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b781a8950 con 0x7f5b78102fe0 2024-05-08T05:29:48.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.994+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b7806a190 con 0x7f5b78102fe0 2024-05-08T05:29:48.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.996+0000 7f5b757fa700 1 -- 172.21.15.59:0/4193004739 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f5b6000f450 con 0x7f5b78102fe0 2024-05-08T05:29:48.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.997+0000 7f5b757fa700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5b640b2b10 0x7f5b640b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:48.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.997+0000 7f5b777fe700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5b640b2b10 0x7f5b640b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:48.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.997+0000 7f5b757fa700 1 -- 172.21.15.59:0/4193004739 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f5b600e2ba0 con 0x7f5b78102fe0 2024-05-08T05:29:48.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:48.997+0000 7f5b777fe700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5b640b2b10 0x7f5b640b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009990 tx=0x7f5b6800b870 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:49.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.001+0000 7f5b757fa700 1 -- 172.21.15.59:0/4193004739 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5b60014030 con 0x7f5b78102fe0 2024-05-08T05:29:49.067 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:49 smithi059 ceph-mon[30304]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-05-08T05:29:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.264+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5b7802e510 con 0x7f5b78102fe0 2024-05-08T05:29:49.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.264+0000 7f5b757fa700 1 -- 172.21.15.59:0/4193004739 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f5b600a9f80 con 0x7f5b78102fe0 2024-05-08T05:29:49.265 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:29:49.266 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:29:31.333500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:29:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5b640b2b10 msgr2=0x7f5b640b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5b640b2b10 0x7f5b640b4fd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5b68009990 tx=0x7f5b6800b870 comp rx=0 tx=0).stop 2024-05-08T05:29:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102fe0 msgr2=0x7f5b781045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:49.267 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102fe0 0x7f5b781045c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f5b6000c9e0 tx=0x7f5b6000ccf0 comp rx=0 tx=0).stop 2024-05-08T05:29:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 shutdown_connections 2024-05-08T05:29:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5b640b2b10 0x7f5b640b4fd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5b781076c0 0x7f5b78104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5b78102630 0x7f5b78104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 --2- 172.21.15.59:0/4193004739 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5b78102fe0 0x7f5b781045c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 >> 172.21.15.59:0/4193004739 conn(0x7f5b78079350 msgr2=0x7f5b7806bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 shutdown_connections 2024-05-08T05:29:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:49.267+0000 7f5b7e038700 1 -- 172.21.15.59:0/4193004739 wait complete. 2024-05-08T05:29:49.268 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 30 2024-05-08T05:29:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:49 smithi143 ceph-mon[34337]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-05-08T05:29:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:49 smithi096 ceph-mon[34253]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 77 KiB/s wr, 47 op/s 2024-05-08T05:29:49.884 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:29:50.140 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:50.263 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:50 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/4193004739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:50.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:50 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/4193004739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:50.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:50 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/4193004739' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:50.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 -- 172.21.15.59:0/1469058571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc10f370 msgr2=0x7fa1cc111760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 --2- 172.21.15.59:0/1469058571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc10f370 0x7fa1cc111760 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009ca0 tx=0x7fa1c402a8c0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 -- 172.21.15.59:0/1469058571 shutdown_connections 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 --2- 172.21.15.59:0/1469058571 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc10f370 0x7fa1cc111760 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 --2- 172.21.15.59:0/1469058571 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1cc1020e0 0x7fa1cc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 --2- 172.21.15.59:0/1469058571 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1cc1017c0 0x7fa1cc101ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 -- 172.21.15.59:0/1469058571 >> 172.21.15.59:0/1469058571 conn(0x7fa1cc078a30 msgr2=0x7fa1cc078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 -- 172.21.15.59:0/1469058571 shutdown_connections 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 -- 172.21.15.59:0/1469058571 wait complete. 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 Processor -- start 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.801+0000 7fa1d383f640 1 -- start start 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d383f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc1017c0 0x7fa1cc19b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d383f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1cc1020e0 0x7fa1cc19b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d383f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1cc10f370 0x7fa1cc19fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d383f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc114e50 con 0x7fa1cc1017c0 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d383f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc114cd0 con 0x7fa1cc1020e0 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d383f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc114fd0 con 0x7fa1cc10f370 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d15db640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc1017c0 0x7fa1cc19b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d15db640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc1017c0 0x7fa1cc19b350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56284/0 (socket says 172.21.15.59:56284) 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d15db640 1 -- 172.21.15.59:0/612796459 learned_addr learned my addr 172.21.15.59:0/612796459 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1cc10f370 0x7fa1cc19fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1cc1020e0 0x7fa1cc19b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d15db640 1 -- 172.21.15.59:0/612796459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1cc10f370 msgr2=0x7fa1cc19fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d15db640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1cc10f370 0x7fa1cc19fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d15db640 1 -- 172.21.15.59:0/612796459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1cc1020e0 msgr2=0x7fa1cc19b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d15db640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1cc1020e0 0x7fa1cc19b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d15db640 1 -- 172.21.15.59:0/612796459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1cc1a0340 con 0x7fa1cc1017c0 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1cc1020e0 0x7fa1cc19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:50.805 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1d15db640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc1017c0 0x7fa1cc19b350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800c920 tx=0x7fa1c800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.802+0000 7fa1c27fc640 1 -- 172.21.15.59:0/612796459 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8007c70 con 0x7fa1cc1017c0 2024-05-08T05:29:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.803+0000 7fa1d383f640 1 -- 172.21.15.59:0/612796459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1cc1a0630 con 0x7fa1cc1017c0 2024-05-08T05:29:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.803+0000 7fa1c27fc640 1 -- 172.21.15.59:0/612796459 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1c8007e10 con 0x7fa1cc1017c0 2024-05-08T05:29:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.803+0000 7fa1c27fc640 1 -- 172.21.15.59:0/612796459 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c8018820 con 0x7fa1cc1017c0 2024-05-08T05:29:50.806 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.803+0000 7fa1d383f640 1 -- 172.21.15.59:0/612796459 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1cc1a7ec0 con 0x7fa1cc1017c0 2024-05-08T05:29:50.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.805+0000 7fa1c27fc640 1 -- 172.21.15.59:0/612796459 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fa1c800f450 con 0x7fa1cc1017c0 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.805+0000 7fa1d6630740 1 Processor -- start 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.805+0000 7fa1d6630740 1 -- start start 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.805+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f262f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.805+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 0x55cef2f30bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.805+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f334e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.805+0000 7fa1d6630740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cef2e8b5d0 con 0x55cef2f310f0 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.805+0000 7fa1d6630740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cef2e44890 con 0x55cef2f26830 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.805+0000 7fa1d6630740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cef2e87590 con 0x55cef2f25f10 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d0dda640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f334e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d0dda640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f334e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56294/0 (socket says 172.21.15.59:56294) 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d0dda640 1 -- 172.21.15.59:0/968861741 learned_addr learned my addr 172.21.15.59:0/968861741 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d15db640 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 0x55cef2f30bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f262f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d0dda640 1 -- 172.21.15.59:0/968861741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 msgr2=0x55cef2f262f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f262f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d0dda640 1 -- 172.21.15.59:0/968861741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 msgr2=0x55cef2f30bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 0x55cef2f30bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d0dda640 1 -- 172.21.15.59:0/968861741 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cef2b9d210 con 0x55cef2f310f0 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1c27fc640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1b80b2b30 0x7fa1b80b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1b80b2b30 0x7fa1b80b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f334e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00bcb0 tx=0x7fa1bc00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2aea40ad46cfe853 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1c27fc640 1 -- 172.21.15.59:0/612796459 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa1c80e2e60 con 0x7fa1cc1017c0 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1b77fe640 1 -- 172.21.15.59:0/968861741 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc015070 con 0x55cef2f310f0 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1b77fe640 1 -- 172.21.15.59:0/968861741 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1bc0118f0 con 0x55cef2f310f0 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.806+0000 7fa1b77fe640 1 -- 172.21.15.59:0/968861741 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc010890 con 0x55cef2f310f0 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d6630740 1 -- 172.21.15.59:0/968861741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 msgr2=0x55cef2f334e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:50.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1b80b2b30 0x7fa1b80b4ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009a60 tx=0x7fa1c402f440 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d6630740 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f334e0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00bcb0 tx=0x7fa1bc00f730 comp rx=0 tx=0).stop 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d6630740 1 -- 172.21.15.59:0/968861741 shutdown_connections 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d6630740 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f334e0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d6630740 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 0x55cef2f30bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d6630740 1 --2- 172.21.15.59:0/968861741 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f262f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d6630740 1 -- 172.21.15.59:0/968861741 >> 172.21.15.59:0/968861741 conn(0x55cef2f22420 msgr2=0x55cef2f22800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d6630740 1 -- 172.21.15.59:0/968861741 shutdown_connections 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.807+0000 7fa1d6630740 1 -- 172.21.15.59:0/968861741 wait complete. 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d6630740 1 -- start start 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d6630740 1 Processor -- start 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f2e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 0x55cef2f2ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d6630740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55cef2f33d20 con 0x55cef2f26830 2024-05-08T05:29:50.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d6630740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55cef2e48370 con 0x55cef2f310f0 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d6630740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55cef2f33ea0 con 0x55cef2f25f10 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d6630740 10 client.0 _ll_register_callbacks cb 0x55cef2b9ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d0dda640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f2e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d0dda640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f2e590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56306/0 (socket says 172.21.15.59:56306) 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d0dda640 1 -- 172.21.15.59:0/2465046336 learned_addr learned my addr 172.21.15.59:0/2465046336 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d15db640 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.808+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 0x55cef2f2ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d0dda640 1 -- 172.21.15.59:0/2465046336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 msgr2=0x55cef2f2e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d0dda640 1 -- 172.21.15.59:0/2465046336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 msgr2=0x55cef2f2ead0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 0x55cef2f2ead0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d0dda640 1 -- 172.21.15.59:0/2465046336 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cef2c69620 con 0x55cef2f26830 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d15db640 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f2e590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00bfb0 tx=0x7fa1bc0056d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7903e649fa2e427 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:50.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/2465046336 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc015050 con 0x55cef2f26830 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/2465046336 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa1bc011650 con 0x55cef2f26830 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d6630740 1 -- 172.21.15.59:0/2465046336 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1bc0118f0 con 0x55cef2f26830 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/2465046336 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1bc011aa0 con 0x55cef2f26830 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d6630740 10 client.15414 fetch_fsmap learned FSMap version 30 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d6630740 1 -- 172.21.15.59:0/2465046336 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cef2f3fbb0 con 0x55cef2f26830 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/2465046336 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1bc0118f0 con 0x55cef2f26830 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/2465046336 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa1bc010880 con 0x55cef2f26830 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.809+0000 7fa1d6630740 10 client.15414 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.810+0000 7fa1d6630740 10 client.15414 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.810+0000 7fa1d6630740 20 client.15414 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.810+0000 7fa1d6630740 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55cef2f40fb0 0x55cef2f433a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.810+0000 7fa1d6630740 4 client.15414 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.810+0000 7fa1d6630740 1 -- 172.21.15.59:0/2465046336 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cef2e991e0 con 0x55cef2f40fb0 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.810+0000 7fa1d15db640 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55cef2f40fb0 0x55cef2f433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.810+0000 7fa1b4ff9640 10 client.15414 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.810+0000 7fa1d15db640 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55cef2f40fb0 0x55cef2f433a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b0fc14cc27a5fcc server_cookie=aacfdd653e636a98 in_seq=0 out_seq=0 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/2465046336 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55cef2e991e0 con 0x55cef2f40fb0 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1b4ff9640 10 client.15414 handle_command_reply: tid=0 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 client.15414 shutdown 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 -- 172.21.15.59:0/2465046336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 msgr2=0x55cef2f2e590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f2e590 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00bfb0 tx=0x7fa1bc0056d0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 -- 172.21.15.59:0/2465046336 shutdown_connections 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55cef2f40fb0 0x55cef2f433a0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 0x55cef2f2ead0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f2e590 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 --2- 172.21.15.59:0/2465046336 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1b4ff9640 0 client.15414 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:29:50.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 -- 172.21.15.59:0/2465046336 >> 172.21.15.59:0/2465046336 conn(0x55cef2f22420 msgr2=0x55cef2f28150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 -- 172.21.15.59:0/2465046336 shutdown_connections 2024-05-08T05:29:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 1 -- 172.21.15.59:0/2465046336 wait complete. 2024-05-08T05:29:50.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.811+0000 7fa1d6630740 20 client.15414 trim_cache size 0 max 16384 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d6630740 1 Processor -- start 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d6630740 1 -- start start 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f48b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 0x55cef2f49050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d6630740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1b1ea0 con 0x55cef2f26830 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d6630740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1b39e0 con 0x55cef2f310f0 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d6630740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1b2570 con 0x55cef2f25f10 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d0dda640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f48b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d0dda640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f48b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56314/0 (socket says 172.21.15.59:56314) 2024-05-08T05:29:50.829 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d0dda640 1 -- 172.21.15.59:0/3494616449 learned_addr learned my addr 172.21.15.59:0/3494616449 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d15db640 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d0dda640 1 -- 172.21.15.59:0/3494616449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 msgr2=0x55cef2f2e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 0x55cef2f49050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d0dda640 1 -- 172.21.15.59:0/3494616449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 msgr2=0x55cef2f49050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 0x55cef2f49050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d0dda640 1 -- 172.21.15.59:0/3494616449 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cef2f26f70 con 0x55cef2f26830 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 0x55cef2f49050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.827+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f48b10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00f770 tx=0x7fa1bc00b520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b05c4fc237a3efd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1b6ffd640 1 -- 172.21.15.59:0/3494616449 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc015070 con 0x55cef2f26830 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1b6ffd640 1 -- 172.21.15.59:0/3494616449 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1bc0054d0 con 0x55cef2f26830 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1b6ffd640 1 -- 172.21.15.59:0/3494616449 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc010740 con 0x55cef2f26830 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1d6630740 1 -- 172.21.15.59:0/3494616449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 msgr2=0x55cef2f48b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:50.830 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1d6630740 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f48b10 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc00f770 tx=0x7fa1bc00b520 comp rx=0 tx=0).stop 2024-05-08T05:29:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1d6630740 1 -- 172.21.15.59:0/3494616449 shutdown_connections 2024-05-08T05:29:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1d6630740 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f310f0 0x55cef2f49050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1d6630740 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f26830 0x55cef2f48b10 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1d6630740 1 --2- 172.21.15.59:0/3494616449 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1d6630740 1 -- 172.21.15.59:0/3494616449 >> 172.21.15.59:0/3494616449 conn(0x55cef2f22420 msgr2=0x55cef2f2e010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1d6630740 1 -- 172.21.15.59:0/3494616449 shutdown_connections 2024-05-08T05:29:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.828+0000 7fa1d6630740 1 -- 172.21.15.59:0/3494616449 wait complete. 2024-05-08T05:29:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d6630740 1 -- start start 2024-05-08T05:29:50.831 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d6630740 1 Processor -- start 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 0x55cef2f2b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d6630740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f2b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d6630740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1b3310 con 0x55cef2f310f0 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d6630740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1b2c40 con 0x55cef2f26830 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d6630740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1cc1afe60 con 0x55cef2f25f10 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d6630740 10 client.0 _ll_register_callbacks cb 0x7fa1c80e6800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d1ddc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f2b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d1ddc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f2b8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56322/0 (socket says 172.21.15.59:56322) 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d1ddc640 1 -- 172.21.15.59:0/1048863534 learned_addr learned my addr 172.21.15.59:0/1048863534 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d0dda640 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 0x55cef2f2b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.829+0000 7fa1d15db640 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1d1ddc640 1 -- 172.21.15.59:0/1048863534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 msgr2=0x55cef2f2ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1d1ddc640 1 -- 172.21.15.59:0/1048863534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 msgr2=0x55cef2f2b390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 0x55cef2f2b390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1d1ddc640 1 -- 172.21.15.59:0/1048863534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cef2faa6c0 con 0x55cef2f310f0 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1d1ddc640 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f2b8d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4035000 tx=0x7fa1c40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5bc2aae923f0dab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1b4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/1048863534 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1c4056060 con 0x55cef2f310f0 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/1048863534 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa1c4004450 con 0x55cef2f310f0 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1d6630740 1 -- 172.21.15.59:0/1048863534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1bc0054d0 con 0x55cef2f310f0 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/1048863534 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1c4004690 con 0x55cef2f310f0 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1d6630740 10 client.15420 fetch_fsmap learned FSMap version 30 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1d6630740 1 -- 172.21.15.59:0/1048863534 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55cef2fab8c0 con 0x55cef2f310f0 2024-05-08T05:29:50.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.830+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/1048863534 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1bc0054d0 con 0x55cef2f310f0 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/1048863534 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa1c4050070 con 0x55cef2f310f0 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1d6630740 10 client.15420 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1d6630740 10 client.15420 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1d6630740 20 client.15420 populate_metadata read hostname 'smithi059' 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1d6630740 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55cef2fac070 0x55cef2fae490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1d6630740 4 client.15420 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1d6630740 1 -- 172.21.15.59:0/1048863534 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cef2e93050 con 0x55cef2fac070 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1d15db640 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55cef2fac070 0x55cef2fae490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1d15db640 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55cef2fac070 0x55cef2fae490 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cf7ffa1577fab00c server_cookie=9a153be367cb23db in_seq=0 out_seq=0 2024-05-08T05:29:50.836 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.831+0000 7fa1b4ff9640 10 client.15420 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1b4ff9640 1 -- 172.21.15.59:0/1048863534 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cef2e93050 con 0x55cef2fac070 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1b4ff9640 10 client.15420 handle_command_reply: tid=0 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 client.15420 shutdown 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 -- 172.21.15.59:0/1048863534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 msgr2=0x55cef2f2b8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f2b8d0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4035000 tx=0x7fa1c40095c0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 -- 172.21.15.59:0/1048863534 shutdown_connections 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55cef2fac070 0x55cef2fae490 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55cef2f310f0 0x55cef2f2b8d0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1b4ff9640 0 client.15420 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55cef2f26830 0x55cef2f2b390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 --2- 172.21.15.59:0/1048863534 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55cef2f25f10 0x55cef2f2ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 -- 172.21.15.59:0/1048863534 >> 172.21.15.59:0/1048863534 conn(0x55cef2f22420 msgr2=0x55cef2f21580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:50.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 -- 172.21.15.59:0/1048863534 shutdown_connections 2024-05-08T05:29:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 1 -- 172.21.15.59:0/1048863534 wait complete. 2024-05-08T05:29:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.844+0000 7fa1d6630740 20 client.15420 trim_cache size 0 max 16384 2024-05-08T05:29:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:29:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:29:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:29:50.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:29:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 -- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1b80b2b30 msgr2=0x7fa1b80b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1b80b2b30 0x7fa1b80b4ff0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009a60 tx=0x7fa1c402f440 comp rx=0 tx=0).stop 2024-05-08T05:29:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 -- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc1017c0 msgr2=0x7fa1cc19b350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc1017c0 0x7fa1cc19b350 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fa1c800c920 tx=0x7fa1c800cde0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 -- 172.21.15.59:0/612796459 shutdown_connections 2024-05-08T05:29:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1b80b2b30 0x7fa1b80b4ff0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1cc10f370 0x7fa1cc19fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1cc1020e0 0x7fa1cc19b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 --2- 172.21.15.59:0/612796459 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1cc1017c0 0x7fa1cc19b350 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 -- 172.21.15.59:0/612796459 >> 172.21.15.59:0/612796459 conn(0x7fa1cc078a30 msgr2=0x7fa1cc110820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 -- 172.21.15.59:0/612796459 shutdown_connections 2024-05-08T05:29:50.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:29:50.845+0000 7fa1d383f640 1 -- 172.21.15.59:0/612796459 wait complete. 2024-05-08T05:29:51.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:51 smithi059 ceph-mon[30304]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-05-08T05:29:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:51 smithi143 ceph-mon[34337]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-05-08T05:29:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:51 smithi096 ceph-mon[34253]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 77 KiB/s wr, 48 op/s 2024-05-08T05:29:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.936+0000 7f577d44d700 1 -- 172.21.15.59:0/3330731796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778102390 msgr2=0x7f5778102810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.936+0000 7f577d44d700 1 --2- 172.21.15.59:0/3330731796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778102390 0x7f5778102810 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f576c009990 tx=0x7f576c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:29:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.936+0000 7f577d44d700 1 -- 172.21.15.59:0/3330731796 shutdown_connections 2024-05-08T05:29:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.936+0000 7f577d44d700 1 --2- 172.21.15.59:0/3330731796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5778102d50 0x7f577810f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.936+0000 7f577d44d700 1 --2- 172.21.15.59:0/3330731796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5778108400 0x7f57781087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.936+0000 7f577d44d700 1 --2- 172.21.15.59:0/3330731796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778102390 0x7f5778102810 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.936+0000 7f577d44d700 1 -- 172.21.15.59:0/3330731796 >> 172.21.15.59:0/3330731796 conn(0x7f5778079310 msgr2=0x7f5778079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.936+0000 7f577d44d700 1 -- 172.21.15.59:0/3330731796 shutdown_connections 2024-05-08T05:29:51.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.936+0000 7f577d44d700 1 -- 172.21.15.59:0/3330731796 wait complete. 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f577d44d700 1 Processor -- start 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f577d44d700 1 -- start start 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f577d44d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5778102390 0x7f577819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f577d44d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5778102d50 0x7f577819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f577d44d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778108400 0x7f5778199850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f577d44d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5778112520 con 0x7f5778108400 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f577d44d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f57781123a0 con 0x7f5778102390 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f577d44d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f57781126a0 con 0x7f5778102d50 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f57777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778108400 0x7f5778199850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f57767fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5778102d50 0x7f577819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f57777fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778108400 0x7f5778199850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56338/0 (socket says 172.21.15.59:56338) 2024-05-08T05:29:51.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.937+0000 7f57777fe700 1 -- 172.21.15.59:0/2900415122 learned_addr learned my addr 172.21.15.59:0/2900415122 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:51.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f5776ffd700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5778102390 0x7f577819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:51.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f57777fe700 1 -- 172.21.15.59:0/2900415122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5778102d50 msgr2=0x7f577819fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:51.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f57777fe700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5778102d50 0x7f577819fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:51.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f57777fe700 1 -- 172.21.15.59:0/2900415122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5778102390 msgr2=0x7f577819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:51.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f57777fe700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5778102390 0x7f577819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:51.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f57777fe700 1 -- 172.21.15.59:0/2900415122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f577819a030 con 0x7f5778108400 2024-05-08T05:29:51.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f57777fe700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778108400 0x7f5778199850 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f577000c900 tx=0x7f577000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:51.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f575ffff700 1 -- 172.21.15.59:0/2900415122 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57700078d0 con 0x7f5778108400 2024-05-08T05:29:51.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f577819a320 con 0x7f5778108400 2024-05-08T05:29:51.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f575ffff700 1 -- 172.21.15.59:0/2900415122 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f577000f450 con 0x7f5778108400 2024-05-08T05:29:51.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f575ffff700 1 -- 172.21.15.59:0/2900415122 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5770018620 con 0x7f5778108400 2024-05-08T05:29:51.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.938+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57781a6530 con 0x7f5778108400 2024-05-08T05:29:51.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.939+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5778103ab0 con 0x7f5778108400 2024-05-08T05:29:51.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.941+0000 7f575ffff700 1 -- 172.21.15.59:0/2900415122 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f5770007a70 con 0x7f5778108400 2024-05-08T05:29:51.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.941+0000 7f575ffff700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f57600b2b10 0x7f57600b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:51.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.942+0000 7f575ffff700 1 -- 172.21.15.59:0/2900415122 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f57700e2b10 con 0x7f5778108400 2024-05-08T05:29:51.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.942+0000 7f5776ffd700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f57600b2b10 0x7f57600b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:51.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.942+0000 7f5776ffd700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f57600b2b10 0x7f57600b4fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5768007fd0 tx=0x7f57680079e0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:51.946 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:51.946+0000 7f575ffff700 1 -- 172.21.15.59:0/2900415122 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5770014030 con 0x7f5778108400 2024-05-08T05:29:52.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.211+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57781a6760 con 0x7f5778108400 2024-05-08T05:29:52.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.211+0000 7f575ffff700 1 -- 172.21.15.59:0/2900415122 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f5770007d60 con 0x7f5778108400 2024-05-08T05:29:52.212 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:29:52.212 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:29:31.333500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:29:52.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f57600b2b10 msgr2=0x7f57600b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:52.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f57600b2b10 0x7f57600b4fd0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f5768007fd0 tx=0x7f57680079e0 comp rx=0 tx=0).stop 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778108400 msgr2=0x7f5778199850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778108400 0x7f5778199850 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f577000c900 tx=0x7f577000cc10 comp rx=0 tx=0).stop 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 shutdown_connections 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f57600b2b10 0x7f57600b4fd0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5778108400 0x7f5778199850 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5778102390 0x7f577819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 --2- 172.21.15.59:0/2900415122 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5778102d50 0x7f577819fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 >> 172.21.15.59:0/2900415122 conn(0x7f5778079310 msgr2=0x7f5778101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 shutdown_connections 2024-05-08T05:29:52.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:52.214+0000 7f577d44d700 1 -- 172.21.15.59:0/2900415122 wait complete. 2024-05-08T05:29:52.216 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 30 2024-05-08T05:29:52.994 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:29:53.255 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:53.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:53 smithi059 ceph-mon[30304]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-05-08T05:29:53.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:53 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2900415122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:53 smithi143 ceph-mon[34337]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-05-08T05:29:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:53 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2900415122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:53 smithi096 ceph-mon[34253]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 77 KiB/s wr, 50 op/s 2024-05-08T05:29:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:53 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2900415122' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:29:55.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.037+0000 7fa4092df700 1 -- 172.21.15.59:0/10967065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4041023a0 msgr2=0x7fa404102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:55.040 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.037+0000 7fa4092df700 1 --2- 172.21.15.59:0/10967065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4041023a0 0x7fa404102820 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8009990 tx=0x7fa3f8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.037+0000 7fa4092df700 1 -- 172.21.15.59:0/10967065 shutdown_connections 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.037+0000 7fa4092df700 1 --2- 172.21.15.59:0/10967065 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa404102d60 0x7fa40410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.037+0000 7fa4092df700 1 --2- 172.21.15.59:0/10967065 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa404108410 0x7fa4041087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.037+0000 7fa4092df700 1 --2- 172.21.15.59:0/10967065 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa4041023a0 0x7fa404102820 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.037+0000 7fa4092df700 1 -- 172.21.15.59:0/10967065 >> 172.21.15.59:0/10967065 conn(0x7fa404079350 msgr2=0x7fa404079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.037+0000 7fa4092df700 1 -- 172.21.15.59:0/10967065 shutdown_connections 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.037+0000 7fa4092df700 1 -- 172.21.15.59:0/10967065 wait complete. 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4092df700 1 Processor -- start 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4092df700 1 -- start start 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4092df700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4041023a0 0x7fa40419f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4092df700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa404102d60 0x7fa40419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4092df700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa404108410 0x7fa4041999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4092df700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa4041125b0 con 0x7fa404102d60 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4092df700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa404112430 con 0x7fa4041023a0 2024-05-08T05:29:55.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4092df700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa404112730 con 0x7fa404108410 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4027fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa404102d60 0x7fa40419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4027fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa404102d60 0x7fa40419fd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56362/0 (socket says 172.21.15.59:56362) 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa4027fc700 1 -- 172.21.15.59:0/996063842 learned_addr learned my addr 172.21.15.59:0/996063842 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.038+0000 7fa402ffd700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4041023a0 0x7fa40419f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa4037fe700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa404108410 0x7fa4041999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa4027fc700 1 -- 172.21.15.59:0/996063842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa404108410 msgr2=0x7fa4041999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa4027fc700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa404108410 0x7fa4041999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa4027fc700 1 -- 172.21.15.59:0/996063842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4041023a0 msgr2=0x7fa40419f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa4027fc700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4041023a0 0x7fa40419f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa4027fc700 1 -- 172.21.15.59:0/996063842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa40419a150 con 0x7fa404102d60 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa402ffd700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4041023a0 0x7fa40419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa4027fc700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa404102d60 0x7fa40419fd70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8000c00 tx=0x7fa3f80047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa3ebfff700 1 -- 172.21.15.59:0/996063842 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f8027070 con 0x7fa404102d60 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa40419a3e0 con 0x7fa404102d60 2024-05-08T05:29:55.042 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa3ebfff700 1 -- 172.21.15.59:0/996063842 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3f80041d0 con 0x7fa404102d60 2024-05-08T05:29:55.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa3ebfff700 1 -- 172.21.15.59:0/996063842 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3f800f460 con 0x7fa404102d60 2024-05-08T05:29:55.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.039+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa404103fb0 con 0x7fa404102d60 2024-05-08T05:29:55.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.040+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa40406a190 con 0x7fa404102d60 2024-05-08T05:29:55.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.041+0000 7fa3ebfff700 1 -- 172.21.15.59:0/996063842 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7fa3f802a070 con 0x7fa404102d60 2024-05-08T05:29:55.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.042+0000 7fa3ebfff700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa3ec0b2b10 0x7fa3ec0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:29:55.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.042+0000 7fa402ffd700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa3ec0b2b10 0x7fa3ec0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:29:55.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.042+0000 7fa3ebfff700 1 -- 172.21.15.59:0/996063842 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa3f80e3c40 con 0x7fa404102d60 2024-05-08T05:29:55.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.043+0000 7fa402ffd700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa3ec0b2b10 0x7fa3ec0b4fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa40419ab30 tx=0x7fa3f4008040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:29:55.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.047+0000 7fa3ebfff700 1 -- 172.21.15.59:0/996063842 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa3f8013070 con 0x7fa404102d60 2024-05-08T05:29:55.263 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.262+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa4040640b0 con 0x7fa404102d60 2024-05-08T05:29:55.263 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:55 smithi059 ceph-mon[30304]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:29:55.265 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.264+0000 7fa3ebfff700 1 -- 172.21.15.59:0/996063842 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+25071 (secure 0 0 0) 0x7fa3f80ab0b0 con 0x7fa404102d60 2024-05-08T05:29:55.265 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:29:55.265 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":91,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:29:33.818980+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"86","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/2839794658":"2024-05-09T05:29:33.818933+0000","172.21.15.59:6801/546809822":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/3042855743":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/1220365613":"2024-05-09T05:29:33.818933+0000","172.21.15.59:6800/546809822":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3190007358":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa3ec0b2b10 msgr2=0x7fa3ec0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa3ec0b2b10 0x7fa3ec0b4fd0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fa40419ab30 tx=0x7fa3f4008040 comp rx=0 tx=0).stop 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa404102d60 msgr2=0x7fa40419fd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa404102d60 0x7fa40419fd70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fa3f8000c00 tx=0x7fa3f80047b0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 shutdown_connections 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa3ec0b2b10 0x7fa3ec0b4fd0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa404108410 0x7fa4041999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa4041023a0 0x7fa40419f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 --2- 172.21.15.59:0/996063842 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa404102d60 0x7fa40419fd70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 >> 172.21.15.59:0/996063842 conn(0x7fa404079350 msgr2=0x7fa4041016d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 shutdown_connections 2024-05-08T05:29:55.268 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:29:55.267+0000 7fa4092df700 1 -- 172.21.15.59:0/996063842 wait complete. 2024-05-08T05:29:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:55 smithi143 ceph-mon[34337]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:29:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:55 smithi096 ceph-mon[34253]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:29:56.105 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mon.a --format=json config get run_dir 2024-05-08T05:29:56.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:56 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/996063842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:29:56.363 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:56.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:56 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/996063842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:29:56.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:56 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/996063842' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:29:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:57 smithi143 ceph-mon[34337]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-05-08T05:29:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:57 smithi096 ceph-mon[34253]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-05-08T05:29:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:57 smithi059 ceph-mon[30304]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 1.7 KiB/s wr, 16 op/s 2024-05-08T05:29:58.928 INFO:teuthology.orchestra.run.smithi059.stdout:{"run_dir":"/var/run/ceph"} 2024-05-08T05:29:58.929 DEBUG:tasks.cephfs.filesystem:_json_asok output { "run_dir": "/var/run/ceph" } 2024-05-08T05:29:58.929 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with seed 5998 2024-05-08T05:29:58.929 DEBUG:tasks.quiescer:Yielding 2024-05-08T05:29:58.929 INFO:teuthology.run_tasks:Running task workunit... 2024-05-08T05:29:58.943 INFO:tasks.workunit:Pulling workunits from ref bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:29:58.944 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-05-08T05:29:58.944 INFO:tasks.workunit:timeout=3h 2024-05-08T05:29:58.944 INFO:tasks.workunit:cleanup=True 2024-05-08T05:29:58.944 DEBUG:teuthology.orchestra.run.smithi059:> stat -- /home/ubuntu/cephtest/mnt.0 2024-05-08T05:29:58.945 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:29:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-05-08T05:29:58.963 INFO:teuthology.orchestra.run.smithi059.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-05-08T05:29:58.963 INFO:teuthology.orchestra.run.smithi059.stdout:Device: 67h/103d Inode: 1099511628283 Links: 2 2024-05-08T05:29:58.963 INFO:teuthology.orchestra.run.smithi059.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-05-08T05:29:58.963 INFO:teuthology.orchestra.run.smithi059.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-05-08T05:29:58.964 INFO:teuthology.orchestra.run.smithi059.stdout:Access: 2024-05-08 05:26:43.165886687 +0000 2024-05-08T05:29:58.964 INFO:teuthology.orchestra.run.smithi059.stdout:Modify: 2024-05-08 05:26:43.165886687 +0000 2024-05-08T05:29:58.964 INFO:teuthology.orchestra.run.smithi059.stdout:Change: 2024-05-08 05:29:30.574433186 +0000 2024-05-08T05:29:58.964 INFO:teuthology.orchestra.run.smithi059.stdout: Birth: 2024-05-08 05:26:43.165886687 +0000 2024-05-08T05:29:58.964 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-05-08T05:29:58.964 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-05-08T05:29:59.037 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bee299f4ac344e4c45e550d7a80abcadd7bc3d5a 2024-05-08T05:29:59.093 INFO:tasks.workunit.client.0.smithi059.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-05-08T05:29:59.205 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:29:59.328 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:29:59 smithi059 ceph-mon[30304]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-08T05:29:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:29:59 smithi143 ceph-mon[34337]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-08T05:29:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:29:59 smithi096 ceph-mon[34253]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 1.7 KiB/s wr, 12 op/s 2024-05-08T05:30:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:00 smithi059 ceph-mon[30304]: overall HEALTH_OK 2024-05-08T05:30:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:00 smithi143 ceph-mon[34337]: overall HEALTH_OK 2024-05-08T05:30:00.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:00 smithi096 ceph-mon[34253]: overall HEALTH_OK 2024-05-08T05:30:00.966 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:30:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:30:00.847+0000 7fe82d3a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:30:00.966 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:30:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:30:00.847+0000 7fe82d3a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:30:00.966 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:30:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:30:00.847+0000 7fe82d3a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:30:00.966 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:30:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:30:00.847+0000 7fe82d3a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:30:00.966 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:30:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x[30539]: 2024-05-08T05:30:00.847+0000 7fe82d3a0700 -1 client.0 error registering admin socket command: (17) File exists 2024-05-08T05:30:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.971+0000 7f1538473700 1 -- 172.21.15.59:0/1726409331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1530103390 msgr2=0x7f1530107000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.971+0000 7f1538473700 1 --2- 172.21.15.59:0/1726409331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1530103390 0x7f1530107000 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f15200099c0 tx=0x7f1520009cd0 comp rx=0 tx=0).stop 2024-05-08T05:30:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.971+0000 7f1538473700 1 -- 172.21.15.59:0/1726409331 shutdown_connections 2024-05-08T05:30:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.971+0000 7f1538473700 1 --2- 172.21.15.59:0/1726409331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1530107730 0x7f153010f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.971+0000 7f1538473700 1 --2- 172.21.15.59:0/1726409331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f15301029e0 0x7f1530102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.971+0000 7f1538473700 1 --2- 172.21.15.59:0/1726409331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1530103390 0x7f1530107000 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.971+0000 7f1538473700 1 -- 172.21.15.59:0/1726409331 >> 172.21.15.59:0/1726409331 conn(0x7f1530079350 msgr2=0x7f1530079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.971+0000 7f1538473700 1 -- 172.21.15.59:0/1726409331 shutdown_connections 2024-05-08T05:30:00.972 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.971+0000 7f1538473700 1 -- 172.21.15.59:0/1726409331 wait complete. 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1538473700 1 Processor -- start 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1538473700 1 -- start start 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1538473700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15301029e0 0x7f153019f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1538473700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1530103390 0x7f153019fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1538473700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1530107730 0x7f1530199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1538473700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f15301125d0 con 0x7f15301029e0 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1538473700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1530112450 con 0x7f1530107730 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1538473700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1530112750 con 0x7f1530103390 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f153620f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15301029e0 0x7f153019f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f153620f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15301029e0 0x7f153019f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46576/0 (socket says 172.21.15.59:46576) 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f153620f700 1 -- 172.21.15.59:0/2699676501 learned_addr learned my addr 172.21.15.59:0/2699676501 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:00.974 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1535a0e700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1530103390 0x7f153019fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f1536a10700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1530107730 0x7f1530199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f153620f700 1 -- 172.21.15.59:0/2699676501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1530103390 msgr2=0x7f153019fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f153620f700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1530103390 0x7f153019fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f153620f700 1 -- 172.21.15.59:0/2699676501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1530107730 msgr2=0x7f1530199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f153620f700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1530107730 0x7f1530199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.972+0000 7f153620f700 1 -- 172.21.15.59:0/2699676501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f153019a260 con 0x7f15301029e0 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.973+0000 7f153620f700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15301029e0 0x7f153019f8f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f152c00ba50 tx=0x7f152c00bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.973+0000 7f15277fe700 1 -- 172.21.15.59:0/2699676501 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c019070 con 0x7f15301029e0 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.973+0000 7f15277fe700 1 -- 172.21.15.59:0/2699676501 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f152c014440 con 0x7f15301029e0 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.973+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f153019a550 con 0x7f15301029e0 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.973+0000 7f15277fe700 1 -- 172.21.15.59:0/2699676501 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f152c00ce60 con 0x7f15301029e0 2024-05-08T05:30:00.975 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.973+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15301a67c0 con 0x7f15301029e0 2024-05-08T05:30:00.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.974+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1530103ae0 con 0x7f15301029e0 2024-05-08T05:30:00.976 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.976+0000 7f15277fe700 1 -- 172.21.15.59:0/2699676501 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 24) ==== 149879+0+0 (secure 0 0 0) 0x7f152c01c070 con 0x7f15301029e0 2024-05-08T05:30:00.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.976+0000 7f15277fe700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f151c0b2b10 0x7f151c0b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:00.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.976+0000 7f1535a0e700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f151c0b2b10 0x7f151c0b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:00.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.977+0000 7f15277fe700 1 -- 172.21.15.59:0/2699676501 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f152c0e2410 con 0x7f15301029e0 2024-05-08T05:30:00.977 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.977+0000 7f1535a0e700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f151c0b2b10 0x7f151c0b4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1520009f50 tx=0x7f1520005c40 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:00.980 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:00.979+0000 7f15277fe700 1 -- 172.21.15.59:0/2699676501 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f152c0a9940 con 0x7f15301029e0 2024-05-08T05:30:01.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.251+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f153019b010 con 0x7f15301029e0 2024-05-08T05:30:01.252 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:01 smithi059 ceph-mon[30304]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-08T05:30:01.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.251+0000 7f15277fe700 1 -- 172.21.15.59:0/2699676501 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f153019b010 con 0x7f15301029e0 2024-05-08T05:30:01.252 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:30:01.252 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:29:31.333500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:30:01.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f151c0b2b10 msgr2=0x7f151c0b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:01.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f151c0b2b10 0x7f151c0b4fd0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1520009f50 tx=0x7f1520005c40 comp rx=0 tx=0).stop 2024-05-08T05:30:01.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15301029e0 msgr2=0x7f153019f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:01.255 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15301029e0 0x7f153019f8f0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f152c00ba50 tx=0x7f152c00bd60 comp rx=0 tx=0).stop 2024-05-08T05:30:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 shutdown_connections 2024-05-08T05:30:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f151c0b2b10 0x7f151c0b4fd0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1530107730 0x7f1530199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f15301029e0 0x7f153019f8f0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 --2- 172.21.15.59:0/2699676501 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1530103390 0x7f153019fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 >> 172.21.15.59:0/2699676501 conn(0x7f1530079350 msgr2=0x7f153006bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 shutdown_connections 2024-05-08T05:30:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:01.255+0000 7f1538473700 1 -- 172.21.15.59:0/2699676501 wait complete. 2024-05-08T05:30:01.256 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 30 2024-05-08T05:30:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:01 smithi143 ceph-mon[34337]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-08T05:30:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:01 smithi096 ceph-mon[34253]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.7 KiB/s wr, 14 op/s 2024-05-08T05:30:01.962 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 30, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-05-08T05:25:27.601131+0000', 'modified': '2024-05-08T05:29:31.333500+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24473, 'mds_1': 24443, 'mds_2': 24433, 'mds_3': 24421, 'mds_4': 24419}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14565': {'gid': 14565, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6839/2756426260', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6838', 'nonce': 2756426260}, {'type': 'v1', 'addr': '172.21.15.59:6839', 'nonce': 2756426260}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14571': {'gid': 14571, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.59:6841/2496793439', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.59:6840', 'nonce': 2496793439}, {'type': 'v1', 'addr': '172.21.15.59:6841', 'nonce': 2496793439}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24385': {'gid': 24385, 'name': 'l', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6833/1879950931', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6832', 'nonce': 1879950931}, {'type': 'v1', 'addr': '172.21.15.143:6833', 'nonce': 1879950931}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24401': {'gid': 24401, 'name': 'k', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.96:6833/2593895617', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6832', 'nonce': 2593895617}, {'type': 'v1', 'addr': '172.21.15.96:6833', 'nonce': 2593895617}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24415': {'gid': 24415, 'name': 'f', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.143:6835/2504523054', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6834', 'nonce': 2504523054}, {'type': 'v1', 'addr': '172.21.15.143:6835', 'nonce': 2504523054}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24419': {'gid': 24419, 'name': 'e', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 26, 'addr': '172.21.15.96:6835/2294126295', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6834', 'nonce': 2294126295}, {'type': 'v1', 'addr': '172.21.15.96:6835', 'nonce': 2294126295}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24421': {'gid': 24421, 'name': 'c', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 24, 'addr': '172.21.15.143:6837/555225813', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6836', 'nonce': 555225813}, {'type': 'v1', 'addr': '172.21.15.143:6837', 'nonce': 555225813}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24433': {'gid': 24433, 'name': 'i', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 22, 'addr': '172.21.15.143:6839/2166229809', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.143:6838', 'nonce': 2166229809}, {'type': 'v1', 'addr': '172.21.15.143:6839', 'nonce': 2166229809}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24443': {'gid': 24443, 'name': 'h', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 23, 'addr': '172.21.15.96:6837/780978870', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6836', 'nonce': 780978870}, {'type': 'v1', 'addr': '172.21.15.96:6837', 'nonce': 780978870}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24473': {'gid': 24473, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 5, 'addr': '172.21.15.96:6839/126053836', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.96:6838', 'nonce': 126053836}, {'type': 'v1', 'addr': '172.21.15.96:6839', 'nonce': 126053836}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24473, 'qdb_cluster': [24473, 24443, 24421, 24419, 24433]} 2024-05-08T05:30:01.962 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-05-08T05:30:01.962 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.e status 2024-05-08T05:30:02.268 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:30:02.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:02 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2699676501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:30:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:02 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2699676501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:30:02.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:02 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2699676501' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:30:03.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:03 smithi059 ceph-mon[30304]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-05-08T05:30:03.417 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:03 smithi059 ceph-mon[30304]: mgrmap e25: x(active, since 28s), standbys: y, z 2024-05-08T05:30:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:03 smithi143 ceph-mon[34337]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-05-08T05:30:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:03 smithi143 ceph-mon[34337]: mgrmap e25: x(active, since 28s), standbys: y, z 2024-05-08T05:30:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:03 smithi096 ceph-mon[34253]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.6 KiB/s wr, 15 op/s 2024-05-08T05:30:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:03 smithi096 ceph-mon[34253]: mgrmap e25: x(active, since 28s), standbys: y, z 2024-05-08T05:30:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.939+0000 7f65681e4640 1 -- 172.21.15.59:0/147848679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6560101c00 msgr2=0x7f6560102080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.939+0000 7f65681e4640 1 --2- 172.21.15.59:0/147848679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6560101c00 0x7f6560102080 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f6550009900 tx=0x7f655002a410 comp rx=0 tx=0).stop 2024-05-08T05:30:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.939+0000 7f65681e4640 1 -- 172.21.15.59:0/147848679 shutdown_connections 2024-05-08T05:30:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.939+0000 7f65681e4640 1 --2- 172.21.15.59:0/147848679 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65601025c0 0x7f656010e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.939+0000 7f65681e4640 1 --2- 172.21.15.59:0/147848679 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6560101c00 0x7f6560102080 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.939+0000 7f65681e4640 1 --2- 172.21.15.59:0/147848679 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6560106850 0x7f6560106c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.939+0000 7f65681e4640 1 -- 172.21.15.59:0/147848679 >> 172.21.15.59:0/147848679 conn(0x7f65600776b0 msgr2=0x7f6560077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 -- 172.21.15.59:0/147848679 shutdown_connections 2024-05-08T05:30:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 -- 172.21.15.59:0/147848679 wait complete. 2024-05-08T05:30:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 Processor -- start 2024-05-08T05:30:03.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 -- start start 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6560101c00 0x7f6560199ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601025c0 0x7f656019a400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560106850 0x7f65601a4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6560111b80 con 0x7f65601025c0 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6560111a00 con 0x7f6560101c00 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.940+0000 7f65681e4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6560111d00 con 0x7f6560106850 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f656577f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601025c0 0x7f656019a400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f6565f80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6560101c00 0x7f6560199ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f6565f80640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6560101c00 0x7f6560199ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:48964/0 (socket says 172.21.15.59:48964) 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f6565f80640 1 -- 172.21.15.59:0/4195921996 learned_addr learned my addr 172.21.15.59:0/4195921996 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:03.943 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f6566781640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560106850 0x7f65601a4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f656577f640 1 -- 172.21.15.59:0/4195921996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560106850 msgr2=0x7f65601a4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f656577f640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560106850 0x7f65601a4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f656577f640 1 -- 172.21.15.59:0/4195921996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6560101c00 msgr2=0x7f6560199ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f656577f640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6560101c00 0x7f6560199ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f656577f640 1 -- 172.21.15.59:0/4195921996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65601a5480 con 0x7f65601025c0 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f6566781640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560106850 0x7f65601a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f6565f80640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6560101c00 0x7f6560199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f656577f640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601025c0 0x7f656019a400 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f655002aa00 tx=0x7f6550003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f6556ffd640 1 -- 172.21.15.59:0/4195921996 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655003f070 con 0x7f65601025c0 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f65681e4640 1 -- 172.21.15.59:0/4195921996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65601a5650 con 0x7f65601025c0 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f6556ffd640 1 -- 172.21.15.59:0/4195921996 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6550031b50 con 0x7f65601025c0 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f6556ffd640 1 -- 172.21.15.59:0/4195921996 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6550035030 con 0x7f65601025c0 2024-05-08T05:30:03.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.941+0000 7f65681e4640 1 -- 172.21.15.59:0/4195921996 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65601a5ac0 con 0x7f65601025c0 2024-05-08T05:30:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.944+0000 7f6556ffd640 1 -- 172.21.15.59:0/4195921996 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f6550042070 con 0x7f65601025c0 2024-05-08T05:30:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.944+0000 7f656afda740 1 Processor -- start 2024-05-08T05:30:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.944+0000 7f656afda740 1 -- start start 2024-05-08T05:30:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.944+0000 7f656afda740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bde1f10 0x56173bde22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.944+0000 7f656afda740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde2830 0x56173bdecbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.944+0000 7f656afda740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bded0f0 0x56173bdef4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.944+0000 7f656afda740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56173bd475d0 con 0x56173bde2830 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.944+0000 7f656afda740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56173bd00890 con 0x56173bded0f0 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.944+0000 7f656afda740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56173bd43590 con 0x56173bde1f10 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6565f80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde2830 0x56173bdecbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6565f80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde2830 0x56173bdecbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46604/0 (socket says 172.21.15.59:46604) 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6565f80640 1 -- 172.21.15.59:0/3040041067 learned_addr learned my addr 172.21.15.59:0/3040041067 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6566781640 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bde1f10 0x56173bde22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f656577f640 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bded0f0 0x56173bdef4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6565f80640 1 -- 172.21.15.59:0/3040041067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bde1f10 msgr2=0x56173bde22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6565f80640 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bde1f10 0x56173bde22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6565f80640 1 -- 172.21.15.59:0/3040041067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bded0f0 msgr2=0x56173bdef4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6565f80640 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bded0f0 0x56173bdef4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6565f80640 1 -- 172.21.15.59:0/3040041067 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173ba59210 con 0x56173bde2830 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6566781640 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bde1f10 0x56173bde22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:03.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f656577f640 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bded0f0 0x56173bdef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6556ffd640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f654c0b2bb0 0x7f654c0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6566781640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f654c0b2bb0 0x7f654c0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6565f80640 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde2830 0x56173bdecbb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f655c0118e0 tx=0x7f655c011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97d90e1af6f84c1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f6556ffd640 1 -- 172.21.15.59:0/4195921996 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f65501032c0 con 0x7f65601025c0 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f654bfff640 1 -- 172.21.15.59:0/3040041067 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c003970 con 0x56173bde2830 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f654bfff640 1 -- 172.21.15.59:0/3040041067 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655c00f9a0 con 0x56173bde2830 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.945+0000 7f654bfff640 1 -- 172.21.15.59:0/3040041067 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c004540 con 0x56173bde2830 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f656afda740 1 -- 172.21.15.59:0/3040041067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde2830 msgr2=0x56173bdecbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f6566781640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f654c0b2bb0 0x7f654c0b5070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f655800bc40 tx=0x7f6558005f10 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f656afda740 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde2830 0x56173bdecbb0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f655c0118e0 tx=0x7f655c011da0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f656afda740 1 -- 172.21.15.59:0/3040041067 shutdown_connections 2024-05-08T05:30:03.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f656afda740 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bded0f0 0x56173bdef4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f656afda740 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde2830 0x56173bdecbb0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f656afda740 1 --2- 172.21.15.59:0/3040041067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bde1f10 0x56173bde22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f656afda740 1 -- 172.21.15.59:0/3040041067 >> 172.21.15.59:0/3040041067 conn(0x56173bdde420 msgr2=0x56173bdde800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f656afda740 1 -- 172.21.15.59:0/3040041067 shutdown_connections 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.946+0000 7f656afda740 1 -- 172.21.15.59:0/3040041067 wait complete. 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656afda740 1 -- start start 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656afda740 1 Processor -- start 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656afda740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bde9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656afda740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bdea4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656afda740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173bdeaa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656afda740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56173bdefd20 con 0x56173bde1f10 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656afda740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56173bd04370 con 0x56173bde2830 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656afda740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56173bdefea0 con 0x56173bded0f0 2024-05-08T05:30:03.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656afda740 10 client.0 _ll_register_callbacks cb 0x56173ba58a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f6565f80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bde9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f6565f80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bde9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46620/0 (socket says 172.21.15.59:46620) 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f6565f80640 1 -- 172.21.15.59:0/1478606524 learned_addr learned my addr 172.21.15.59:0/1478606524 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.947+0000 7f656577f640 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bdea4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f6565f80640 1 -- 172.21.15.59:0/1478606524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 msgr2=0x56173bdeaa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f6565f80640 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173bdeaa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f6565f80640 1 -- 172.21.15.59:0/1478606524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 msgr2=0x56173bdea4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f6565f80640 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bdea4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f6565f80640 1 -- 172.21.15.59:0/1478606524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173bb25620 con 0x56173bde1f10 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f656577f640 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bdea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f6565f80640 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bde9fb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f655c009370 tx=0x7f655c009f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838f9d9e052dbc2a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f65497fa640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f65497fa640 1 -- 172.21.15.59:0/1478606524 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c004a40 con 0x56173bde1f10 2024-05-08T05:30:03.955 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f65497fa640 1 -- 172.21.15.59:0/1478606524 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f655c004be0 con 0x56173bde1f10 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f656afda740 1 -- 172.21.15.59:0/1478606524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f655c00f9a0 con 0x56173bde1f10 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f65497fa640 1 -- 172.21.15.59:0/1478606524 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f655c031050 con 0x56173bde1f10 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f656afda740 10 client.15456 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f656afda740 1 -- 172.21.15.59:0/1478606524 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56173bdfbb20 con 0x56173bde1f10 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.948+0000 7f65497fa640 1 -- 172.21.15.59:0/1478606524 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655c00f9a0 con 0x56173bde1f10 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f65497fa640 1 -- 172.21.15.59:0/1478606524 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f655c018070 con 0x56173bde1f10 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f656afda740 10 client.15456 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f656afda740 10 client.15456 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f656afda740 20 client.15456 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f656afda740 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56173bdfcf20 0x56173bdff310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f656afda740 4 client.15456 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f656afda740 1 -- 172.21.15.59:0/1478606524 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56173bd551e0 con 0x56173bdfcf20 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f656577f640 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56173bdfcf20 0x56173bdff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f65497fa640 10 client.15456 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.949+0000 7f656577f640 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56173bdfcf20 0x56173bdff310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cbad5f8e982b149 server_cookie=42eefd0aa492cfca in_seq=0 out_seq=0 2024-05-08T05:30:03.956 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f65497fa640 1 -- 172.21.15.59:0/1478606524 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56173bd551e0 con 0x56173bdfcf20 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f65497fa640 10 client.15456 handle_command_reply: tid=0 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f656afda740 1 client.15456 shutdown 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f656afda740 1 -- 172.21.15.59:0/1478606524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 msgr2=0x56173bde9fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f656afda740 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bde9fb0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f655c009370 tx=0x7f655c009f40 comp rx=0 tx=0).stop 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f656afda740 1 -- 172.21.15.59:0/1478606524 shutdown_connections 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f656afda740 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56173bdfcf20 0x56173bdff310 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f656afda740 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173bdeaa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f65497fa640 0 client.15456 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f656afda740 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bdea4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f656afda740 1 --2- 172.21.15.59:0/1478606524 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bde9fb0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.950+0000 7f656afda740 1 -- 172.21.15.59:0/1478606524 >> 172.21.15.59:0/1478606524 conn(0x56173bdde420 msgr2=0x56173bde4150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.951+0000 7f656afda740 1 -- 172.21.15.59:0/1478606524 shutdown_connections 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.951+0000 7f656afda740 1 -- 172.21.15.59:0/1478606524 wait complete. 2024-05-08T05:30:03.957 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.951+0000 7f656afda740 20 client.15456 trim_cache size 0 max 16384 2024-05-08T05:30:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f656afda740 1 Processor -- start 2024-05-08T05:30:03.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f656afda740 1 -- start start 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f656afda740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdea520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f656afda740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bdeaa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f656afda740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173be04ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f656afda740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65601afce0 con 0x56173bde1f10 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f656afda740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b1820 con 0x56173bde2830 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f656afda740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b03b0 con 0x56173bded0f0 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f6565f80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdea520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f6565f80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdea520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46628/0 (socket says 172.21.15.59:46628) 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f6565f80640 1 -- 172.21.15.59:0/2330269301 learned_addr learned my addr 172.21.15.59:0/2330269301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f656577f640 1 --2- 172.21.15.59:0/2330269301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bdeaa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f6566781640 1 --2- 172.21.15.59:0/2330269301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173be04ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.972 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f6565f80640 1 -- 172.21.15.59:0/2330269301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 msgr2=0x56173be04ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.966+0000 7f6565f80640 1 --2- 172.21.15.59:0/2330269301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173be04ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f6565f80640 1 -- 172.21.15.59:0/2330269301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 msgr2=0x56173bdeaa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f6565f80640 1 --2- 172.21.15.59:0/2330269301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bdeaa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f6565f80640 1 -- 172.21.15.59:0/2330269301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56173bde2660 con 0x56173bde1f10 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f6565f80640 1 --2- 172.21.15.59:0/2330269301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdea520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f655c011890 tx=0x7f655c009480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d06233e67d0812c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f654b7fe640 1 -- 172.21.15.59:0/2330269301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c03b070 con 0x56173bde1f10 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f654b7fe640 1 -- 172.21.15.59:0/2330269301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655c028070 con 0x56173bde1f10 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f654b7fe640 1 -- 172.21.15.59:0/2330269301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c003970 con 0x56173bde1f10 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f656afda740 1 -- 172.21.15.59:0/2330269301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 msgr2=0x56173bdea520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f656afda740 1 --2- 172.21.15.59:0/2330269301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdea520 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f655c011890 tx=0x7f655c009480 comp rx=0 tx=0).stop 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f656afda740 1 -- 172.21.15.59:0/2330269301 shutdown_connections 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f656afda740 1 --2- 172.21.15.59:0/2330269301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173be04ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f656afda740 1 --2- 172.21.15.59:0/2330269301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bdeaa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f656afda740 1 --2- 172.21.15.59:0/2330269301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdea520 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.967+0000 7f656afda740 1 -- 172.21.15.59:0/2330269301 >> 172.21.15.59:0/2330269301 conn(0x56173bdde420 msgr2=0x56173bdf7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 -- 172.21.15.59:0/2330269301 shutdown_connections 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 -- 172.21.15.59:0/2330269301 wait complete. 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 -- start start 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 Processor -- start 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdf5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bde6ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173bde70e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b1150 con 0x56173bde1f10 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65601b0a80 con 0x56173bde2830 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65601adca0 con 0x56173bded0f0 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656afda740 10 client.0 _ll_register_callbacks cb 0x7f655c0092a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f6565f80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdf5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f6565f80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdf5720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46632/0 (socket says 172.21.15.59:46632) 2024-05-08T05:30:03.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f6565f80640 1 -- 172.21.15.59:0/1553818662 learned_addr learned my addr 172.21.15.59:0/1553818662 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.968+0000 7f656577f640 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bde6ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f6566781640 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173bde70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f6565f80640 1 -- 172.21.15.59:0/1553818662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 msgr2=0x56173bde70e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f6565f80640 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173bde70e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f6565f80640 1 -- 172.21.15.59:0/1553818662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 msgr2=0x56173bde6ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f6565f80640 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bde6ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f6565f80640 1 -- 172.21.15.59:0/1553818662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56173bde7910 con 0x56173bde1f10 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f6565f80640 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdf5720 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f655c0118e0 tx=0x7f655c00fbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1930ef09d2779262 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f65497fa640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f65497fa640 1 -- 172.21.15.59:0/1553818662 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f655c03b070 con 0x56173bde1f10 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f65497fa640 1 -- 172.21.15.59:0/1553818662 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f655c01a440 con 0x56173bde1f10 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f656afda740 1 -- 172.21.15.59:0/1553818662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f655c028070 con 0x56173bde1f10 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f65497fa640 1 -- 172.21.15.59:0/1553818662 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f655c046050 con 0x56173bde1f10 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f656afda740 10 client.15462 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f656afda740 1 -- 172.21.15.59:0/1553818662 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56173be67700 con 0x56173bde1f10 2024-05-08T05:30:03.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.969+0000 7f65497fa640 1 -- 172.21.15.59:0/1553818662 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f655c028070 con 0x56173bde1f10 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f65497fa640 1 -- 172.21.15.59:0/1553818662 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f655c018030 con 0x56173bde1f10 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f656afda740 10 client.15462 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f656afda740 10 client.15462 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f656afda740 20 client.15462 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f656afda740 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56173be67ce0 0x56173be6a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f656afda740 4 client.15462 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f656afda740 1 -- 172.21.15.59:0/1553818662 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56173bd4f050 con 0x56173be67ce0 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f656577f640 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56173be67ce0 0x56173be6a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f656577f640 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56173be67ce0 0x56173be6a100 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=724b2ac880a0b4f0 server_cookie=151baf71c4c86442 in_seq=0 out_seq=0 2024-05-08T05:30:03.976 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.970+0000 7f65497fa640 10 client.15462 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f65497fa640 1 -- 172.21.15.59:0/1553818662 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56173bd4f050 con 0x56173be67ce0 2024-05-08T05:30:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f65497fa640 10 client.15462 handle_command_reply: tid=0 2024-05-08T05:30:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 client.15462 shutdown 2024-05-08T05:30:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 -- 172.21.15.59:0/1553818662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 msgr2=0x56173bdf5720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdf5720 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f655c0118e0 tx=0x7f655c00fbf0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 -- 172.21.15.59:0/1553818662 shutdown_connections 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56173be67ce0 0x56173be6a100 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56173bded0f0 0x56173bde70e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f65497fa640 0 client.15462 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56173bde2830 0x56173bde6ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 --2- 172.21.15.59:0/1553818662 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56173bde1f10 0x56173bdf5720 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 -- 172.21.15.59:0/1553818662 >> 172.21.15.59:0/1553818662 conn(0x56173bdde420 msgr2=0x56173bdf7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.978+0000 7f656afda740 1 -- 172.21.15.59:0/1553818662 shutdown_connections 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f656afda740 1 -- 172.21.15.59:0/1553818662 wait complete. 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f656afda740 20 client.15462 trim_cache size 0 max 16384 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:30:03.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:30:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:30:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:30:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f65681e4640 1 -- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f654c0b2bb0 msgr2=0x7f654c0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f65681e4640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f654c0b2bb0 0x7f654c0b5070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f655800bc40 tx=0x7f6558005f10 comp rx=0 tx=0).stop 2024-05-08T05:30:03.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f65681e4640 1 -- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601025c0 msgr2=0x7f656019a400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f65681e4640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601025c0 0x7f656019a400 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f655002aa00 tx=0x7f6550003b90 comp rx=0 tx=0).stop 2024-05-08T05:30:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f65681e4640 1 -- 172.21.15.59:0/4195921996 shutdown_connections 2024-05-08T05:30:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f65681e4640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f654c0b2bb0 0x7f654c0b5070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f65681e4640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6560106850 0x7f65601a4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f65681e4640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65601025c0 0x7f656019a400 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.979+0000 7f65681e4640 1 --2- 172.21.15.59:0/4195921996 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6560101c00 0x7f6560199ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.980+0000 7f65681e4640 1 -- 172.21.15.59:0/4195921996 >> 172.21.15.59:0/4195921996 conn(0x7f65600776b0 msgr2=0x7f65600ff790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.980+0000 7f65681e4640 1 -- 172.21.15.59:0/4195921996 shutdown_connections 2024-05-08T05:30:03.982 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:03.980+0000 7f65681e4640 1 -- 172.21.15.59:0/4195921996 wait complete. 2024-05-08T05:30:04.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.083+0000 7f62680a1700 1 -- 172.21.15.59:0/2036649510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6260109600 msgr2=0x7f62601099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.083+0000 7f62680a1700 1 --2- 172.21.15.59:0/2036649510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6260109600 0x7f62601099e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7f625000b0a0 tx=0x7f625000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.083+0000 7f62680a1700 1 -- 172.21.15.59:0/2036649510 shutdown_connections 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.083+0000 7f62680a1700 1 --2- 172.21.15.59:0/2036649510 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6260103eb0 0x7f6260110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.083+0000 7f62680a1700 1 --2- 172.21.15.59:0/2036649510 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6260109600 0x7f62601099e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.083+0000 7f62680a1700 1 --2- 172.21.15.59:0/2036649510 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62601034f0 0x7f6260103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.083+0000 7f62680a1700 1 -- 172.21.15.59:0/2036649510 >> 172.21.15.59:0/2036649510 conn(0x7f626007a650 msgr2=0x7f626007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.083+0000 7f62680a1700 1 -- 172.21.15.59:0/2036649510 shutdown_connections 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.084+0000 7f62680a1700 1 -- 172.21.15.59:0/2036649510 wait complete. 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.084+0000 7f62680a1700 1 Processor -- start 2024-05-08T05:30:04.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.084+0000 7f62680a1700 1 -- start start 2024-05-08T05:30:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.084+0000 7f62680a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62601034f0 0x7f626019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.084+0000 7f62680a1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6260103eb0 0x7f626019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.084+0000 7f62680a1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6260109600 0x7f6260199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f62680a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6260113650 con 0x7f62601034f0 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f62680a1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62601134d0 con 0x7f6260109600 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f62680a1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f62601137d0 con 0x7f6260103eb0 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f6265e3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62601034f0 0x7f626019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f6265e3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62601034f0 0x7f626019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46642/0 (socket says 172.21.15.59:46642) 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f6265e3d700 1 -- 172.21.15.59:0/1483258158 learned_addr learned my addr 172.21.15.59:0/1483258158 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f626563c700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6260103eb0 0x7f626019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f626663e700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6260109600 0x7f6260199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f6265e3d700 1 -- 172.21.15.59:0/1483258158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6260103eb0 msgr2=0x7f626019fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f6265e3d700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6260103eb0 0x7f626019fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f6265e3d700 1 -- 172.21.15.59:0/1483258158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6260109600 msgr2=0x7f6260199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f6265e3d700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6260109600 0x7f6260199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f6265e3d700 1 -- 172.21.15.59:0/1483258158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f626019a070 con 0x7f62601034f0 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f6265e3d700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62601034f0 0x7f626019f750 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f625000b070 tx=0x7f6250004db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:04.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f625effd700 1 -- 172.21.15.59:0/1483258158 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250021070 con 0x7f62601034f0 2024-05-08T05:30:04.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f62680a1700 1 -- 172.21.15.59:0/1483258158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f626019a300 con 0x7f62601034f0 2024-05-08T05:30:04.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f625effd700 1 -- 172.21.15.59:0/1483258158 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6250003c00 con 0x7f62601034f0 2024-05-08T05:30:04.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f625effd700 1 -- 172.21.15.59:0/1483258158 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250004880 con 0x7f62601034f0 2024-05-08T05:30:04.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.085+0000 7f62680a1700 1 -- 172.21.15.59:0/1483258158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62601a6620 con 0x7f62601034f0 2024-05-08T05:30:04.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.088+0000 7f625effd700 1 -- 172.21.15.59:0/1483258158 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f6250004020 con 0x7f62601034f0 2024-05-08T05:30:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.089+0000 7f625effd700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62480b2ba0 0x7f62480b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.089+0000 7f626563c700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62480b2ba0 0x7f62480b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.089 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.089+0000 7f625effd700 1 -- 172.21.15.59:0/1483258158 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f625000f020 con 0x7f62601034f0 2024-05-08T05:30:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.089+0000 7f6272934b80 1 Processor -- start 2024-05-08T05:30:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.089+0000 7f626563c700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62480b2ba0 0x7f62480b5060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f626007ac50 tx=0x7f625400b5b0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f6272934b80 1 -- start start 2024-05-08T05:30:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f6272934b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 0x558e608cfc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f6272934b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608d3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f6272934b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 0x558e608d4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f6272934b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558e6081d810 con 0x558e608d01b0 2024-05-08T05:30:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f6272934b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e607bc770 con 0x558e608d3ff0 2024-05-08T05:30:04.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f6272934b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558e608197d0 con 0x558e608cf890 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626563c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608d3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626563c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608d3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46646/0 (socket says 172.21.15.59:46646) 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626563c700 1 -- 172.21.15.59:0/549935124 learned_addr learned my addr 172.21.15.59:0/549935124 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f6265e3d700 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 0x558e608cfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626663e700 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 0x558e608d4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626563c700 1 -- 172.21.15.59:0/549935124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 msgr2=0x558e608cfc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626563c700 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 0x558e608cfc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626563c700 1 -- 172.21.15.59:0/549935124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 msgr2=0x558e608d4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626563c700 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 0x558e608d4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626563c700 1 -- 172.21.15.59:0/549935124 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e6076a880 con 0x558e608d01b0 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f6265e3d700 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 0x558e608cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.090+0000 7f626663e700 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 0x558e608d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.091+0000 7f626563c700 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608d3ab0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f625400fce0 tx=0x7f6254012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=158eb5b0423a2e49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:04.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.091+0000 7f6247fff700 1 -- 172.21.15.59:0/549935124 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254022070 con 0x558e608d01b0 2024-05-08T05:30:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.091+0000 7f6247fff700 1 -- 172.21.15.59:0/549935124 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6254011b80 con 0x558e608d01b0 2024-05-08T05:30:04.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.091+0000 7f6247fff700 1 -- 172.21.15.59:0/549935124 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f625401d610 con 0x558e608d01b0 2024-05-08T05:30:04.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.091+0000 7f6272934b80 1 -- 172.21.15.59:0/549935124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 msgr2=0x558e608d3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.091+0000 7f6272934b80 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608d3ab0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7f625400fce0 tx=0x7f6254012000 comp rx=0 tx=0).stop 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.092+0000 7f6272934b80 1 -- 172.21.15.59:0/549935124 shutdown_connections 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.092+0000 7f6272934b80 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 0x558e608d4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.092+0000 7f6272934b80 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 0x558e608cfc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.092+0000 7f6272934b80 1 --2- 172.21.15.59:0/549935124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608d3ab0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.092+0000 7f6272934b80 1 -- 172.21.15.59:0/549935124 >> 172.21.15.59:0/549935124 conn(0x558e607f70b0 msgr2=0x558e608c5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.092+0000 7f6272934b80 1 -- 172.21.15.59:0/549935124 shutdown_connections 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.092+0000 7f6272934b80 1 -- 172.21.15.59:0/549935124 wait complete. 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.092+0000 7f6272934b80 1 -- start start 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.092+0000 7f6272934b80 1 Processor -- start 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6272934b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608db470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6272934b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 0x558e608d8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6272934b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 0x558e608d8d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.096 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6272934b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558e608dc220 con 0x558e608cf890 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6272934b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e60783be0 con 0x558e608d01b0 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6272934b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558e608dc3a0 con 0x558e608d3ff0 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6272934b80 10 client.0 _ll_register_callbacks cb 0x558e607b58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6265e3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608db470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6265e3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608db470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46660/0 (socket says 172.21.15.59:46660) 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6265e3d700 1 -- 172.21.15.59:0/2100047967 learned_addr learned my addr 172.21.15.59:0/2100047967 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f626663e700 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 0x558e608d8d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f626563c700 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 0x558e608d8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6265e3d700 1 -- 172.21.15.59:0/2100047967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 msgr2=0x558e608d8d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6265e3d700 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 0x558e608d8d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6265e3d700 1 -- 172.21.15.59:0/2100047967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 msgr2=0x558e608d8810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6265e3d700 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 0x558e608d8810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6265e3d700 1 -- 172.21.15.59:0/2100047967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e607b9640 con 0x558e608cf890 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f6265e3d700 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608db470 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f62500aaed0 tx=0x7f62500aab00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c2e3278296a0cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f62457fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:04.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.093+0000 7f62457fa700 1 -- 172.21.15.59:0/2100047967 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500af2f0 con 0x558e608cf890 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f62457fa700 1 -- 172.21.15.59:0/2100047967 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f62500074e0 con 0x558e608cf890 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f6272934b80 1 -- 172.21.15.59:0/2100047967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6254011b80 con 0x558e608cf890 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f62457fa700 1 -- 172.21.15.59:0/2100047967 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f62500b3d10 con 0x558e608cf890 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f6272934b80 10 client.15474 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f6272934b80 1 -- 172.21.15.59:0/2100047967 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e608e69d0 con 0x558e608cf890 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f62457fa700 1 -- 172.21.15.59:0/2100047967 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6254011b80 con 0x558e608cf890 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f62457fa700 1 -- 172.21.15.59:0/2100047967 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6250011070 con 0x558e608cf890 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f6272934b80 10 client.15474 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f6272934b80 10 client.15474 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f6272934b80 20 client.15474 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f6272934b80 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x558e608e8fb0 0x558e608ea420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f6272934b80 4 client.15474 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.094+0000 7f6272934b80 1 -- 172.21.15.59:0/2100047967 --> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e60897600 con 0x558e608e8fb0 2024-05-08T05:30:04.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.095+0000 7f626563c700 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x558e608e8fb0 0x558e608ea420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.095+0000 7f626563c700 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x558e608e8fb0 0x558e608ea420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=79dbe66e407cecaf server_cookie=c57e40dcb1b2f04e in_seq=0 out_seq=0 2024-05-08T05:30:04.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.095+0000 7f62457fa700 10 client.15474 ms_handle_connect on v2:172.21.15.96:6834/2294126295 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f62457fa700 1 -- 172.21.15.59:0/2100047967 <== mds.4 v2:172.21.15.96:6834/2294126295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558e60897600 con 0x558e608e8fb0 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f62457fa700 10 client.15474 handle_command_reply: tid=0 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 client.15474 shutdown 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 -- 172.21.15.59:0/2100047967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 msgr2=0x558e608db470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608db470 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f62500aaed0 tx=0x7f62500aab00 comp rx=0 tx=0).stop 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 -- 172.21.15.59:0/2100047967 shutdown_connections 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x558e608e8fb0 0x558e608ea420 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 0x558e608d8d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f62457fa700 0 client.15474 ms_handle_reset on v2:172.21.15.96:6834/2294126295 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608db470 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 --2- 172.21.15.59:0/2100047967 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 0x558e608d8810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 -- 172.21.15.59:0/2100047967 >> 172.21.15.59:0/2100047967 conn(0x558e607f70b0 msgr2=0x558e608c06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:04.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 -- 172.21.15.59:0/2100047967 shutdown_connections 2024-05-08T05:30:04.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 1 -- 172.21.15.59:0/2100047967 wait complete. 2024-05-08T05:30:04.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.096+0000 7f6272934b80 20 client.15474 trim_cache size 0 max 16384 2024-05-08T05:30:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.117+0000 7f6272934b80 1 Processor -- start 2024-05-08T05:30:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.117+0000 7f6272934b80 1 -- start start 2024-05-08T05:30:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6272934b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608d53e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6272934b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 0x558e608d6320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6272934b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 0x558e608bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6272934b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558e608e8760 con 0x558e608cf890 2024-05-08T05:30:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6272934b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e608e8e30 con 0x558e608d01b0 2024-05-08T05:30:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6272934b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558e608e9500 con 0x558e608d3ff0 2024-05-08T05:30:04.118 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6265e3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608d53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6265e3d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608d53e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46674/0 (socket says 172.21.15.59:46674) 2024-05-08T05:30:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6265e3d700 1 -- 172.21.15.59:0/2617458006 learned_addr learned my addr 172.21.15.59:0/2617458006 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f626563c700 1 --2- 172.21.15.59:0/2617458006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 0x558e608d6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f626663e700 1 --2- 172.21.15.59:0/2617458006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 0x558e608bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6265e3d700 1 -- 172.21.15.59:0/2617458006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 msgr2=0x558e608bfe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6265e3d700 1 --2- 172.21.15.59:0/2617458006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 0x558e608bfe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6265e3d700 1 -- 172.21.15.59:0/2617458006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 msgr2=0x558e608d6320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6265e3d700 1 --2- 172.21.15.59:0/2617458006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 0x558e608d6320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6265e3d700 1 -- 172.21.15.59:0/2617458006 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e608d6d80 con 0x558e608cf890 2024-05-08T05:30:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f6265e3d700 1 --2- 172.21.15.59:0/2617458006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608d53e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f62500a81b0 tx=0x7f62500ab0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffbae5cc8fa22cb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.118+0000 7f62477fe700 1 -- 172.21.15.59:0/2617458006 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500c9070 con 0x558e608cf890 2024-05-08T05:30:04.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f62477fe700 1 -- 172.21.15.59:0/2617458006 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62500e1950 con 0x558e608cf890 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f62477fe700 1 -- 172.21.15.59:0/2617458006 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500074e0 con 0x558e608cf890 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f6272934b80 1 -- 172.21.15.59:0/2617458006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 msgr2=0x558e608d53e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f6272934b80 1 --2- 172.21.15.59:0/2617458006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608d53e0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7f62500a81b0 tx=0x7f62500ab0c0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f6272934b80 1 -- 172.21.15.59:0/2617458006 shutdown_connections 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f6272934b80 1 --2- 172.21.15.59:0/2617458006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608d3ff0 0x558e608bfe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f6272934b80 1 --2- 172.21.15.59:0/2617458006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608cf890 0x558e608d53e0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f6272934b80 1 --2- 172.21.15.59:0/2617458006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d01b0 0x558e608d6320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f6272934b80 1 -- 172.21.15.59:0/2617458006 >> 172.21.15.59:0/2617458006 conn(0x558e607f70b0 msgr2=0x558e608d5c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f6272934b80 1 -- 172.21.15.59:0/2617458006 shutdown_connections 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.119+0000 7f6272934b80 1 -- 172.21.15.59:0/2617458006 wait complete. 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6272934b80 1 -- start start 2024-05-08T05:30:04.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6272934b80 1 Processor -- start 2024-05-08T05:30:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6272934b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 0x558e608e4a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6272934b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608e4f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6272934b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 0x558e608e54b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6272934b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558e608ea2a0 con 0x558e608d01b0 2024-05-08T05:30:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6272934b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558e608e9bd0 con 0x558e608d3ff0 2024-05-08T05:30:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6272934b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558e608ea970 con 0x558e608cf890 2024-05-08T05:30:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6272934b80 10 client.0 _ll_register_callbacks cb 0x558e607b58d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f626563c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608e4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f626563c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608e4f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46684/0 (socket says 172.21.15.59:46684) 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f626563c700 1 -- 172.21.15.59:0/364244123 learned_addr learned my addr 172.21.15.59:0/364244123 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.120+0000 7f6265e3d700 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 0x558e608e4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f626663e700 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 0x558e608e54b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f626563c700 1 -- 172.21.15.59:0/364244123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 msgr2=0x558e608e4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f626563c700 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 0x558e608e4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f626563c700 1 -- 172.21.15.59:0/364244123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 msgr2=0x558e608e54b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f626563c700 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 0x558e608e54b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f626563c700 1 -- 172.21.15.59:0/364244123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e608f1fa0 con 0x558e608d01b0 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f626563c700 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608e4f70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f625401ee40 tx=0x7f6254011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5ae0fe136816c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f62457fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f62457fa700 1 -- 172.21.15.59:0/364244123 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6254036020 con 0x558e608d01b0 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f6272934b80 1 -- 172.21.15.59:0/364244123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f62500e1950 con 0x558e608d01b0 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f62457fa700 1 -- 172.21.15.59:0/364244123 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f625401e860 con 0x558e608d01b0 2024-05-08T05:30:04.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f62457fa700 1 -- 172.21.15.59:0/364244123 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f6254041050 con 0x558e608d01b0 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f6272934b80 10 client.15480 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.121+0000 7f6272934b80 1 -- 172.21.15.59:0/364244123 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558e609548c0 con 0x558e608d01b0 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.122+0000 7f62457fa700 1 -- 172.21.15.59:0/364244123 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f62500e1950 con 0x558e608d01b0 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.122+0000 7f62457fa700 1 -- 172.21.15.59:0/364244123 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f6254031070 con 0x558e608d01b0 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.122+0000 7f6272934b80 10 client.15480 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.122+0000 7f6272934b80 10 client.15480 resolve_mds: resolved name 'e' to daemon mds.e 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.122+0000 7f6272934b80 20 client.15480 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.122+0000 7f6272934b80 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x558e60959140 0x558e60957e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.122+0000 7f6272934b80 4 client.15480 mds_command: new command op to 24419 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.122+0000 7f6272934b80 1 -- 172.21.15.59:0/364244123 --> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] -- command(tid 0: {"prefix": "status"}) -- 0x558e6082b1f0 con 0x558e60959140 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.122+0000 7f6265e3d700 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x558e60959140 0x558e60957e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6265e3d700 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x558e60959140 0x558e60957e50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c76c1814f595121e server_cookie=84df6cef3d561dec in_seq=0 out_seq=0 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f62457fa700 10 client.15480 ms_handle_connect on v2:172.21.15.96:6834/2294126295 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f62457fa700 1 -- 172.21.15.59:0/364244123 <== mds.4 v2:172.21.15.96:6834/2294126295 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x558e6082b1f0 con 0x558e60959140 2024-05-08T05:30:04.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f62457fa700 10 client.15480 handle_command_reply: tid=0 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 client.15480 shutdown 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 -- 172.21.15.59:0/364244123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 msgr2=0x558e608e4f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608e4f70 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7f625401ee40 tx=0x7f6254011d90 comp rx=0 tx=0).stop 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 -- 172.21.15.59:0/364244123 shutdown_connections 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.96:6834/2294126295,v1:172.21.15.96:6835/2294126295] conn(0x558e60959140 0x558e60957e50 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558e608d3ff0 0x558e608e54b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558e608cf890 0x558e608e4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 --2- 172.21.15.59:0/364244123 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558e608d01b0 0x558e608e4f70 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f62457fa700 0 client.15480 ms_handle_reset on v2:172.21.15.96:6834/2294126295 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 -- 172.21.15.59:0/364244123 >> 172.21.15.59:0/364244123 conn(0x558e607f70b0 msgr2=0x558e608c06d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 -- 172.21.15.59:0/364244123 shutdown_connections 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 1 -- 172.21.15.59:0/364244123 wait complete. 2024-05-08T05:30:04.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.123+0000 7f6272934b80 20 client.15480 trim_cache size 0 max 16384 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 4, 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24419, 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 216.71486307699999, 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 30, 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 90, 2024-05-08T05:30:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 313.81946792000002 2024-05-08T05:30:04.131 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:30:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 -- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62480b2ba0 msgr2=0x7f62480b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62480b2ba0 0x7f62480b5060 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f626007ac50 tx=0x7f625400b5b0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 -- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62601034f0 msgr2=0x7f626019f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62601034f0 0x7f626019f750 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f625000b070 tx=0x7f6250004db0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 -- 172.21.15.59:0/1483258158 shutdown_connections 2024-05-08T05:30:04.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62480b2ba0 0x7f62480b5060 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6260109600 0x7f6260199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62601034f0 0x7f626019f750 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 --2- 172.21.15.59:0/1483258158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6260103eb0 0x7f626019fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:04.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 -- 172.21.15.59:0/1483258158 >> 172.21.15.59:0/1483258158 conn(0x7f626007a650 msgr2=0x7f62601021e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:04.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 -- 172.21.15.59:0/1483258158 shutdown_connections 2024-05-08T05:30:04.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:04.124+0000 7f62680a1700 1 -- 172.21.15.59:0/1483258158 wait complete. 2024-05-08T05:30:04.812 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.c status 2024-05-08T05:30:05.068 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:30:05.451 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:05 smithi059 ceph-mon[30304]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-05-08T05:30:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:05 smithi143 ceph-mon[34337]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-05-08T05:30:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:05 smithi096 ceph-mon[34253]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 2.6 KiB/s wr, 13 op/s 2024-05-08T05:30:06.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.732+0000 7f7c5ee43700 1 -- 172.21.15.59:0/3917211657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 msgr2=0x7f7c58110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:06.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.732+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/3917211657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 0x7f7c58110330 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f7c5000b0a0 tx=0x7f7c5000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.732+0000 7f7c5ee43700 1 -- 172.21.15.59:0/3917211657 shutdown_connections 2024-05-08T05:30:06.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.732+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/3917211657 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 0x7f7c58110330 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.732+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/3917211657 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c58109510 0x7f7c581098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.732+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/3917211657 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5806bbd0 0x7f7c5806c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.736 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.732+0000 7f7c5ee43700 1 -- 172.21.15.59:0/3917211657 >> 172.21.15.59:0/3917211657 conn(0x7f7c58079350 msgr2=0x7f7c58079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.732+0000 7f7c5ee43700 1 -- 172.21.15.59:0/3917211657 shutdown_connections 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.732+0000 7f7c5ee43700 1 -- 172.21.15.59:0/3917211657 wait complete. 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.733+0000 7f7c5ee43700 1 Processor -- start 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.733+0000 7f7c5ee43700 1 -- start start 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.733+0000 7f7c5ee43700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5806bbd0 0x7f7c5819c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.733+0000 7f7c5ee43700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 0x7f7c5819c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.733+0000 7f7c5ee43700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c58109510 0x7f7c581a0620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c5ee43700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58102af0 con 0x7f7c5806c590 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c5ee43700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58102970 con 0x7f7c5806bbd0 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c5ee43700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7c58102c70 con 0x7f7c58109510 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 0x7f7c5819c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 0x7f7c5819c7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46712/0 (socket says 172.21.15.59:46712) 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c4ffff700 1 -- 172.21.15.59:0/4041492172 learned_addr learned my addr 172.21.15.59:0/4041492172 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:06.737 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5806bbd0 0x7f7c5819c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c5d3e0700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c58109510 0x7f7c581a0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c4ffff700 1 -- 172.21.15.59:0/4041492172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c58109510 msgr2=0x7f7c581a0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c58109510 0x7f7c581a0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c4ffff700 1 -- 172.21.15.59:0/4041492172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5806bbd0 msgr2=0x7f7c5819c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5806bbd0 0x7f7c5819c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c4ffff700 1 -- 172.21.15.59:0/4041492172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c581a0d30 con 0x7f7c5806c590 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.734+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 0x7f7c5819c7b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800b530 tx=0x7f7c4800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.735+0000 7f7c4dffb700 1 -- 172.21.15.59:0/4041492172 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48015020 con 0x7f7c5806c590 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.735+0000 7f7c5ee43700 1 -- 172.21.15.59:0/4041492172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c581a1020 con 0x7f7c5806c590 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.735+0000 7f7c4dffb700 1 -- 172.21.15.59:0/4041492172 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c48011ce0 con 0x7f7c5806c590 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.735+0000 7f7c4dffb700 1 -- 172.21.15.59:0/4041492172 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c48010dd0 con 0x7f7c5806c590 2024-05-08T05:30:06.738 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.735+0000 7f7c5ee43700 1 -- 172.21.15.59:0/4041492172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c581a88d0 con 0x7f7c5806c590 2024-05-08T05:30:06.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.737+0000 7f7c4dffb700 1 -- 172.21.15.59:0/4041492172 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f7c48018020 con 0x7f7c5806c590 2024-05-08T05:30:06.739 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.738+0000 7f7c4dffb700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7c400b2c20 0x7f7c400b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.738+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7c400b2c20 0x7f7c400b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.738+0000 7f7c4dffb700 1 -- 172.21.15.59:0/4041492172 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7c480e2140 con 0x7f7c5806c590 2024-05-08T05:30:06.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7c400b2c20 0x7f7c400b50e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400ac50 tx=0x7f7c5400a380 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:06.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c696d6b80 1 Processor -- start 2024-05-08T05:30:06.740 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c696d6b80 1 -- start start 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 0x555c995bf860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995c36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c99511810 con 0x555c995bfda0 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555c994b0770 con 0x555c995bf480 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555c9950d7d0 con 0x555c995c3c30 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995c36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995c36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46726/0 (socket says 172.21.15.59:46726) 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c4ffff700 1 -- 172.21.15.59:0/1602258944 learned_addr learned my addr 172.21.15.59:0/1602258944 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.739+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 0x555c995bf860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c5d3e0700 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c5cbdf700 1 -- 172.21.15.59:0/1602258944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 msgr2=0x555c995c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c5cbdf700 1 -- 172.21.15.59:0/1602258944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 msgr2=0x555c995c36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995c36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c5cbdf700 1 -- 172.21.15.59:0/1602258944 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c9945e880 con 0x555c995bf480 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 0x555c995bf860 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400fa30 tx=0x7f7c54012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18ab16c62e1c83d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c3effd700 1 -- 172.21.15.59:0/1602258944 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54016070 con 0x555c995bf480 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c3effd700 1 -- 172.21.15.59:0/1602258944 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c54014070 con 0x555c995bf480 2024-05-08T05:30:06.743 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.740+0000 7f7c3effd700 1 -- 172.21.15.59:0/1602258944 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54010d90 con 0x555c995bf480 2024-05-08T05:30:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1602258944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 msgr2=0x555c995bf860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 0x555c995bf860 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400fa30 tx=0x7f7c54012000 comp rx=0 tx=0).stop 2024-05-08T05:30:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1602258944 shutdown_connections 2024-05-08T05:30:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 0x555c995bf860 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1602258944 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995c36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1602258944 >> 172.21.15.59:0/1602258944 conn(0x555c994eb0b0 msgr2=0x555c995b9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1602258944 shutdown_connections 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1602258944 wait complete. 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.741+0000 7f7c696d6b80 1 -- start start 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c696d6b80 1 Processor -- start 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995bfda0 0x555c995c8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995c3c30 0x555c995c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c995cbe40 con 0x555c995bf480 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555c99477be0 con 0x555c995c3c30 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555c995cbfc0 con 0x555c995bfda0 2024-05-08T05:30:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c696d6b80 10 client.0 _ll_register_callbacks cb 0x555c994a98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c5cbdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c5cbdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46736/0 (socket says 172.21.15.59:46736) 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c5cbdf700 1 -- 172.21.15.59:0/3861442999 learned_addr learned my addr 172.21.15.59:0/3861442999 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c5d3e0700 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995c3c30 0x555c995c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.742+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995bfda0 0x555c995c8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c5d3e0700 1 -- 172.21.15.59:0/3861442999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995bfda0 msgr2=0x555c995c8370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c5d3e0700 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995bfda0 0x555c995c8370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c5d3e0700 1 -- 172.21.15.59:0/3861442999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 msgr2=0x555c995cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c5d3e0700 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c5d3e0700 1 -- 172.21.15.59:0/3861442999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c994ad640 con 0x555c995c3c30 2024-05-08T05:30:06.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c5d3e0700 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995c3c30 0x555c995c88b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009fd0 tx=0x7f7c50003c30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f4bd7c53356877d1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c37fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c37fff700 1 -- 172.21.15.59:0/3861442999 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c500076e0 con 0x555c995c3c30 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3861442999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c54014070 con 0x555c995c3c30 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c37fff700 1 -- 172.21.15.59:0/3861442999 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7c50007880 con 0x555c995c3c30 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c37fff700 1 -- 172.21.15.59:0/3861442999 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c5002f050 con 0x555c995c3c30 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c696d6b80 10 client.25157 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3861442999 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c995d6680 con 0x555c995c3c30 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.743+0000 7f7c37fff700 1 -- 172.21.15.59:0/3861442999 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c54014070 con 0x555c995c3c30 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.744+0000 7f7c37fff700 1 -- 172.21.15.59:0/3861442999 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7c50011070 con 0x555c995c3c30 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.744+0000 7f7c696d6b80 10 client.25157 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.744+0000 7f7c696d6b80 10 client.25157 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:30:06.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.744+0000 7f7c696d6b80 20 client.25157 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.744+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x555c995d8c10 0x555c995da080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.744+0000 7f7c696d6b80 4 client.25157 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.744+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3861442999 --> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555c9958b600 con 0x555c995d8c10 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.744+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x555c995d8c10 0x555c995da080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.745+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x555c995d8c10 0x555c995da080 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=484fafc12df9e18f server_cookie=c4d51ba472c28571 in_seq=0 out_seq=0 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.745+0000 7f7c37fff700 10 client.25157 ms_handle_connect on v2:172.21.15.143:6836/555225813 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.745+0000 7f7c37fff700 1 -- 172.21.15.59:0/3861442999 <== mds.3 v2:172.21.15.143:6836/555225813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555c9958b600 con 0x555c995d8c10 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.745+0000 7f7c37fff700 10 client.25157 handle_command_reply: tid=0 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.745+0000 7f7c696d6b80 1 client.25157 shutdown 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3861442999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995c3c30 msgr2=0x555c995c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995c3c30 0x555c995c88b0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f7c50009fd0 tx=0x7f7c50003c30 comp rx=0 tx=0).stop 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3861442999 shutdown_connections 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995c3c30 0x555c995c88b0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x555c995d8c10 0x555c995da080 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3861442999 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995bfda0 0x555c995c8370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3861442999 >> 172.21.15.59:0/3861442999 conn(0x555c994eb0b0 msgr2=0x555c995b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c37fff700 0 client.25157 ms_handle_reset on v2:172.21.15.143:6836/555225813 2024-05-08T05:30:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3861442999 shutdown_connections 2024-05-08T05:30:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3861442999 wait complete. 2024-05-08T05:30:06.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.746+0000 7f7c696d6b80 20 client.25157 trim_cache size 0 max 16384 2024-05-08T05:30:06.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.767+0000 7f7c696d6b80 1 Processor -- start 2024-05-08T05:30:06.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.767+0000 7f7c696d6b80 1 -- start start 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.767+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995c15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.767+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bfda0 0x555c995c1ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.767+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995b3e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.767+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c995d83c0 con 0x555c995bf480 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.767+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555c995d8a90 con 0x555c995bfda0 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.767+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555c995d9160 con 0x555c995c3c30 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5cbdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995c15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5cbdf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995c15a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46748/0 (socket says 172.21.15.59:46748) 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5cbdf700 1 -- 172.21.15.59:0/3313675494 learned_addr learned my addr 172.21.15.59:0/3313675494 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bfda0 0x555c995c1ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5d3e0700 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995b3e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5cbdf700 1 -- 172.21.15.59:0/3313675494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 msgr2=0x555c995b3e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:06.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995b3e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5cbdf700 1 -- 172.21.15.59:0/3313675494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bfda0 msgr2=0x555c995c1ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bfda0 0x555c995c1ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5cbdf700 1 -- 172.21.15.59:0/3313675494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555c995d5290 con 0x555c995bf480 2024-05-08T05:30:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bfda0 0x555c995c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995c15a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400a4a0 tx=0x7f7c54016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c54ac1f048dfcac server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c3e7fc700 1 -- 172.21.15.59:0/3313675494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54010c00 con 0x555c995bf480 2024-05-08T05:30:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c3e7fc700 1 -- 172.21.15.59:0/3313675494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c54014050 con 0x555c995bf480 2024-05-08T05:30:06.771 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.768+0000 7f7c3e7fc700 1 -- 172.21.15.59:0/3313675494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c54031690 con 0x555c995bf480 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3313675494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 msgr2=0x555c995c15a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995c15a0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400a4a0 tx=0x7f7c54016000 comp rx=0 tx=0).stop 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3313675494 shutdown_connections 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995b3e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bf480 0x555c995c15a0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/3313675494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bfda0 0x555c995c1ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3313675494 >> 172.21.15.59:0/3313675494 conn(0x555c994eb0b0 msgr2=0x555c995c6cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3313675494 shutdown_connections 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 -- 172.21.15.59:0/3313675494 wait complete. 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 -- start start 2024-05-08T05:30:06.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.769+0000 7f7c696d6b80 1 Processor -- start 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 0x555c995dd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995dd7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c696d6b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995ddd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555c995d9f00 con 0x555c995bfda0 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555c995d9830 con 0x555c995bf480 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c696d6b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555c995da5d0 con 0x555c995c3c30 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c696d6b80 10 client.0 _ll_register_callbacks cb 0x555c994a98d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995dd7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c4ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995dd7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46762/0 (socket says 172.21.15.59:46762) 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c4ffff700 1 -- 172.21.15.59:0/1535840331 learned_addr learned my addr 172.21.15.59:0/1535840331 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 0x555c995dd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c5d3e0700 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995ddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c4ffff700 1 -- 172.21.15.59:0/1535840331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 msgr2=0x555c995ddd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995ddd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c4ffff700 1 -- 172.21.15.59:0/1535840331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 msgr2=0x555c995dd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 0x555c995dd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c4ffff700 1 -- 172.21.15.59:0/1535840331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555c995e1cf0 con 0x555c995bfda0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.770+0000 7f7c4ffff700 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995dd7c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7c48014040 tx=0x7f7c48004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b82bfc7fbef77be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c37fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c37fff700 1 -- 172.21.15.59:0/1535840331 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c480c8060 con 0x555c995bfda0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c37fff700 1 -- 172.21.15.59:0/1535840331 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7c48011a40 con 0x555c995bfda0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1535840331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c54014050 con 0x555c995bfda0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c37fff700 1 -- 172.21.15.59:0/1535840331 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f7c480a8f80 con 0x555c995bfda0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c696d6b80 10 client.15498 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1535840331 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555c99644630 con 0x555c995bfda0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c37fff700 1 -- 172.21.15.59:0/1535840331 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c54014050 con 0x555c995bfda0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c37fff700 1 -- 172.21.15.59:0/1535840331 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f7c480a48f0 con 0x555c995bfda0 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c696d6b80 10 client.15498 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c696d6b80 10 client.15498 resolve_mds: resolved name 'c' to daemon mds.c 2024-05-08T05:30:06.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c696d6b80 20 client.15498 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x555c99649230 0x555c99647f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c696d6b80 4 client.15498 mds_command: new command op to 24421 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.771+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1535840331 --> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] -- command(tid 0: {"prefix": "status"}) -- 0x555c9951f1f0 con 0x555c99649230 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x555c99649230 0x555c99647f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c5cbdf700 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x555c99649230 0x555c99647f40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=8e43a3c7bc5ecba3 server_cookie=f21f1bdfe845793e in_seq=0 out_seq=0 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c37fff700 10 client.15498 ms_handle_connect on v2:172.21.15.143:6836/555225813 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c37fff700 1 -- 172.21.15.59:0/1535840331 <== mds.3 v2:172.21.15.143:6836/555225813 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x555c9951f1f0 con 0x555c99649230 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c37fff700 10 client.15498 handle_command_reply: tid=0 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c696d6b80 1 client.15498 shutdown 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1535840331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 msgr2=0x555c995dd7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995dd7c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7c48014040 tx=0x7f7c48004750 comp rx=0 tx=0).stop 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1535840331 shutdown_connections 2024-05-08T05:30:06.778 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.772+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.143:6836/555225813,v1:172.21.15.143:6837/555225813] conn(0x555c99649230 0x555c99647f40 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555c995c3c30 0x555c995ddd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555c995bf480 0x555c995dd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c696d6b80 1 --2- 172.21.15.59:0/1535840331 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555c995bfda0 0x555c995dd7c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1535840331 >> 172.21.15.59:0/1535840331 conn(0x555c994eb0b0 msgr2=0x555c995b46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c37fff700 0 client.15498 ms_handle_reset on v2:172.21.15.143:6836/555225813 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1535840331 shutdown_connections 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c696d6b80 1 -- 172.21.15.59:0/1535840331 wait complete. 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c696d6b80 20 client.15498 trim_cache size 0 max 16384 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:30:06.779 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 3, 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24421, 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 220.373341348, 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 30, 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 90, 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 310.88975171300001 2024-05-08T05:30:06.780 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:30:06.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 -- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7c400b2c20 msgr2=0x7f7c400b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:06.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7c400b2c20 0x7f7c400b50e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f7c5400ac50 tx=0x7f7c5400a380 comp rx=0 tx=0).stop 2024-05-08T05:30:06.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 -- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 msgr2=0x7f7c5819c7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:06.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 0x7f7c5819c7b0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f7c4800b530 tx=0x7f7c4800b840 comp rx=0 tx=0).stop 2024-05-08T05:30:06.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 -- 172.21.15.59:0/4041492172 shutdown_connections 2024-05-08T05:30:06.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7c400b2c20 0x7f7c400b50e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7c58109510 0x7f7c581a0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7c5806bbd0 0x7f7c5819c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 --2- 172.21.15.59:0/4041492172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7c5806c590 0x7f7c5819c7b0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:06.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.773+0000 7f7c5ee43700 1 -- 172.21.15.59:0/4041492172 >> 172.21.15.59:0/4041492172 conn(0x7f7c58079350 msgr2=0x7f7c581013a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:06.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.774+0000 7f7c5ee43700 1 -- 172.21.15.59:0/4041492172 shutdown_connections 2024-05-08T05:30:06.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:06.774+0000 7f7c5ee43700 1 -- 172.21.15.59:0/4041492172 wait complete. 2024-05-08T05:30:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:07 smithi143 ceph-mon[34337]: pgmap v19: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 35 op/s 2024-05-08T05:30:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:07 smithi096 ceph-mon[34253]: pgmap v19: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 35 op/s 2024-05-08T05:30:07.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:07 smithi059 ceph-mon[30304]: pgmap v19: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 35 op/s 2024-05-08T05:30:07.644 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.i status 2024-05-08T05:30:07.905 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:30:09.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:09 smithi059 ceph-mon[30304]: pgmap v20: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-05-08T05:30:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:09 smithi143 ceph-mon[34337]: pgmap v20: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-05-08T05:30:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:09 smithi096 ceph-mon[34253]: pgmap v20: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 25 KiB/s wr, 26 op/s 2024-05-08T05:30:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.611+0000 7f134407b700 1 -- 172.21.15.59:0/3089813006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c102d60 msgr2=0x7f133c10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.611+0000 7f134407b700 1 --2- 172.21.15.59:0/3089813006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c102d60 0x7f133c10f250 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f133400b0a0 tx=0x7f133400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.611+0000 7f134407b700 1 -- 172.21.15.59:0/3089813006 shutdown_connections 2024-05-08T05:30:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.611+0000 7f134407b700 1 --2- 172.21.15.59:0/3089813006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c102d60 0x7f133c10f250 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.611+0000 7f134407b700 1 --2- 172.21.15.59:0/3089813006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f133c108410 0x7f133c1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.611+0000 7f134407b700 1 --2- 172.21.15.59:0/3089813006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c1023a0 0x7f133c102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.611+0000 7f134407b700 1 -- 172.21.15.59:0/3089813006 >> 172.21.15.59:0/3089813006 conn(0x7f133c079350 msgr2=0x7f133c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.611+0000 7f134407b700 1 -- 172.21.15.59:0/3089813006 shutdown_connections 2024-05-08T05:30:09.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.611+0000 7f134407b700 1 -- 172.21.15.59:0/3089813006 wait complete. 2024-05-08T05:30:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f134407b700 1 Processor -- start 2024-05-08T05:30:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f134407b700 1 -- start start 2024-05-08T05:30:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f134407b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c1023a0 0x7f133c104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f134407b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f133c102d60 0x7f133c1045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f134407b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c108410 0x7f133c104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f134407b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f133c1125b0 con 0x7f133c108410 2024-05-08T05:30:09.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f134407b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f133c112430 con 0x7f133c102d60 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f134407b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f133c112730 con 0x7f133c1023a0 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f1342618700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c108410 0x7f133c104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f1342618700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c108410 0x7f133c104b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55414/0 (socket says 172.21.15.59:55414) 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f1342618700 1 -- 172.21.15.59:0/1564460161 learned_addr learned my addr 172.21.15.59:0/1564460161 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f1341616700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f133c102d60 0x7f133c1045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.612+0000 7f1341e17700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c1023a0 0x7f133c104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f1342618700 1 -- 172.21.15.59:0/1564460161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c1023a0 msgr2=0x7f133c104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f1342618700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c1023a0 0x7f133c104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f1342618700 1 -- 172.21.15.59:0/1564460161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f133c102d60 msgr2=0x7f133c1045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f1342618700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f133c102d60 0x7f133c1045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f1342618700 1 -- 172.21.15.59:0/1564460161 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f133c1a80e0 con 0x7f133c108410 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f1342618700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c108410 0x7f133c104b00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f133400b830 tx=0x7f1334004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f1332ffd700 1 -- 172.21.15.59:0/1564460161 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334021070 con 0x7f133c108410 2024-05-08T05:30:09.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f1332ffd700 1 -- 172.21.15.59:0/1564460161 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1334007e30 con 0x7f133c108410 2024-05-08T05:30:09.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f134407b700 1 -- 172.21.15.59:0/1564460161 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f133c1a8370 con 0x7f133c108410 2024-05-08T05:30:09.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f1332ffd700 1 -- 172.21.15.59:0/1564460161 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1334004700 con 0x7f133c108410 2024-05-08T05:30:09.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.613+0000 7f134407b700 1 -- 172.21.15.59:0/1564460161 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f133c1a8870 con 0x7f133c108410 2024-05-08T05:30:09.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.616+0000 7f1332ffd700 1 -- 172.21.15.59:0/1564460161 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f1334007980 con 0x7f133c108410 2024-05-08T05:30:09.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.616+0000 7f1332ffd700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13280b2c20 0x7f13280b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.616+0000 7f1341e17700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13280b2c20 0x7f13280b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.618 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.616+0000 7f1332ffd700 1 -- 172.21.15.59:0/1564460161 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f133400f020 con 0x7f133c108410 2024-05-08T05:30:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.617+0000 7f1341e17700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13280b2c20 0x7f13280b50e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1338009a10 tx=0x7f1338009190 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.617+0000 7f134e90eb80 1 Processor -- start 2024-05-08T05:30:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.617+0000 7f134e90eb80 1 -- start start 2024-05-08T05:30:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 0x5555b719d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71a16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b71a1c30 0x5555b71a20b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f134e90eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555b70ef810 con 0x5555b719dda0 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f134e90eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5555b708e770 con 0x5555b71a1c30 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f134e90eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5555b70eb7d0 con 0x5555b719d480 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341616700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71a16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341616700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71a16a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55422/0 (socket says 172.21.15.59:55422) 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341616700 1 -- 172.21.15.59:0/4279879543 learned_addr learned my addr 172.21.15.59:0/4279879543 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1342618700 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b71a1c30 0x5555b71a20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341e17700 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 0x5555b719d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341616700 1 -- 172.21.15.59:0/4279879543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 msgr2=0x5555b719d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341616700 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 0x5555b719d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341616700 1 -- 172.21.15.59:0/4279879543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b71a1c30 msgr2=0x5555b71a20b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341616700 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b71a1c30 0x5555b71a20b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341616700 1 -- 172.21.15.59:0/4279879543 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b703c880 con 0x5555b719dda0 2024-05-08T05:30:09.620 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1342618700 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b71a1c30 0x5555b71a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1341616700 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71a16a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f132c00bcf0 tx=0x7f132c0103d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=307d1be909730163 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1327fff700 1 -- 172.21.15.59:0/4279879543 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c010dc0 con 0x5555b719dda0 2024-05-08T05:30:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1327fff700 1 -- 172.21.15.59:0/4279879543 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f132c0044e0 con 0x5555b719dda0 2024-05-08T05:30:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.618+0000 7f1327fff700 1 -- 172.21.15.59:0/4279879543 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c005280 con 0x5555b719dda0 2024-05-08T05:30:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.619+0000 7f134e90eb80 1 -- 172.21.15.59:0/4279879543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 msgr2=0x5555b71a16a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.619+0000 7f134e90eb80 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71a16a0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f132c00bcf0 tx=0x7f132c0103d0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.619+0000 7f134e90eb80 1 -- 172.21.15.59:0/4279879543 shutdown_connections 2024-05-08T05:30:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.619+0000 7f134e90eb80 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b71a1c30 0x5555b71a20b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.624 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.619+0000 7f134e90eb80 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 0x5555b719d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.619+0000 7f134e90eb80 1 --2- 172.21.15.59:0/4279879543 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71a16a0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.619+0000 7f134e90eb80 1 -- 172.21.15.59:0/4279879543 >> 172.21.15.59:0/4279879543 conn(0x5555b70c90b0 msgr2=0x5555b7197c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.619+0000 7f134e90eb80 1 -- 172.21.15.59:0/4279879543 shutdown_connections 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.619+0000 7f134e90eb80 1 -- 172.21.15.59:0/4279879543 wait complete. 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f134e90eb80 1 -- start start 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f134e90eb80 1 Processor -- start 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 0x5555b71a8ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719dda0 0x5555b71a6290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b71a67d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f134e90eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555b71a9e40 con 0x5555b71a1c30 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f134e90eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5555b7055be0 con 0x5555b719d480 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f134e90eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5555b71a9fc0 con 0x5555b719dda0 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f134e90eb80 10 client.0 _ll_register_callbacks cb 0x5555b70878d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f1342618700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b71a67d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.625 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f1342618700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b71a67d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55436/0 (socket says 172.21.15.59:55436) 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.620+0000 7f1342618700 1 -- 172.21.15.59:0/2194782575 learned_addr learned my addr 172.21.15.59:0/2194782575 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f1341e17700 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 0x5555b71a8ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f1341616700 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719dda0 0x5555b71a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f1342618700 1 -- 172.21.15.59:0/2194782575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719dda0 msgr2=0x5555b71a6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f1342618700 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719dda0 0x5555b71a6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f1342618700 1 -- 172.21.15.59:0/2194782575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 msgr2=0x5555b71a8ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f1342618700 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 0x5555b71a8ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f1342618700 1 -- 172.21.15.59:0/2194782575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b708b640 con 0x5555b71a1c30 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f1342618700 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b71a67d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1334009fd0 tx=0x7f13340aa5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895d0907d085cb1c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f13257fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f13257fa700 1 -- 172.21.15.59:0/2194782575 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13340ab550 con 0x5555b71a1c30 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f13257fa700 1 -- 172.21.15.59:0/2194782575 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f13340ab2c0 con 0x5555b71a1c30 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f134e90eb80 1 -- 172.21.15.59:0/2194782575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f132c0044e0 con 0x5555b71a1c30 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f13257fa700 1 -- 172.21.15.59:0/2194782575 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f13340aae50 con 0x5555b71a1c30 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f134e90eb80 10 client.15510 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:09.626 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.621+0000 7f134e90eb80 1 -- 172.21.15.59:0/2194782575 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555b71b4520 con 0x5555b71a1c30 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.622+0000 7f13257fa700 1 -- 172.21.15.59:0/2194782575 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f132c0044e0 con 0x5555b71a1c30 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.622+0000 7f13257fa700 1 -- 172.21.15.59:0/2194782575 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1334011070 con 0x5555b71a1c30 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.622+0000 7f134e90eb80 10 client.15510 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.622+0000 7f134e90eb80 10 client.15510 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.622+0000 7f134e90eb80 20 client.15510 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.622+0000 7f134e90eb80 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5555b71b6ca0 0x5555b71b8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.622+0000 7f134e90eb80 4 client.15510 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.622+0000 7f134e90eb80 1 -- 172.21.15.59:0/2194782575 --> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5555b7169600 con 0x5555b71b6ca0 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.622+0000 7f1341e17700 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5555b71b6ca0 0x5555b71b8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.623+0000 7f1341e17700 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5555b71b6ca0 0x5555b71b8110 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=221dc8271a087691 server_cookie=1bdfb37c8782f5bb in_seq=0 out_seq=0 2024-05-08T05:30:09.627 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.623+0000 7f13257fa700 10 client.15510 ms_handle_connect on v2:172.21.15.143:6838/2166229809 2024-05-08T05:30:09.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.623+0000 7f13257fa700 1 -- 172.21.15.59:0/2194782575 <== mds.2 v2:172.21.15.143:6838/2166229809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5555b7169600 con 0x5555b71b6ca0 2024-05-08T05:30:09.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.623+0000 7f13257fa700 10 client.15510 handle_command_reply: tid=0 2024-05-08T05:30:09.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.623+0000 7f134e90eb80 1 client.15510 shutdown 2024-05-08T05:30:09.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.623+0000 7f134e90eb80 1 -- 172.21.15.59:0/2194782575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 msgr2=0x5555b71a67d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:09.628 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.623+0000 7f134e90eb80 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b71a67d0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f1334009fd0 tx=0x7f13340aa5c0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f134e90eb80 1 -- 172.21.15.59:0/2194782575 shutdown_connections 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f134e90eb80 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5555b71b6ca0 0x5555b71b8110 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f134e90eb80 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b71a67d0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f134e90eb80 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 0x5555b71a8ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f13257fa700 0 client.15510 ms_handle_reset on v2:172.21.15.143:6838/2166229809 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f134e90eb80 1 --2- 172.21.15.59:0/2194782575 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719dda0 0x5555b71a6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f134e90eb80 1 -- 172.21.15.59:0/2194782575 >> 172.21.15.59:0/2194782575 conn(0x5555b70c90b0 msgr2=0x5555b71926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f134e90eb80 1 -- 172.21.15.59:0/2194782575 shutdown_connections 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f134e90eb80 1 -- 172.21.15.59:0/2194782575 wait complete. 2024-05-08T05:30:09.629 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.624+0000 7f134e90eb80 20 client.15510 trim_cache size 0 max 16384 2024-05-08T05:30:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f134e90eb80 1 Processor -- start 2024-05-08T05:30:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f134e90eb80 1 -- start start 2024-05-08T05:30:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 0x5555b71a5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.647 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719dda0 0x5555b7192550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b719f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f134e90eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555b71b6450 con 0x5555b71a1c30 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f134e90eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5555b71b6b20 con 0x5555b719dda0 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f134e90eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5555b71b71f0 con 0x5555b719d480 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f1342618700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b719f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f1342618700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b719f5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55446/0 (socket says 172.21.15.59:55446) 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f1342618700 1 -- 172.21.15.59:0/948505796 learned_addr learned my addr 172.21.15.59:0/948505796 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f1341e17700 1 --2- 172.21.15.59:0/948505796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 0x5555b71a5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.645+0000 7f1341616700 1 --2- 172.21.15.59:0/948505796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719dda0 0x5555b7192550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.646+0000 7f1342618700 1 -- 172.21.15.59:0/948505796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 msgr2=0x5555b71a5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.646+0000 7f1342618700 1 --2- 172.21.15.59:0/948505796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 0x5555b71a5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.646+0000 7f1342618700 1 -- 172.21.15.59:0/948505796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719dda0 msgr2=0x5555b7192550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.646+0000 7f1342618700 1 --2- 172.21.15.59:0/948505796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719dda0 0x5555b7192550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.646+0000 7f1342618700 1 -- 172.21.15.59:0/948505796 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5555b71a9d20 con 0x5555b71a1c30 2024-05-08T05:30:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.646+0000 7f1342618700 1 --2- 172.21.15.59:0/948505796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b719f5a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f13340b3b90 tx=0x7f1334009db0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed9066821035ed65 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.646+0000 7f13277fe700 1 -- 172.21.15.59:0/948505796 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13340af200 con 0x5555b71a1c30 2024-05-08T05:30:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.646+0000 7f13277fe700 1 -- 172.21.15.59:0/948505796 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13340ad070 con 0x5555b71a1c30 2024-05-08T05:30:09.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.646+0000 7f13277fe700 1 -- 172.21.15.59:0/948505796 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13340b3790 con 0x5555b71a1c30 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 -- 172.21.15.59:0/948505796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 msgr2=0x5555b719f5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 --2- 172.21.15.59:0/948505796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b719f5a0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f13340b3b90 tx=0x7f1334009db0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 -- 172.21.15.59:0/948505796 shutdown_connections 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 --2- 172.21.15.59:0/948505796 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b71a1c30 0x5555b719f5a0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 --2- 172.21.15.59:0/948505796 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b719d480 0x5555b71a5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 --2- 172.21.15.59:0/948505796 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719dda0 0x5555b7192550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 -- 172.21.15.59:0/948505796 >> 172.21.15.59:0/948505796 conn(0x5555b70c90b0 msgr2=0x5555b7191e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 -- 172.21.15.59:0/948505796 shutdown_connections 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 -- 172.21.15.59:0/948505796 wait complete. 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 -- start start 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.647+0000 7f134e90eb80 1 Processor -- start 2024-05-08T05:30:09.653 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 0x5555b71bb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71bba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f134e90eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b71a1c30 0x5555b71bfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f134e90eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5555b71b7f90 con 0x5555b719dda0 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f134e90eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5555b71b78c0 con 0x5555b719d480 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f134e90eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5555b71b8660 con 0x5555b71a1c30 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f134e90eb80 10 client.0 _ll_register_callbacks cb 0x5555b70878d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f1341616700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71bba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f1341616700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71bba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55450/0 (socket says 172.21.15.59:55450) 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f1341616700 1 -- 172.21.15.59:0/550521671 learned_addr learned my addr 172.21.15.59:0/550521671 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f1341e17700 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 0x5555b71bb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f1342618700 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b71a1c30 0x5555b71bfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f1341616700 1 -- 172.21.15.59:0/550521671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b71a1c30 msgr2=0x5555b71bfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.648+0000 7f1341616700 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b71a1c30 0x5555b71bfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f1341616700 1 -- 172.21.15.59:0/550521671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 msgr2=0x5555b71bb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:09.654 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f1341616700 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 0x5555b71bb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f1341616700 1 -- 172.21.15.59:0/550521671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5555b71947c0 con 0x5555b719dda0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f1341e17700 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 0x5555b71bb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f1341616700 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71bba20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f132c00bcc0 tx=0x7f132c010f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b8f84e4e87f811 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f13257fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f13257fa700 1 -- 172.21.15.59:0/550521671 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f132c02d060 con 0x5555b719dda0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f134e90eb80 1 -- 172.21.15.59:0/550521671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13340ad070 con 0x5555b719dda0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f13257fa700 1 -- 172.21.15.59:0/550521671 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f132c003a90 con 0x5555b719dda0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f13257fa700 1 -- 172.21.15.59:0/550521671 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f132c034050 con 0x5555b719dda0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f134e90eb80 10 client.15516 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f134e90eb80 1 -- 172.21.15.59:0/550521671 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5555b7222730 con 0x5555b719dda0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.649+0000 7f13257fa700 1 -- 172.21.15.59:0/550521671 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13340ad070 con 0x5555b719dda0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.650+0000 7f13257fa700 1 -- 172.21.15.59:0/550521671 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f132c0284c0 con 0x5555b719dda0 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.650+0000 7f134e90eb80 10 client.15516 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.650+0000 7f134e90eb80 10 client.15516 resolve_mds: resolved name 'i' to daemon mds.i 2024-05-08T05:30:09.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.650+0000 7f134e90eb80 20 client.15516 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.650+0000 7f134e90eb80 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5555b7226fc0 0x5555b7225cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.650+0000 7f134e90eb80 4 client.15516 mds_command: new command op to 24433 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.650+0000 7f134e90eb80 1 -- 172.21.15.59:0/550521671 --> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] -- command(tid 0: {"prefix": "status"}) -- 0x5555b70fd1f0 con 0x5555b7226fc0 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.650+0000 7f1341e17700 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5555b7226fc0 0x5555b7225cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f1341e17700 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5555b7226fc0 0x5555b7225cd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d90c0422e39a8a3e server_cookie=30a1e707e806957f in_seq=0 out_seq=0 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f13257fa700 10 client.15516 ms_handle_connect on v2:172.21.15.143:6838/2166229809 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f13257fa700 1 -- 172.21.15.59:0/550521671 <== mds.2 v2:172.21.15.143:6838/2166229809 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x5555b70fd1f0 con 0x5555b7226fc0 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f13257fa700 10 client.15516 handle_command_reply: tid=0 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 client.15516 shutdown 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 -- 172.21.15.59:0/550521671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 msgr2=0x5555b71bba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71bba20 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f132c00bcc0 tx=0x7f132c010f30 comp rx=0 tx=0).stop 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 -- 172.21.15.59:0/550521671 shutdown_connections 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5555b71a1c30 0x5555b71bfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.656 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5555b719d480 0x5555b71bb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.143:6838/2166229809,v1:172.21.15.143:6839/2166229809] conn(0x5555b7226fc0 0x5555b7225cd0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 --2- 172.21.15.59:0/550521671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5555b719dda0 0x5555b71bba20 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f13257fa700 0 client.15516 ms_handle_reset on v2:172.21.15.143:6838/2166229809 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 -- 172.21.15.59:0/550521671 >> 172.21.15.59:0/550521671 conn(0x5555b70c90b0 msgr2=0x5555b71926d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 -- 172.21.15.59:0/550521671 shutdown_connections 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 1 -- 172.21.15.59:0/550521671 wait complete. 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.651+0000 7f134e90eb80 20 client.15516 trim_cache size 0 max 16384 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 2, 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24433, 2024-05-08T05:30:09.657 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:30:09.658 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:30:09.658 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:30:09.658 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 224.259471577, 2024-05-08T05:30:09.658 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 30, 2024-05-08T05:30:09.658 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-05-08T05:30:09.658 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 90, 2024-05-08T05:30:09.658 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 308.122292571 2024-05-08T05:30:09.658 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 -- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13280b2c20 msgr2=0x7f13280b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13280b2c20 0x7f13280b50e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f1338009a10 tx=0x7f1338009190 comp rx=0 tx=0).stop 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 -- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c108410 msgr2=0x7f133c104b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c108410 0x7f133c104b00 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f133400b830 tx=0x7f1334004ce0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 -- 172.21.15.59:0/1564460161 shutdown_connections 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13280b2c20 0x7f13280b50e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f133c108410 0x7f133c104b00 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f133c1023a0 0x7f133c104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 --2- 172.21.15.59:0/1564460161 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f133c102d60 0x7f133c1045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 -- 172.21.15.59:0/1564460161 >> 172.21.15.59:0/1564460161 conn(0x7f133c079350 msgr2=0x7f133c101670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:09.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 -- 172.21.15.59:0/1564460161 shutdown_connections 2024-05-08T05:30:09.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:09.652+0000 7f134407b700 1 -- 172.21.15.59:0/1564460161 wait complete. 2024-05-08T05:30:10.582 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.h status 2024-05-08T05:30:10.845 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:30:11.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:11 smithi059 ceph-mon[30304]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-05-08T05:30:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:11 smithi143 ceph-mon[34337]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-05-08T05:30:11.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:11 smithi096 ceph-mon[34253]: pgmap v21: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-05-08T05:30:12.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.540+0000 7f9723390700 1 -- 172.21.15.59:0/3614614207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c102630 msgr2=0x7f971c102a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:12.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.540+0000 7f9723390700 1 --2- 172.21.15.59:0/3614614207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c102630 0x7f971c102a10 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f97180099c0 tx=0x7f9718009cd0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.540+0000 7f9723390700 1 -- 172.21.15.59:0/3614614207 shutdown_connections 2024-05-08T05:30:12.542 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.540+0000 7f9723390700 1 --2- 172.21.15.59:0/3614614207 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f971c1076c0 0x7f971c10f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.540+0000 7f9723390700 1 --2- 172.21.15.59:0/3614614207 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c102630 0x7f971c102a10 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.540+0000 7f9723390700 1 --2- 172.21.15.59:0/3614614207 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f971c102fe0 0x7f971c106fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.540+0000 7f9723390700 1 -- 172.21.15.59:0/3614614207 >> 172.21.15.59:0/3614614207 conn(0x7f971c079350 msgr2=0x7f971c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:12.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.541+0000 7f9723390700 1 -- 172.21.15.59:0/3614614207 shutdown_connections 2024-05-08T05:30:12.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.541+0000 7f9723390700 1 -- 172.21.15.59:0/3614614207 wait complete. 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.541+0000 7f9723390700 1 Processor -- start 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.541+0000 7f9723390700 1 -- start start 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f9723390700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f971c102630 0x7f971c19e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f9723390700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f971c102fe0 0x7f971c19ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f9723390700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c1076c0 0x7f971c1988b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f9723390700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f971c112550 con 0x7f971c1076c0 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f9723390700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1123d0 con 0x7f971c102fe0 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f9723390700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f971c1126d0 con 0x7f971c102630 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972192d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c1076c0 0x7f971c1988b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972192d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c1076c0 0x7f971c1988b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55464/0 (socket says 172.21.15.59:55464) 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972112c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f971c102630 0x7f971c19e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.545 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972192d700 1 -- 172.21.15.59:0/861773118 learned_addr learned my addr 172.21.15.59:0/861773118 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972092b700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f971c102fe0 0x7f971c19ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972192d700 1 -- 172.21.15.59:0/861773118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f971c102630 msgr2=0x7f971c19e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972192d700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f971c102630 0x7f971c19e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972192d700 1 -- 172.21.15.59:0/861773118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f971c102fe0 msgr2=0x7f971c19ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972192d700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f971c102fe0 0x7f971c19ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972192d700 1 -- 172.21.15.59:0/861773118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f971c199010 con 0x7f971c1076c0 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972092b700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f971c102fe0 0x7f971c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.542+0000 7f972192d700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c1076c0 0x7f971c1988b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9714007ee0 tx=0x7f971400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.543+0000 7f97127fc700 1 -- 172.21.15.59:0/861773118 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971400dbd0 con 0x7f971c1076c0 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.543+0000 7f97127fc700 1 -- 172.21.15.59:0/861773118 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971400dd70 con 0x7f971c1076c0 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.543+0000 7f9723390700 1 -- 172.21.15.59:0/861773118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f971c199300 con 0x7f971c1076c0 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.543+0000 7f97127fc700 1 -- 172.21.15.59:0/861773118 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9714005990 con 0x7f971c1076c0 2024-05-08T05:30:12.546 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.543+0000 7f9723390700 1 -- 172.21.15.59:0/861773118 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f971c1a55c0 con 0x7f971c1076c0 2024-05-08T05:30:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.545+0000 7f97127fc700 1 -- 172.21.15.59:0/861773118 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f971401a020 con 0x7f971c1076c0 2024-05-08T05:30:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.546+0000 7f97127fc700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f97080b2c20 0x7f97080b50e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.546+0000 7f97127fc700 1 -- 172.21.15.59:0/861773118 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f97140e2300 con 0x7f971c1076c0 2024-05-08T05:30:12.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.546+0000 7f972112c700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f97080b2c20 0x7f97080b50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972dc23b80 1 Processor -- start 2024-05-08T05:30:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972dc23b80 1 -- start start 2024-05-08T05:30:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972112c700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f97080b2c20 0x7f97080b50e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f971c1999f0 tx=0x7f9718005c40 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb4852e0 0x563cfb4856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 0x563cfb47f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 0x563cfb480350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972dc23b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563cfb3cf8f0 con 0x563cfb4852e0 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972dc23b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563cfb334d50 con 0x563cfb47ff10 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972dc23b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563cfb3cb8b0 con 0x563cfb485c00 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972112c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb4852e0 0x563cfb4856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972112c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb4852e0 0x563cfb4856c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55474/0 (socket says 172.21.15.59:55474) 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972112c700 1 -- 172.21.15.59:0/3291851270 learned_addr learned my addr 172.21.15.59:0/3291851270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972092b700 1 --2- 172.21.15.59:0/3291851270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 0x563cfb47f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972192d700 1 --2- 172.21.15.59:0/3291851270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 0x563cfb480350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972112c700 1 -- 172.21.15.59:0/3291851270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 msgr2=0x563cfb47f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972112c700 1 --2- 172.21.15.59:0/3291851270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 0x563cfb47f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972112c700 1 -- 172.21.15.59:0/3291851270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 msgr2=0x563cfb480350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972112c700 1 --2- 172.21.15.59:0/3291851270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 0x563cfb480350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.547+0000 7f972112c700 1 -- 172.21.15.59:0/3291851270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cfb31c880 con 0x563cfb4852e0 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.548+0000 7f972112c700 1 --2- 172.21.15.59:0/3291851270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb4852e0 0x563cfb4856c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f97180099c0 tx=0x7f9718020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefe0a5634bd1833 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:12.550 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.548+0000 7f97077fe700 1 -- 172.21.15.59:0/3291851270 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971800f620 con 0x563cfb4852e0 2024-05-08T05:30:12.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.548+0000 7f97077fe700 1 -- 172.21.15.59:0/3291851270 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971800fc70 con 0x563cfb4852e0 2024-05-08T05:30:12.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.548+0000 7f97077fe700 1 -- 172.21.15.59:0/3291851270 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9718024ca0 con 0x563cfb4852e0 2024-05-08T05:30:12.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 -- 172.21.15.59:0/3291851270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb4852e0 msgr2=0x563cfb4856c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:12.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 --2- 172.21.15.59:0/3291851270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb4852e0 0x563cfb4856c0 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f97180099c0 tx=0x7f9718020000 comp rx=0 tx=0).stop 2024-05-08T05:30:12.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 -- 172.21.15.59:0/3291851270 shutdown_connections 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 --2- 172.21.15.59:0/3291851270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 0x563cfb480350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 --2- 172.21.15.59:0/3291851270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb4852e0 0x563cfb4856c0 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 --2- 172.21.15.59:0/3291851270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 0x563cfb47f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 -- 172.21.15.59:0/3291851270 >> 172.21.15.59:0/3291851270 conn(0x563cfb3a9b20 msgr2=0x563cfb478250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 -- 172.21.15.59:0/3291851270 shutdown_connections 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 -- 172.21.15.59:0/3291851270 wait complete. 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 -- start start 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.549+0000 7f972dc23b80 1 Processor -- start 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb47ff10 0x563cfb48b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 0x563cfb488bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb489110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972dc23b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563cfb48c530 con 0x563cfb485c00 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972dc23b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563cfb388e80 con 0x563cfb4852e0 2024-05-08T05:30:12.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972dc23b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563cfb48c6b0 con 0x563cfb47ff10 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972dc23b80 10 client.0 _ll_register_callbacks cb 0x563cfb363560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972192d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb489110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972192d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb489110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55476/0 (socket says 172.21.15.59:55476) 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972092b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 0x563cfb488bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972192d700 1 -- 172.21.15.59:0/1949925482 learned_addr learned my addr 172.21.15.59:0/1949925482 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972112c700 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb47ff10 0x563cfb48b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972192d700 1 -- 172.21.15.59:0/1949925482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb47ff10 msgr2=0x563cfb48b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972192d700 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb47ff10 0x563cfb48b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972192d700 1 -- 172.21.15.59:0/1949925482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 msgr2=0x563cfb488bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972192d700 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 0x563cfb488bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972192d700 1 -- 172.21.15.59:0/1949925482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cfb386710 con 0x563cfb485c00 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972092b700 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 0x563cfb488bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f972192d700 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb489110 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f971400d7b0 tx=0x7f97140a5540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d97fe8dad324cfe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:12.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.550+0000 7f9704ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f9704ff9700 1 -- 172.21.15.59:0/1949925482 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140a9730 con 0x563cfb485c00 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f9704ff9700 1 -- 172.21.15.59:0/1949925482 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f97140b1c10 con 0x563cfb485c00 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f972dc23b80 1 -- 172.21.15.59:0/1949925482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f971800fc70 con 0x563cfb485c00 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f9704ff9700 1 -- 172.21.15.59:0/1949925482 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f97140a9030 con 0x563cfb485c00 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f972dc23b80 10 client.15528 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f972dc23b80 1 -- 172.21.15.59:0/1949925482 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563cfb47d870 con 0x563cfb485c00 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f9704ff9700 1 -- 172.21.15.59:0/1949925482 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f971800fc70 con 0x563cfb485c00 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f9704ff9700 1 -- 172.21.15.59:0/1949925482 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f97140df8d0 con 0x563cfb485c00 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f972dc23b80 10 client.15528 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f972dc23b80 10 client.15528 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f972dc23b80 20 client.15528 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x563cfb4978c0 0x563cfb498660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f972dc23b80 4 client.15528 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.551+0000 7f972dc23b80 1 -- 172.21.15.59:0/1949925482 --> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563cfb3dd2d0 con 0x563cfb4978c0 2024-05-08T05:30:12.557 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.552+0000 7f972112c700 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x563cfb4978c0 0x563cfb498660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.552+0000 7f972112c700 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x563cfb4978c0 0x563cfb498660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=1147a9e3aaa6d9a3 server_cookie=67a3b4613aa17997 in_seq=0 out_seq=0 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.552+0000 7f9704ff9700 10 client.15528 ms_handle_connect on v2:172.21.15.96:6836/780978870 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f9704ff9700 1 -- 172.21.15.59:0/1949925482 <== mds.1 v2:172.21.15.96:6836/780978870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563cfb3dd2d0 con 0x563cfb4978c0 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f9704ff9700 10 client.15528 handle_command_reply: tid=0 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 client.15528 shutdown 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 -- 172.21.15.59:0/1949925482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 msgr2=0x563cfb489110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb489110 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f971400d7b0 tx=0x7f97140a5540 comp rx=0 tx=0).stop 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 -- 172.21.15.59:0/1949925482 shutdown_connections 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x563cfb4978c0 0x563cfb498660 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb489110 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb47ff10 0x563cfb48b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f9704ff9700 0 client.15528 ms_handle_reset on v2:172.21.15.96:6836/780978870 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1949925482 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 0x563cfb488bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 -- 172.21.15.59:0/1949925482 >> 172.21.15.59:0/1949925482 conn(0x563cfb3a9b20 msgr2=0x563cfb472ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:12.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 -- 172.21.15.59:0/1949925482 shutdown_connections 2024-05-08T05:30:12.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 1 -- 172.21.15.59:0/1949925482 wait complete. 2024-05-08T05:30:12.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.553+0000 7f972dc23b80 20 client.15528 trim_cache size 0 max 16384 2024-05-08T05:30:12.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972dc23b80 1 Processor -- start 2024-05-08T05:30:12.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972dc23b80 1 -- start start 2024-05-08T05:30:12.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 0x563cfb48b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.577 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb4852e0 0x563cfb472bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb4874c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972dc23b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563cfb47f5e0 con 0x563cfb485c00 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972dc23b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563cfb497070 con 0x563cfb47ff10 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972dc23b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563cfb497740 con 0x563cfb4852e0 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972192d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb4874c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972192d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb4874c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55486/0 (socket says 172.21.15.59:55486) 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972192d700 1 -- 172.21.15.59:0/3278911701 learned_addr learned my addr 172.21.15.59:0/3278911701 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972112c700 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 0x563cfb48b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.575+0000 7f972092b700 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb4852e0 0x563cfb472bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f972192d700 1 -- 172.21.15.59:0/3278911701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb4852e0 msgr2=0x563cfb472bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f972192d700 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb4852e0 0x563cfb472bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f972192d700 1 -- 172.21.15.59:0/3278911701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 msgr2=0x563cfb48b7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:12.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f972192d700 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 0x563cfb48b7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f972192d700 1 -- 172.21.15.59:0/3278911701 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563cfb4790c0 con 0x563cfb485c00 2024-05-08T05:30:12.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f972112c700 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 0x563cfb48b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:12.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f972192d700 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb4874c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f97140df590 tx=0x7f9714004e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd1c4faa26f71c8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:12.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f9706ffd700 1 -- 172.21.15.59:0/3278911701 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140a87a0 con 0x563cfb485c00 2024-05-08T05:30:12.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f9706ffd700 1 -- 172.21.15.59:0/3278911701 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97140a9730 con 0x563cfb485c00 2024-05-08T05:30:12.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.576+0000 7f9706ffd700 1 -- 172.21.15.59:0/3278911701 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97140df7b0 con 0x563cfb485c00 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 -- 172.21.15.59:0/3278911701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 msgr2=0x563cfb4874c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb4874c0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f97140df590 tx=0x7f9714004e10 comp rx=0 tx=0).stop 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 -- 172.21.15.59:0/3278911701 shutdown_connections 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb485c00 0x563cfb4874c0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb47ff10 0x563cfb48b7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 --2- 172.21.15.59:0/3278911701 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb4852e0 0x563cfb472bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 -- 172.21.15.59:0/3278911701 >> 172.21.15.59:0/3278911701 conn(0x563cfb3a9b20 msgr2=0x563cfb478160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 -- 172.21.15.59:0/3278911701 shutdown_connections 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 -- 172.21.15.59:0/3278911701 wait complete. 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 -- start start 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.577+0000 7f972dc23b80 1 Processor -- start 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb47ff10 0x563cfb4721a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 0x563cfb4726e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972dc23b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 0x563cfb472c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972dc23b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563cfb4984e0 con 0x563cfb47ff10 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972dc23b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563cfb497e10 con 0x563cfb4852e0 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972dc23b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563cfb498bb0 con 0x563cfb485c00 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972dc23b80 10 client.0 _ll_register_callbacks cb 0x563cfb363560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972112c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb47ff10 0x563cfb4721a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972112c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb47ff10 0x563cfb4721a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55494/0 (socket says 172.21.15.59:55494) 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972112c700 1 -- 172.21.15.59:0/1609619580 learned_addr learned my addr 172.21.15.59:0/1609619580 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972092b700 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 0x563cfb4726e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972192d700 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 0x563cfb472c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972112c700 1 -- 172.21.15.59:0/1609619580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 msgr2=0x563cfb472c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972112c700 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 0x563cfb472c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972112c700 1 -- 172.21.15.59:0/1609619580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 msgr2=0x563cfb4726e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972112c700 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 0x563cfb4726e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972112c700 1 -- 172.21.15.59:0/1609619580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563cfb4995c0 con 0x563cfb47ff10 2024-05-08T05:30:12.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972192d700 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 0x563cfb472c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f972112c700 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb47ff10 0x563cfb4721a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f971800b9f0 tx=0x7f971801a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5c97c29a6c9e827 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.578+0000 7f9704ff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f9704ff9700 1 -- 172.21.15.59:0/1609619580 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f971800f830 con 0x563cfb47ff10 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f9704ff9700 1 -- 172.21.15.59:0/1609619580 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f971800f9d0 con 0x563cfb47ff10 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f972dc23b80 1 -- 172.21.15.59:0/1609619580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97140a9730 con 0x563cfb47ff10 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f9704ff9700 1 -- 172.21.15.59:0/1609619580 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f971804e050 con 0x563cfb47ff10 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f972dc23b80 10 client.15534 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f972dc23b80 1 -- 172.21.15.59:0/1609619580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563cfb502820 con 0x563cfb47ff10 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f9704ff9700 1 -- 172.21.15.59:0/1609619580 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97140a9730 con 0x563cfb47ff10 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f9704ff9700 1 -- 172.21.15.59:0/1609619580 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f9718037660 con 0x563cfb47ff10 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f972dc23b80 10 client.15534 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f972dc23b80 10 client.15534 resolve_mds: resolved name 'h' to daemon mds.h 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f972dc23b80 20 client.15534 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x563cfb506ea0 0x563cfb505bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:12.585 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f972dc23b80 4 client.15534 mds_command: new command op to 24443 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.579+0000 7f972dc23b80 1 -- 172.21.15.59:0/1609619580 --> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] -- command(tid 0: {"prefix": "status"}) -- 0x563cfb3d7370 con 0x563cfb506ea0 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f972092b700 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x563cfb506ea0 0x563cfb505bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f9704ff9700 10 client.15534 ms_handle_connect on v2:172.21.15.96:6836/780978870 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f972092b700 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x563cfb506ea0 0x563cfb505bb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=782e37bb63e2d238 server_cookie=d3577415d7b239a2 in_seq=0 out_seq=0 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f9704ff9700 1 -- 172.21.15.59:0/1609619580 <== mds.1 v2:172.21.15.96:6836/780978870 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x563cfb3d7370 con 0x563cfb506ea0 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f9704ff9700 10 client.15534 handle_command_reply: tid=0 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f972dc23b80 1 client.15534 shutdown 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f972dc23b80 1 -- 172.21.15.59:0/1609619580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb47ff10 msgr2=0x563cfb4721a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb47ff10 0x563cfb4721a0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f971800b9f0 tx=0x7f971801a040 comp rx=0 tx=0).stop 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f972dc23b80 1 -- 172.21.15.59:0/1609619580 shutdown_connections 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.580+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.96:6836/780978870,v1:172.21.15.96:6837/780978870] conn(0x563cfb506ea0 0x563cfb505bb0 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563cfb485c00 0x563cfb472c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563cfb47ff10 0x563cfb4721a0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f972dc23b80 1 --2- 172.21.15.59:0/1609619580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563cfb4852e0 0x563cfb4726e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f972dc23b80 1 -- 172.21.15.59:0/1609619580 >> 172.21.15.59:0/1609619580 conn(0x563cfb3a9b20 msgr2=0x563cfb4860c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f9704ff9700 0 client.15534 ms_handle_reset on v2:172.21.15.96:6836/780978870 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f972dc23b80 1 -- 172.21.15.59:0/1609619580 shutdown_connections 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f972dc23b80 1 -- 172.21.15.59:0/1609619580 wait complete. 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f972dc23b80 20 client.15534 trim_cache size 0 max 16384 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 1, 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24443, 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:30:12.587 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:30:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 228.19330652900001, 2024-05-08T05:30:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 30, 2024-05-08T05:30:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-05-08T05:30:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 90, 2024-05-08T05:30:12.588 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 313.78016621799998 2024-05-08T05:30:12.588 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f9723390700 1 -- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f97080b2c20 msgr2=0x7f97080b50e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f9723390700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f97080b2c20 0x7f97080b50e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f971c1999f0 tx=0x7f9718005c40 comp rx=0 tx=0).stop 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f9723390700 1 -- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c1076c0 msgr2=0x7f971c1988b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f9723390700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c1076c0 0x7f971c1988b0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f9714007ee0 tx=0x7f971400d3b0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f9723390700 1 -- 172.21.15.59:0/861773118 shutdown_connections 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f9723390700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f97080b2c20 0x7f97080b50e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.581+0000 7f9723390700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f971c1076c0 0x7f971c1988b0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.582+0000 7f9723390700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f971c102630 0x7f971c19e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.582+0000 7f9723390700 1 --2- 172.21.15.59:0/861773118 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f971c102fe0 0x7f971c19ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.582+0000 7f9723390700 1 -- 172.21.15.59:0/861773118 >> 172.21.15.59:0/861773118 conn(0x7f971c079350 msgr2=0x7f971c100670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.582+0000 7f9723390700 1 -- 172.21.15.59:0/861773118 shutdown_connections 2024-05-08T05:30:12.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:12.582+0000 7f9723390700 1 -- 172.21.15.59:0/861773118 wait complete. 2024-05-08T05:30:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:13 smithi059 ceph-mon[30304]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-05-08T05:30:13.390 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph tell mds.b status 2024-05-08T05:30:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:13 smithi143 ceph-mon[34337]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-05-08T05:30:13.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:13 smithi096 ceph-mon[34253]: pgmap v22: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 29 op/s 2024-05-08T05:30:13.648 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:30:15.361 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:15 smithi059 ceph-mon[30304]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-08T05:30:15.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.471+0000 7f0f0210d700 1 -- 172.21.15.59:0/2429188851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06ba10 msgr2=0x7f0efc06be90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:15.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.471+0000 7f0f0210d700 1 --2- 172.21.15.59:0/2429188851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06ba10 0x7f0efc06be90 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4009990 tx=0x7f0ee4009ca0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.471+0000 7f0f0210d700 1 -- 172.21.15.59:0/2429188851 shutdown_connections 2024-05-08T05:30:15.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.471+0000 7f0f0210d700 1 --2- 172.21.15.59:0/2429188851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0efc06c3d0 0x7f0efc110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.471+0000 7f0f0210d700 1 --2- 172.21.15.59:0/2429188851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0efc109500 0x7f0efc1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.471+0000 7f0f0210d700 1 --2- 172.21.15.59:0/2429188851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06ba10 0x7f0efc06be90 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.471+0000 7f0f0210d700 1 -- 172.21.15.59:0/2429188851 >> 172.21.15.59:0/2429188851 conn(0x7f0efc079310 msgr2=0x7f0efc079720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:15.473 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.471+0000 7f0f0210d700 1 -- 172.21.15.59:0/2429188851 shutdown_connections 2024-05-08T05:30:15.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.471+0000 7f0f0210d700 1 -- 172.21.15.59:0/2429188851 wait complete. 2024-05-08T05:30:15.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0f0210d700 1 Processor -- start 2024-05-08T05:30:15.474 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0f0210d700 1 -- start start 2024-05-08T05:30:15.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0f0210d700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0efc06ba10 0x7f0efc19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0f0210d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06c3d0 0x7f0efc19fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0f0210d700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0efc109500 0x7f0efc1999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0f0210d700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc113680 con 0x7f0efc06c3d0 2024-05-08T05:30:15.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0f0210d700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc113500 con 0x7f0efc109500 2024-05-08T05:30:15.475 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0f0210d700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0efc113800 con 0x7f0efc06ba10 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06c3d0 0x7f0efc19fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06c3d0 0x7f0efc19fd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55514/0 (socket says 172.21.15.59:55514) 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/1381407794 learned_addr learned my addr 172.21.15.59:0/1381407794 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0efbfff700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0efc109500 0x7f0efc1999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.472+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0efc06ba10 0x7f0efc19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/1381407794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0efc06ba10 msgr2=0x7f0efc19f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0efc06ba10 0x7f0efc19f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/1381407794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0efc109500 msgr2=0x7f0efc1999e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0efc109500 0x7f0efc1999e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/1381407794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0efc19a140 con 0x7f0efc06c3d0 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06c3d0 0x7f0efc19fd60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4005910 tx=0x7f0ee40047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0ef97fa700 1 -- 172.21.15.59:0/1381407794 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4027070 con 0x7f0efc06c3d0 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0f0210d700 1 -- 172.21.15.59:0/1381407794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0efc19a3d0 con 0x7f0efc06c3d0 2024-05-08T05:30:15.476 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0ef97fa700 1 -- 172.21.15.59:0/1381407794 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee4003f20 con 0x7f0efc06c3d0 2024-05-08T05:30:15.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0ef97fa700 1 -- 172.21.15.59:0/1381407794 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee4022c60 con 0x7f0efc06c3d0 2024-05-08T05:30:15.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.473+0000 7f0f0210d700 1 -- 172.21.15.59:0/1381407794 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0efc1a66f0 con 0x7f0efc06c3d0 2024-05-08T05:30:15.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.476+0000 7f0ef97fa700 1 -- 172.21.15.59:0/1381407794 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f0ee402a070 con 0x7f0efc06c3d0 2024-05-08T05:30:15.477 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.476+0000 7f0ef97fa700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0ee80b2ba0 0x7f0ee80b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0ee80b2ba0 0x7f0ee80b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0ef97fa700 1 -- 172.21.15.59:0/1381407794 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0ee4022e00 con 0x7f0efc06c3d0 2024-05-08T05:30:15.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0f0c9a0b80 1 Processor -- start 2024-05-08T05:30:15.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0f0c9a0b80 1 -- start start 2024-05-08T05:30:15.479 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0ee80b2ba0 0x7f0ee80b5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0efc19ab20 tx=0x7f0eec00b410 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 0x5646f565a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5654bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5655550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5646f55a28f0 con 0x5646f5655110 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5646f5507d50 con 0x5646f565ae90 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5646f559e8b0 con 0x5646f565a570 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0efbfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5655550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5654bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5654bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:42718/0 (socket says 172.21.15.59:42718) 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/184876231 learned_addr learned my addr 172.21.15.59:0/184876231 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.477+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/184876231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 0x5646f565a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.478+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/184876231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 msgr2=0x5646f565a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.478+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/184876231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 0x5646f565a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.478+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/184876231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 msgr2=0x5646f5655550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:15.480 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.478+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/184876231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5655550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.478+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/184876231 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f54ef880 con 0x5646f565ae90 2024-05-08T05:30:15.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.478+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/184876231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5654bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4003800 tx=0x7f0ee4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12d0aab837ae17c9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:15.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.478+0000 7f0ef1ffb700 1 -- 172.21.15.59:0/184876231 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40cd0d0 con 0x5646f565ae90 2024-05-08T05:30:15.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.478+0000 7f0ef1ffb700 1 -- 172.21.15.59:0/184876231 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee40b10f0 con 0x5646f565ae90 2024-05-08T05:30:15.481 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.478+0000 7f0ef1ffb700 1 -- 172.21.15.59:0/184876231 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40b3140 con 0x5646f565ae90 2024-05-08T05:30:15.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/184876231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 msgr2=0x5646f5654bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:15.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/184876231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5654bd0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4003800 tx=0x7f0ee4005fd0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/184876231 shutdown_connections 2024-05-08T05:30:15.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/184876231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5655550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/184876231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 0x5646f565a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/184876231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5654bd0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/184876231 >> 172.21.15.59:0/184876231 conn(0x5646f557cb20 msgr2=0x5646f564b250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:15.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/184876231 shutdown_connections 2024-05-08T05:30:15.485 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/184876231 wait complete. 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 -- start start 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.479+0000 7f0f0c9a0b80 1 Processor -- start 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 0x5646f5660ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 0x5646f565e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5646f56615d0 con 0x5646f565a570 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5646f555be80 con 0x5646f5655110 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5646f5661750 con 0x5646f565ae90 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0f0c9a0b80 10 client.0 _ll_register_callbacks cb 0x5646f5536560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565de70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55526/0 (socket says 172.21.15.59:55526) 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/720165231 learned_addr learned my addr 172.21.15.59:0/720165231 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:15.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 0x5646f5660ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0efbfff700 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 0x5646f565e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/720165231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 msgr2=0x5646f565e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 0x5646f565e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/720165231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 msgr2=0x5646f5660ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 0x5646f5660ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/720165231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f5559710 con 0x5646f565a570 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565de70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40a6880 tx=0x7f0ee40b3470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64db91ce03163f3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0edaffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.480+0000 7f0edaffd700 1 -- 172.21.15.59:0/720165231 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40c4060 con 0x5646f565a570 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0edaffd700 1 -- 172.21.15.59:0/720165231 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0ee4092320 con 0x5646f565a570 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/720165231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5646f55b02d0 con 0x5646f565a570 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0edaffd700 1 -- 172.21.15.59:0/720165231 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ee40cd0d0 con 0x5646f565a570 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0f0c9a0b80 10 client.15546 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:15.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/720165231 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646f56615d0 con 0x5646f565a570 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0edaffd700 1 -- 172.21.15.59:0/720165231 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5646f55b02d0 con 0x5646f565a570 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0edaffd700 1 -- 172.21.15.59:0/720165231 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0ee40bfbc0 con 0x5646f565a570 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0f0c9a0b80 10 client.15546 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0f0c9a0b80 10 client.15546 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0f0c9a0b80 20 client.15546 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5646f566e190 0x5646f566f600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0f0c9a0b80 4 client.15546 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/720165231 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646f55aa370 con 0x5646f566e190 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.481+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5646f566e190 0x5646f566f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.482+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5646f566e190 0x5646f566f600 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4be98e027568d85 server_cookie=ca359b5be201d94 in_seq=0 out_seq=0 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.482+0000 7f0edaffd700 10 client.15546 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0edaffd700 1 -- 172.21.15.59:0/720165231 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5646f55aa370 con 0x5646f566e190 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0edaffd700 10 client.15546 handle_command_reply: tid=0 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 client.15546 shutdown 2024-05-08T05:30:15.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/720165231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 msgr2=0x5646f565de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565de70 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40a6880 tx=0x7f0ee40b3470 comp rx=0 tx=0).stop 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/720165231 shutdown_connections 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5646f566e190 0x5646f566f600 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 0x5646f565e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 0x5646f5660ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0edaffd700 0 client.15546 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/720165231 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565de70 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/720165231 >> 172.21.15.59:0/720165231 conn(0x5646f557cb20 msgr2=0x5646f5645ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/720165231 shutdown_connections 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/720165231 wait complete. 2024-05-08T05:30:15.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.483+0000 7f0f0c9a0b80 20 client.15546 trim_cache size 0 max 16384 2024-05-08T05:30:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:15 smithi143 ceph-mon[34337]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-08T05:30:15.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.504+0000 7f0f0c9a0b80 1 Processor -- start 2024-05-08T05:30:15.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.504+0000 7f0f0c9a0b80 1 -- start start 2024-05-08T05:30:15.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.504+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 0x5646f5654bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.508 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.504+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.504+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 0x5646f5645bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.504+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5646f566d940 con 0x5646f565a570 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.504+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5646f566e010 con 0x5646f5655110 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.504+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5646f566e6e0 con 0x5646f565ae90 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565b980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55542/0 (socket says 172.21.15.59:55542) 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/4063479066 learned_addr learned my addr 172.21.15.59:0/4063479066 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 0x5646f5654bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0efbfff700 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 0x5646f5645bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/4063479066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 msgr2=0x5646f5645bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 0x5646f5645bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/4063479066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 msgr2=0x5646f5654bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:15.509 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 0x5646f5654bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef2ffd700 1 -- 172.21.15.59:0/4063479066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646f5646130 con 0x5646f565a570 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0efbfff700 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 0x5646f5645bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565b980 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40bff60 tx=0x7f0ee4009ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ade67177341bf4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef17fa700 1 -- 172.21.15.59:0/4063479066 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40c4030 con 0x5646f565a570 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef17fa700 1 -- 172.21.15.59:0/4063479066 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee40b35d0 con 0x5646f565a570 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.505+0000 7f0ef17fa700 1 -- 172.21.15.59:0/4063479066 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ee40abe60 con 0x5646f565a570 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/4063479066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 msgr2=0x5646f565b980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565b980 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f0ee40bff60 tx=0x7f0ee4009ca0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/4063479066 shutdown_connections 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565ae90 0x5646f5645bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f5655110 0x5646f5654bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/4063479066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f565a570 0x5646f565b980 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/4063479066 >> 172.21.15.59:0/4063479066 conn(0x5646f557cb20 msgr2=0x5646f565b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:15.510 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/4063479066 shutdown_connections 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/4063479066 wait complete. 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 -- start start 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.506+0000 7f0f0c9a0b80 1 Processor -- start 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5678fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 0x5646f5644be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0f0c9a0b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5645140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5646f566f480 con 0x5646f5655110 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5646f566edb0 con 0x5646f565ae90 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0f0c9a0b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5646f566fb50 con 0x5646f565a570 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0f0c9a0b80 10 client.0 _ll_register_callbacks cb 0x5646f5536560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5678fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efbfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5645140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efb7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5678fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55554/0 (socket says 172.21.15.59:55554) 2024-05-08T05:30:15.511 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efb7fe700 1 -- 172.21.15.59:0/1210370867 learned_addr learned my addr 172.21.15.59:0/1210370867 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:15.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0ef2ffd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 0x5646f5644be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efb7fe700 1 -- 172.21.15.59:0/1210370867 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 msgr2=0x5646f5644be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:15.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 0x5646f5644be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efb7fe700 1 -- 172.21.15.59:0/1210370867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 msgr2=0x5646f5645140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:15.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5645140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.512 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efb7fe700 1 -- 172.21.15.59:0/1210370867 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646f5645980 con 0x5646f5655110 2024-05-08T05:30:15.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0efb7fe700 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5678fa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5646f566fe20 tx=0x7f0eec0047c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a491d3a2b1760a78 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:15.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.507+0000 7f0edaffd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:15.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0edaffd700 1 -- 172.21.15.59:0/1210370867 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0eec011dd0 con 0x5646f5655110 2024-05-08T05:30:15.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0edaffd700 1 -- 172.21.15.59:0/1210370867 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0eec02a380 con 0x5646f5655110 2024-05-08T05:30:15.514 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/1210370867 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ee40b35d0 con 0x5646f5655110 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0edaffd700 1 -- 172.21.15.59:0/1210370867 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0eec02a580 con 0x5646f5655110 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0f0c9a0b80 10 client.15552 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/1210370867 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5646f56d9660 con 0x5646f5655110 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0edaffd700 1 -- 172.21.15.59:0/1210370867 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ee40b35d0 con 0x5646f5655110 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0edaffd700 1 -- 172.21.15.59:0/1210370867 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0eec01fb40 con 0x5646f5655110 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0f0c9a0b80 10 client.15552 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0f0c9a0b80 10 client.15552 resolve_mds: resolved name 'b' to daemon mds.b 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0f0c9a0b80 20 client.15552 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5646f56de240 0x5646f56dcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0f0c9a0b80 4 client.15552 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "status"}] 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.508+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/1210370867 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "status"}) -- 0x5646f5422200 con 0x5646f56de240 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5646f56de240 0x5646f56dcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0edaffd700 10 client.15552 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0ef2ffd700 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5646f56de240 0x5646f56dcf50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=497563a74dee9538 server_cookie=2163f928cf6664c1 in_seq=0 out_seq=0 2024-05-08T05:30:15.515 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0edaffd700 1 -- 172.21.15.59:0/1210370867 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5646f5422200 con 0x5646f56de240 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0edaffd700 10 client.15552 handle_command_reply: tid=0 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0f0c9a0b80 1 client.15552 shutdown 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/1210370867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 msgr2=0x5646f5678fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5678fa0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x5646f566fe20 tx=0x7f0eec0047c0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/1210370867 shutdown_connections 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5646f56de240 0x5646f56dcf50 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5646f565ae90 0x5646f5645140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5646f5655110 0x5646f5678fa0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0f0c9a0b80 1 --2- 172.21.15.59:0/1210370867 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5646f565a570 0x5646f5644be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/1210370867 >> 172.21.15.59:0/1210370867 conn(0x5646f557cb20 msgr2=0x5646f5659870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.509+0000 7f0edaffd700 0 client.15552 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/1210370867 shutdown_connections 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0c9a0b80 1 -- 172.21.15.59:0/1210370867 wait complete. 2024-05-08T05:30:15.516 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0c9a0b80 20 client.15552 trim_cache size 0 max 16384 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "cluster_fsid": "0bf01a10-0cfa-11ef-bc97-c7b262605968", 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "whoami": 0, 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "id": 24473, 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "want_state": "up:active", 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "state": "up:active", 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "fs_name": "cephfs", 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "rank_uptime": 287.84768732100002, 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "mdsmap_epoch": 30, 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch": 91, 2024-05-08T05:30:15.517 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_epoch_barrier": 90, 2024-05-08T05:30:15.518 INFO:teuthology.orchestra.run.smithi059.stdout: "uptime": 303.76669311299997 2024-05-08T05:30:15.518 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:30:15.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 -- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0ee80b2ba0 msgr2=0x7f0ee80b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:15.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0ee80b2ba0 0x7f0ee80b5060 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0efc19ab20 tx=0x7f0eec00b410 comp rx=0 tx=0).stop 2024-05-08T05:30:15.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 -- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06c3d0 msgr2=0x7f0efc19fd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:15.518 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06c3d0 0x7f0efc19fd60 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f0ee4005910 tx=0x7f0ee40047b0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 -- 172.21.15.59:0/1381407794 shutdown_connections 2024-05-08T05:30:15.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0ee80b2ba0 0x7f0ee80b5060 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0efc109500 0x7f0efc1999e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0efc06ba10 0x7f0efc19f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 --2- 172.21.15.59:0/1381407794 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0efc06c3d0 0x7f0efc19fd60 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:15.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 -- 172.21.15.59:0/1381407794 >> 172.21.15.59:0/1381407794 conn(0x7f0efc079310 msgr2=0x7f0efc101190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:15.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 -- 172.21.15.59:0/1381407794 shutdown_connections 2024-05-08T05:30:15.519 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:15.510+0000 7f0f0210d700 1 -- 172.21.15.59:0/1381407794 wait complete. 2024-05-08T05:30:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:15 smithi096 ceph-mon[34253]: pgmap v23: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 26 op/s 2024-05-08T05:30:16.360 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 15 sec 2024-05-08T05:30:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.083+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/3846315158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141018e0 msgr2=0x7f8d14101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.083+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/3846315158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141018e0 0x7f8d14101d60 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc009930 tx=0x7f8cfc02a420 comp rx=0 tx=0).stop 2024-05-08T05:30:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.083+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/3846315158 shutdown_connections 2024-05-08T05:30:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.083+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/3846315158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d141022a0 0x7f8d1410e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.083+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/3846315158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141018e0 0x7f8d14101d60 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.085 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.083+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/3846315158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d141078a0 0x7f8d14107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.083+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/3846315158 >> 172.21.15.59:0/3846315158 conn(0x7f8d14078a30 msgr2=0x7f8d14078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.083+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/3846315158 shutdown_connections 2024-05-08T05:30:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.083+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/3846315158 wait complete. 2024-05-08T05:30:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d1a0fc640 1 Processor -- start 2024-05-08T05:30:17.086 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d1a0fc640 1 -- start start 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d1a0fc640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d141018e0 0x7f8d14199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d1a0fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141022a0 0x7f8d141996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d1a0fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d141078a0 0x7f8d1419da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d1a0fc640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14111bb0 con 0x7f8d141022a0 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d1a0fc640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14111a30 con 0x7f8d141078a0 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d1a0fc640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8d14111d30 con 0x7f8d141018e0 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d0affd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141022a0 0x7f8d141996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d0affd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141022a0 0x7f8d141996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55570/0 (socket says 172.21.15.59:55570) 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d0affd640 1 -- 172.21.15.59:0/337082494 learned_addr learned my addr 172.21.15.59:0/337082494 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.084+0000 7f8d137fe640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d141018e0 0x7f8d14199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d13fff640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d141078a0 0x7f8d1419da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d0affd640 1 -- 172.21.15.59:0/337082494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d141018e0 msgr2=0x7f8d14199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d0affd640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d141018e0 0x7f8d14199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d0affd640 1 -- 172.21.15.59:0/337082494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d141078a0 msgr2=0x7f8d1419da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d0affd640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d141078a0 0x7f8d1419da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d0affd640 1 -- 172.21.15.59:0/337082494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d1419e180 con 0x7f8d141022a0 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d13fff640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d141078a0 0x7f8d1419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d137fe640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d141018e0 0x7f8d14199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d0affd640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141022a0 0x7f8d141996d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005ab0 tx=0x7f8cfc003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d117fa640 1 -- 172.21.15.59:0/337082494 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc03f070 con 0x7f8d141022a0 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/337082494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d1419e410 con 0x7f8d141022a0 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d117fa640 1 -- 172.21.15.59:0/337082494 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cfc031b50 con 0x7f8d141022a0 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d117fa640 1 -- 172.21.15.59:0/337082494 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cfc035030 con 0x7f8d141022a0 2024-05-08T05:30:17.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.085+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/337082494 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d141a5d00 con 0x7f8d141022a0 2024-05-08T05:30:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.088+0000 7f8d117fa640 1 -- 172.21.15.59:0/337082494 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f8cfc042070 con 0x7f8d141022a0 2024-05-08T05:30:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.088+0000 7f8d1ce14740 1 Processor -- start 2024-05-08T05:30:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.088+0000 7f8d1ce14740 1 -- start start 2024-05-08T05:30:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.088+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce02f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.088+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998ceabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998ced4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f998c455d0 con 0x55f998ceb0f0 2024-05-08T05:30:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f998bfe890 con 0x55f998ce0830 2024-05-08T05:30:17.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f998c41590 con 0x55f998cdff10 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d13fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce02f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d13fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce02f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:37986/0 (socket says 172.21.15.59:37986) 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d13fff640 1 -- 172.21.15.59:0/3462003882 learned_addr learned my addr 172.21.15.59:0/3462003882 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d0affd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998ced4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d137fe640 1 --2- 172.21.15.59:0/3462003882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998ceabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d13fff640 1 -- 172.21.15.59:0/3462003882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 msgr2=0x55f998ceabb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d13fff640 1 --2- 172.21.15.59:0/3462003882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998ceabb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d13fff640 1 -- 172.21.15.59:0/3462003882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 msgr2=0x55f998ced4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d13fff640 1 --2- 172.21.15.59:0/3462003882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998ced4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d13fff640 1 -- 172.21.15.59:0/3462003882 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f998957210 con 0x55f998cdff10 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d117fa640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8d000b2bb0 0x7f8d000b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d13fff640 1 --2- 172.21.15.59:0/3462003882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce02f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009fd0 tx=0x7f8d0c0108e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c6d67ee93cf64d7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:17.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d137fe640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8d000b2bb0 0x7f8d000b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d117fa640 1 -- 172.21.15.59:0/337082494 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8cfc103260 con 0x7f8d141022a0 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d09ffb640 1 -- 172.21.15.59:0/3462003882 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c011760 con 0x55f998cdff10 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d09ffb640 1 -- 172.21.15.59:0/3462003882 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0c011db0 con 0x55f998cdff10 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.089+0000 7f8d09ffb640 1 -- 172.21.15.59:0/3462003882 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0c00e3c0 con 0x55f998cdff10 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d1ce14740 1 -- 172.21.15.59:0/3462003882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 msgr2=0x55f998ce02f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d137fe640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8d000b2bb0 0x7f8d000b5070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55f998ce14b0 tx=0x7f8d04005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/3462003882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce02f0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8d0c009fd0 tx=0x7f8d0c0108e0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d1ce14740 1 -- 172.21.15.59:0/3462003882 shutdown_connections 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/3462003882 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998ced4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/3462003882 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998ceabb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/3462003882 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce02f0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d1ce14740 1 -- 172.21.15.59:0/3462003882 >> 172.21.15.59:0/3462003882 conn(0x55f998cdc420 msgr2=0x55f998cdc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d1ce14740 1 -- 172.21.15.59:0/3462003882 shutdown_connections 2024-05-08T05:30:17.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.090+0000 7f8d1ce14740 1 -- 172.21.15.59:0/3462003882 wait complete. 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d1ce14740 1 -- start start 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d1ce14740 1 Processor -- start 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ce0830 0x55f998ce8630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ceb0f0 0x55f998ce8b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f998cedd20 con 0x55f998ce0830 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f998c02370 con 0x55f998ceb0f0 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f998cedea0 con 0x55f998cdff10 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d1ce14740 10 client.0 _ll_register_callbacks cb 0x55f998956a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d137fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d0affd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ce0830 0x55f998ce8630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d137fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce80f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:37994/0 (socket says 172.21.15.59:37994) 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.091+0000 7f8d137fe640 1 -- 172.21.15.59:0/880100098 learned_addr learned my addr 172.21.15.59:0/880100098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:17.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d13fff640 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ceb0f0 0x55f998ce8b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d137fe640 1 -- 172.21.15.59:0/880100098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ceb0f0 msgr2=0x55f998ce8b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d137fe640 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ceb0f0 0x55f998ce8b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d137fe640 1 -- 172.21.15.59:0/880100098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ce0830 msgr2=0x55f998ce8630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d137fe640 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ce0830 0x55f998ce8630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d137fe640 1 -- 172.21.15.59:0/880100098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f998a23620 con 0x55f998cdff10 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d13fff640 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ceb0f0 0x55f998ce8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d137fe640 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce80f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d04003730 tx=0x7f8d040046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6826fa6b8d6b18a7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8cf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/880100098 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0400f860 con 0x55f998cdff10 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/880100098 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8d0400fa00 con 0x55f998cdff10 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d1ce14740 1 -- 172.21.15.59:0/880100098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0c011db0 con 0x55f998cdff10 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/880100098 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d0400fc00 con 0x55f998cdff10 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d1ce14740 10 client.25201 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.092+0000 7f8d1ce14740 1 -- 172.21.15.59:0/880100098 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f998cf9cb0 con 0x55f998cdff10 2024-05-08T05:30:17.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/880100098 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0c011db0 con 0x55f998cdff10 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/880100098 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8d04027020 con 0x55f998cdff10 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8d1ce14740 10 client.25201 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8d1ce14740 10 client.25201 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8d1ce14740 20 client.25201 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f998cfb170 0x55f998cfd560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8d1ce14740 4 client.25201 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8d1ce14740 1 -- 172.21.15.59:0/880100098 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f998c531e0 con 0x55f998cfb170 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8d0affd640 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f998cfb170 0x55f998cfd560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8d0affd640 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f998cfb170 0x55f998cfd560 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9e07f969c31f281 server_cookie=349cc613eeae370c in_seq=0 out_seq=0 2024-05-08T05:30:17.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.093+0000 7f8cf2ffd640 10 client.25201 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/880100098 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f998c531e0 con 0x55f998cfb170 2024-05-08T05:30:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8cf2ffd640 10 client.25201 handle_command_reply: tid=0 2024-05-08T05:30:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8d1ce14740 1 client.25201 shutdown 2024-05-08T05:30:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8d1ce14740 1 -- 172.21.15.59:0/880100098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 msgr2=0x55f998ce80f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:17.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce80f0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f8d04003730 tx=0x7f8d040046c0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8d1ce14740 1 -- 172.21.15.59:0/880100098 shutdown_connections 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f998cfb170 0x55f998cfd560 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ceb0f0 0x55f998ce8b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ce0830 0x55f998ce8630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/880100098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce80f0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.094+0000 7f8cf2ffd640 0 client.25201 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.095+0000 7f8d1ce14740 1 -- 172.21.15.59:0/880100098 >> 172.21.15.59:0/880100098 conn(0x55f998cdc420 msgr2=0x55f998ce2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.095+0000 7f8d1ce14740 1 -- 172.21.15.59:0/880100098 shutdown_connections 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.095+0000 7f8d1ce14740 1 -- 172.21.15.59:0/880100098 wait complete. 2024-05-08T05:30:17.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.095+0000 7f8d1ce14740 20 client.25201 trim_cache size 0 max 16384 2024-05-08T05:30:17.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d1ce14740 1 Processor -- start 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d1ce14740 1 -- start start 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998d02e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998d03390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141afdf0 con 0x55f998ceb0f0 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141b1930 con 0x55f998ce0830 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141b04c0 con 0x55f998cdff10 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d13fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998d03390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d13fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998d03390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55592/0 (socket says 172.21.15.59:55592) 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.110+0000 7f8d13fff640 1 -- 172.21.15.59:0/1148250518 learned_addr learned my addr 172.21.15.59:0/1148250518 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d137fe640 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d0affd640 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998d02e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d137fe640 1 -- 172.21.15.59:0/1148250518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 msgr2=0x55f998d02e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d137fe640 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998d02e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d137fe640 1 -- 172.21.15.59:0/1148250518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 msgr2=0x55f998d03390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d137fe640 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998d03390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d137fe640 1 -- 172.21.15.59:0/1148250518 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f998cfd890 con 0x55f998cdff10 2024-05-08T05:30:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d13fff640 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998d03390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d137fe640 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce8960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8d04012dc0 tx=0x7f8d04009e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8d176c8cc553a261 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d097fa640 1 -- 172.21.15.59:0/1148250518 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0403b070 con 0x55f998cdff10 2024-05-08T05:30:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d097fa640 1 -- 172.21.15.59:0/1148250518 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0400f860 con 0x55f998cdff10 2024-05-08T05:30:17.114 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.111+0000 7f8d097fa640 1 -- 172.21.15.59:0/1148250518 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d04005400 con 0x55f998cdff10 2024-05-08T05:30:17.117 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 -- 172.21.15.59:0/1148250518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 msgr2=0x55f998ce8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce8960 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8d04012dc0 tx=0x7f8d04009e40 comp rx=0 tx=0).stop 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 -- 172.21.15.59:0/1148250518 shutdown_connections 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998ceb0f0 0x55f998d03390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998d02e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/1148250518 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998cdff10 0x55f998ce8960 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 -- 172.21.15.59:0/1148250518 >> 172.21.15.59:0/1148250518 conn(0x55f998cdc420 msgr2=0x55f998cdb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 -- 172.21.15.59:0/1148250518 shutdown_connections 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 -- 172.21.15.59:0/1148250518 wait complete. 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 -- start start 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.112+0000 7f8d1ce14740 1 Processor -- start 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998cdff10 0x55f998cef550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.118 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998cefa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d1ce14740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998ceb0f0 0x55f998d5ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141b1260 con 0x55f998cdff10 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141b0b90 con 0x55f998ce0830 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d1ce14740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8d141addb0 con 0x55f998ceb0f0 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d1ce14740 10 client.0 _ll_register_callbacks cb 0x7f8d04009d90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d137fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998cdff10 0x55f998cef550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d137fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998cdff10 0x55f998cef550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55594/0 (socket says 172.21.15.59:55594) 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d137fe640 1 -- 172.21.15.59:0/731097349 learned_addr learned my addr 172.21.15.59:0/731097349 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d13fff640 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998ceb0f0 0x55f998d5ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d0affd640 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998cefa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d137fe640 1 -- 172.21.15.59:0/731097349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998ceb0f0 msgr2=0x55f998d5ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d137fe640 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998ceb0f0 0x55f998d5ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d137fe640 1 -- 172.21.15.59:0/731097349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 msgr2=0x55f998cefa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d137fe640 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998cefa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d137fe640 1 -- 172.21.15.59:0/731097349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f998d5f000 con 0x55f998cdff10 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8d137fe640 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998cdff10 0x55f998cef550 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ffd0 tx=0x7f8d0400fa10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ca6fab9da935f76 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8cf2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.113+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/731097349 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d0403b070 con 0x55f998cdff10 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/731097349 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8d04005140 con 0x55f998cdff10 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d1ce14740 1 -- 172.21.15.59:0/731097349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d0400f860 con 0x55f998cdff10 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/731097349 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f8d04005340 con 0x55f998cdff10 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d1ce14740 10 client.15570 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d1ce14740 1 -- 172.21.15.59:0/731097349 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55f998d663c0 con 0x55f998cdff10 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/731097349 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d0400f860 con 0x55f998cdff10 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/731097349 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f8d04027070 con 0x55f998cdff10 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d1ce14740 10 client.15570 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d1ce14740 10 client.15570 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:17.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d1ce14740 20 client.15570 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f998d669c0 0x55f998d68db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d1ce14740 4 client.15570 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:30:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d1ce14740 1 -- 172.21.15.59:0/731097349 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f998c4d050 con 0x55f998d669c0 2024-05-08T05:30:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.114+0000 7f8d0affd640 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f998d669c0 0x55f998d68db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.115+0000 7f8d0affd640 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f998d669c0 0x55f998d68db0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d93d52192cc0fc3 server_cookie=a3425251a14e34b6 in_seq=0 out_seq=0 2024-05-08T05:30:17.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.115+0000 7f8cf2ffd640 10 client.15570 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8cf2ffd640 1 -- 172.21.15.59:0/731097349 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f998c4d050 con 0x55f998d669c0 2024-05-08T05:30:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8cf2ffd640 10 client.15570 handle_command_reply: tid=0 2024-05-08T05:30:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 client.15570 shutdown 2024-05-08T05:30:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 -- 172.21.15.59:0/731097349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998cdff10 msgr2=0x55f998cef550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998cdff10 0x55f998cef550 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f8d0400ffd0 tx=0x7f8d0400fa10 comp rx=0 tx=0).stop 2024-05-08T05:30:17.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 -- 172.21.15.59:0/731097349 shutdown_connections 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f998d669c0 0x55f998d68db0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f998ceb0f0 0x55f998d5ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f998ce0830 0x55f998cefa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8cf2ffd640 0 client.15570 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 --2- 172.21.15.59:0/731097349 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f998cdff10 0x55f998cef550 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 -- 172.21.15.59:0/731097349 >> 172.21.15.59:0/731097349 conn(0x55f998cdc420 msgr2=0x55f998ce18c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.118+0000 7f8d1ce14740 1 -- 172.21.15.59:0/731097349 shutdown_connections 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1ce14740 1 -- 172.21.15.59:0/731097349 wait complete. 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1ce14740 20 client.15570 trim_cache size 0 max 16384 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:30:17.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:30:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:30:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:30:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8d000b2bb0 msgr2=0x7f8d000b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8d000b2bb0 0x7f8d000b5070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x55f998ce14b0 tx=0x7f8d04005f90 comp rx=0 tx=0).stop 2024-05-08T05:30:17.124 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141022a0 msgr2=0x7f8d141996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141022a0 0x7f8d141996d0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f8cfc005ab0 tx=0x7f8cfc003b90 comp rx=0 tx=0).stop 2024-05-08T05:30:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/337082494 shutdown_connections 2024-05-08T05:30:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8d000b2bb0 0x7f8d000b5070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8d141078a0 0x7f8d1419da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8d141022a0 0x7f8d141996d0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 --2- 172.21.15.59:0/337082494 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8d141018e0 0x7f8d14199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/337082494 >> 172.21.15.59:0/337082494 conn(0x7f8d14078a30 msgr2=0x7f8d1406a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/337082494 shutdown_connections 2024-05-08T05:30:17.125 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:17.119+0000 7f8d1a0fc640 1 -- 172.21.15.59:0/337082494 wait complete. 2024-05-08T05:30:17.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:17 smithi143 ceph-mon[34337]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-08T05:30:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:17 smithi096 ceph-mon[34253]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-08T05:30:17.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:17 smithi059 ceph-mon[30304]: pgmap v24: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-08T05:30:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:19 smithi143 ceph-mon[34337]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:30:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:19 smithi096 ceph-mon[34253]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:30:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:19 smithi059 ceph-mon[30304]: pgmap v25: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:30:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:21 smithi143 ceph-mon[34337]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:30:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:21 smithi096 ceph-mon[34253]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:30:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:21 smithi059 ceph-mon[30304]: pgmap v26: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:30:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:23 smithi143 ceph-mon[34337]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-08T05:30:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:23 smithi096 ceph-mon[34253]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-08T05:30:23.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:23 smithi059 ceph-mon[30304]: pgmap v27: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 9 op/s 2024-05-08T05:30:25.408 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:25 smithi059 ceph-mon[30304]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:30:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:25 smithi143 ceph-mon[34337]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:30:25.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:25 smithi096 ceph-mon[34253]: pgmap v28: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:30:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:27 smithi143 ceph-mon[34337]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:30:27.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:27 smithi096 ceph-mon[34253]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:30:27.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:27 smithi059 ceph-mon[30304]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:30:29.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:29 smithi143 ceph-mon[34337]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:30:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:29 smithi096 ceph-mon[34253]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:30:29.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:29 smithi059 ceph-mon[30304]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:30:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.223+0000 7fa15cd2b640 1 -- 172.21.15.59:0/2909312981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 msgr2=0x7fa15810ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.223+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/2909312981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 0x7fa15810ca60 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fa14c009ca0 tx=0x7fa14c02a950 comp rx=0 tx=0).stop 2024-05-08T05:30:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.223+0000 7fa15cd2b640 1 -- 172.21.15.59:0/2909312981 shutdown_connections 2024-05-08T05:30:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.223+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/2909312981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15810d030 0x7fa15810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.223+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/2909312981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 0x7fa15810ca60 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.223+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/2909312981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa158103c60 0x7fa158104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.223+0000 7fa15cd2b640 1 -- 172.21.15.59:0/2909312981 >> 172.21.15.59:0/2909312981 conn(0x7fa158077c30 msgr2=0x7fa158078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.223+0000 7fa15cd2b640 1 -- 172.21.15.59:0/2909312981 shutdown_connections 2024-05-08T05:30:30.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.223+0000 7fa15cd2b640 1 -- 172.21.15.59:0/2909312981 wait complete. 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.224+0000 7fa15cd2b640 1 Processor -- start 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.224+0000 7fa15cd2b640 1 -- start start 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.224+0000 7fa15cd2b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa158103c60 0x7fa158199090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.224+0000 7fa15cd2b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 0x7fa1581995d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.224+0000 7fa15cd2b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15810d030 0x7fa15819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.224+0000 7fa15cd2b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa158112c70 con 0x7fa158104580 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.224+0000 7fa15cd2b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa158112af0 con 0x7fa158103c60 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.224+0000 7fa15cd2b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa158112df0 con 0x7fa15810d030 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.224+0000 7fa155d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 0x7fa1581995d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa155d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 0x7fa1581995d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43716/0 (socket says 172.21.15.59:43716) 2024-05-08T05:30:30.227 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa155d9b640 1 -- 172.21.15.59:0/1758025377 learned_addr learned my addr 172.21.15.59:0/1758025377 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa15659c640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa158103c60 0x7fa158199090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa156d9d640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15810d030 0x7fa15819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa155d9b640 1 -- 172.21.15.59:0/1758025377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15810d030 msgr2=0x7fa15819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa155d9b640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15810d030 0x7fa15819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa155d9b640 1 -- 172.21.15.59:0/1758025377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa158103c60 msgr2=0x7fa158199090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa155d9b640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa158103c60 0x7fa158199090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa155d9b640 1 -- 172.21.15.59:0/1758025377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa15819e080 con 0x7fa158104580 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa156d9d640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15810d030 0x7fa15819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa155d9b640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 0x7fa1581995d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa14c005cc0 tx=0x7fa14c02f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa1477fe640 1 -- 172.21.15.59:0/1758025377 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c034040 con 0x7fa158104580 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa1477fe640 1 -- 172.21.15.59:0/1758025377 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa14c003e40 con 0x7fa158104580 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa15cd2b640 1 -- 172.21.15.59:0/1758025377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa15819e310 con 0x7fa158104580 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa1477fe640 1 -- 172.21.15.59:0/1758025377 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa14c045480 con 0x7fa158104580 2024-05-08T05:30:30.228 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.225+0000 7fa15cd2b640 1 -- 172.21.15.59:0/1758025377 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1581a5c00 con 0x7fa158104580 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.228+0000 7fa1477fe640 1 -- 172.21.15.59:0/1758025377 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7fa14c037020 con 0x7fa158104580 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.228+0000 7fa15fa49740 1 Processor -- start 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.228+0000 7fa15fa49740 1 -- start start 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.228+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20302f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.228+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e203abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.228+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e203d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.228+0000 7fa15fa49740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5645e1f955d0 con 0x5645e202ff10 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.228+0000 7fa15fa49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5645e1f4e890 con 0x5645e2030830 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.228+0000 7fa15fa49740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5645e1f91590 con 0x5645e203b0f0 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20302f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20302f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43728/0 (socket says 172.21.15.59:43728) 2024-05-08T05:30:30.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 -- 172.21.15.59:0/641968962 learned_addr learned my addr 172.21.15.59:0/641968962 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa15659c640 1 --2- 172.21.15.59:0/641968962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e203abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa155d9b640 1 --2- 172.21.15.59:0/641968962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e203d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 -- 172.21.15.59:0/641968962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 msgr2=0x5645e203d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 --2- 172.21.15.59:0/641968962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e203d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa1477fe640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1400b2bb0 0x7fa1400b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 -- 172.21.15.59:0/641968962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 msgr2=0x5645e203abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 --2- 172.21.15.59:0/641968962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e203abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 -- 172.21.15.59:0/641968962 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e1ca7210 con 0x5645e202ff10 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa1477fe640 1 -- 172.21.15.59:0/1758025377 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa14c103db0 con 0x7fa158104580 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1400b2bb0 0x7fa1400b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.229+0000 7fa156d9d640 1 --2- 172.21.15.59:0/641968962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20302f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa15000bab0 tx=0x7fa15000bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed75f7fc0203688e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.230+0000 7fa144ff9640 1 -- 172.21.15.59:0/641968962 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150019590 con 0x5645e202ff10 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.230+0000 7fa144ff9640 1 -- 172.21.15.59:0/641968962 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa15000fdf0 con 0x5645e202ff10 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.230+0000 7fa144ff9640 1 -- 172.21.15.59:0/641968962 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa150018940 con 0x5645e202ff10 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.230+0000 7fa15fa49740 1 -- 172.21.15.59:0/641968962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 msgr2=0x5645e20302f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:30.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.230+0000 7fa156d9d640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1400b2bb0 0x7fa1400b5070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa15000dfd0 tx=0x7fa150020000 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.230+0000 7fa15fa49740 1 --2- 172.21.15.59:0/641968962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20302f0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fa15000bab0 tx=0x7fa15000bf70 comp rx=0 tx=0).stop 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.230+0000 7fa15fa49740 1 -- 172.21.15.59:0/641968962 shutdown_connections 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.230+0000 7fa15fa49740 1 --2- 172.21.15.59:0/641968962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e203d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.230+0000 7fa15fa49740 1 --2- 172.21.15.59:0/641968962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e203abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 --2- 172.21.15.59:0/641968962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20302f0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 -- 172.21.15.59:0/641968962 >> 172.21.15.59:0/641968962 conn(0x5645e202c420 msgr2=0x5645e202c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 -- 172.21.15.59:0/641968962 shutdown_connections 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 -- 172.21.15.59:0/641968962 wait complete. 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 -- start start 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 Processor -- start 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e202ff10 0x5645e20380a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e2030830 0x5645e20385e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.238 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e2038b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5645e203dd20 con 0x5645e2030830 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5645e1f52370 con 0x5645e202ff10 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5645e203dea0 con 0x5645e203b0f0 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa15fa49740 10 client.0 _ll_register_callbacks cb 0x5645e1ca6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.231+0000 7fa155d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e2030830 0x5645e20385e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e202ff10 0x5645e20380a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15659c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e202ff10 0x5645e20380a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:35702/0 (socket says 172.21.15.59:35702) 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa155d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e2030830 0x5645e20385e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43730/0 (socket says 172.21.15.59:43730) 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15659c640 1 -- 172.21.15.59:0/1798708926 learned_addr learned my addr 172.21.15.59:0/1798708926 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa156d9d640 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e2038b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15659c640 1 -- 172.21.15.59:0/1798708926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 msgr2=0x5645e2038b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15659c640 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e2038b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15659c640 1 -- 172.21.15.59:0/1798708926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e2030830 msgr2=0x5645e20385e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15659c640 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e2030830 0x5645e20385e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15659c640 1 -- 172.21.15.59:0/1798708926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e1d73620 con 0x5645e202ff10 2024-05-08T05:30:30.239 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa155d9b640 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e2030830 0x5645e20385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa156d9d640 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e2038b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15659c640 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e202ff10 0x5645e20380a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa14800ce00 tx=0x7fa1480039c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f8912af922f7e9ba server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa13dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa13dffb640 1 -- 172.21.15.59:0/1798708926 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148011970 con 0x5645e202ff10 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa13dffb640 1 -- 172.21.15.59:0/1798708926 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa148011b10 con 0x5645e202ff10 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15fa49740 1 -- 172.21.15.59:0/1798708926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa15000fdf0 con 0x5645e202ff10 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa13dffb640 1 -- 172.21.15.59:0/1798708926 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa148011d10 con 0x5645e202ff10 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15fa49740 10 client.25241 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.232+0000 7fa15fa49740 1 -- 172.21.15.59:0/1798708926 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645e2049c10 con 0x5645e202ff10 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.233+0000 7fa13dffb640 1 -- 172.21.15.59:0/1798708926 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa15000fdf0 con 0x5645e202ff10 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.233+0000 7fa13dffb640 1 -- 172.21.15.59:0/1798708926 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa148018070 con 0x5645e202ff10 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.233+0000 7fa15fa49740 10 client.25241 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.233+0000 7fa15fa49740 10 client.25241 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.233+0000 7fa15fa49740 20 client.25241 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:30.240 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.233+0000 7fa15fa49740 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5645e204b120 0x5645e204d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.233+0000 7fa15fa49740 4 client.25241 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.233+0000 7fa15fa49740 1 -- 172.21.15.59:0/1798708926 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645e1fa31e0 con 0x5645e204b120 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.233+0000 7fa155d9b640 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5645e204b120 0x5645e204d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa13dffb640 10 client.25241 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa155d9b640 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5645e204b120 0x5645e204d570 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1481b704ce26cd22 server_cookie=1fe1131a63d67783 in_seq=0 out_seq=0 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa13dffb640 1 -- 172.21.15.59:0/1798708926 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5645e1fa31e0 con 0x5645e204b120 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa13dffb640 10 client.25241 handle_command_reply: tid=0 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa15fa49740 1 client.25241 shutdown 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa15fa49740 1 -- 172.21.15.59:0/1798708926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e202ff10 msgr2=0x5645e20380a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa15fa49740 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e202ff10 0x5645e20380a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa14800ce00 tx=0x7fa1480039c0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa15fa49740 1 -- 172.21.15.59:0/1798708926 shutdown_connections 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa15fa49740 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5645e204b120 0x5645e204d570 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa15fa49740 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e2038b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa15fa49740 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e2030830 0x5645e20385e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa15fa49740 1 --2- 172.21.15.59:0/1798708926 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e202ff10 0x5645e20380a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa13dffb640 0 client.25241 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.234+0000 7fa15fa49740 1 -- 172.21.15.59:0/1798708926 >> 172.21.15.59:0/1798708926 conn(0x5645e202c420 msgr2=0x5645e20321a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.235+0000 7fa15fa49740 1 -- 172.21.15.59:0/1798708926 shutdown_connections 2024-05-08T05:30:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.235+0000 7fa15fa49740 1 -- 172.21.15.59:0/1798708926 wait complete. 2024-05-08T05:30:30.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.235+0000 7fa15fa49740 20 client.25241 trim_cache size 0 max 16384 2024-05-08T05:30:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15fa49740 1 Processor -- start 2024-05-08T05:30:30.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15fa49740 1 -- start start 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e2038940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2052e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e20533c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15fa49740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581afbd0 con 0x5645e202ff10 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15fa49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581b1710 con 0x5645e2030830 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15fa49740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581b02a0 con 0x5645e203b0f0 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e2038940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e2038940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43738/0 (socket says 172.21.15.59:43738) 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa15659c640 1 -- 172.21.15.59:0/544717046 learned_addr learned my addr 172.21.15.59:0/544717046 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.250+0000 7fa156d9d640 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e20533c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa155d9b640 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2052e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa15659c640 1 -- 172.21.15.59:0/544717046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 msgr2=0x5645e20533c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa15659c640 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e20533c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa15659c640 1 -- 172.21.15.59:0/544717046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 msgr2=0x5645e2052e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa15659c640 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2052e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa15659c640 1 -- 172.21.15.59:0/544717046 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645e20365f0 con 0x5645e202ff10 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa156d9d640 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e20533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa155d9b640 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2052e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa15659c640 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e2038940 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa148006010 tx=0x7fa1480199b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8364f000a1fff3f7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa13ffff640 1 -- 172.21.15.59:0/544717046 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148019d70 con 0x5645e202ff10 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa13ffff640 1 -- 172.21.15.59:0/544717046 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa14802d070 con 0x5645e202ff10 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.251+0000 7fa13ffff640 1 -- 172.21.15.59:0/544717046 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1480055a0 con 0x5645e202ff10 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 -- 172.21.15.59:0/544717046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 msgr2=0x5645e2038940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:30.257 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e2038940 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fa148006010 tx=0x7fa1480199b0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 -- 172.21.15.59:0/544717046 shutdown_connections 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e20533c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2052e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 --2- 172.21.15.59:0/544717046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e2038940 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 -- 172.21.15.59:0/544717046 >> 172.21.15.59:0/544717046 conn(0x5645e202c420 msgr2=0x5645e20273e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 -- 172.21.15.59:0/544717046 shutdown_connections 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 -- 172.21.15.59:0/544717046 wait complete. 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 -- start start 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 Processor -- start 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20432e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2034930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e2034e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581b1040 con 0x5645e202ff10 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581b0970 con 0x5645e2030830 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1581adb90 con 0x5645e203b0f0 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15fa49740 10 client.0 _ll_register_callbacks cb 0x7fa148004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20432e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20432e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43740/0 (socket says 172.21.15.59:43740) 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.252+0000 7fa15659c640 1 -- 172.21.15.59:0/3155689063 learned_addr learned my addr 172.21.15.59:0/3155689063 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa155d9b640 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2034930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa15659c640 1 -- 172.21.15.59:0/3155689063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 msgr2=0x5645e2034e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa156d9d640 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e2034e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa15659c640 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e2034e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa15659c640 1 -- 172.21.15.59:0/3155689063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 msgr2=0x5645e2034930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa15659c640 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2034930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa15659c640 1 -- 172.21.15.59:0/3155689063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645e20353b0 con 0x5645e202ff10 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa155d9b640 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2034930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa15659c640 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20432e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa14800ce00 tx=0x7fa148019470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c81c08eb30ae32ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa13dffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa13dffb640 1 -- 172.21.15.59:0/3155689063 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa148005320 con 0x5645e202ff10 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa13dffb640 1 -- 172.21.15.59:0/3155689063 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa1480054c0 con 0x5645e202ff10 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa15fa49740 1 -- 172.21.15.59:0/3155689063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa14802d070 con 0x5645e202ff10 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa13dffb640 1 -- 172.21.15.59:0/3155689063 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1480056c0 con 0x5645e202ff10 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa15fa49740 10 client.15588 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.253+0000 7fa15fa49740 1 -- 172.21.15.59:0/3155689063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5645e20b5630 con 0x5645e202ff10 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa13dffb640 1 -- 172.21.15.59:0/3155689063 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa14802d070 con 0x5645e202ff10 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa13dffb640 1 -- 172.21.15.59:0/3155689063 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa148018030 con 0x5645e202ff10 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa15fa49740 10 client.15588 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa15fa49740 10 client.15588 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa15fa49740 20 client.15588 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa15fa49740 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5645e20b5ca0 0x5645e20b80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:30.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa15fa49740 4 client.15588 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:30:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa15fa49740 1 -- 172.21.15.59:0/3155689063 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5645e1f9d050 con 0x5645e20b5ca0 2024-05-08T05:30:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa155d9b640 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5645e20b5ca0 0x5645e20b80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa155d9b640 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5645e20b5ca0 0x5645e20b80c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d590c10cceaf224 server_cookie=df02e3067105689f in_seq=0 out_seq=0 2024-05-08T05:30:30.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.254+0000 7fa13dffb640 10 client.15588 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.258+0000 7fa13dffb640 1 -- 172.21.15.59:0/3155689063 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5645e1f9d050 con 0x5645e20b5ca0 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.258+0000 7fa13dffb640 10 client.15588 handle_command_reply: tid=0 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.258+0000 7fa15fa49740 1 client.15588 shutdown 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 -- 172.21.15.59:0/3155689063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 msgr2=0x5645e20432e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20432e0 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa14800ce00 tx=0x7fa148019470 comp rx=0 tx=0).stop 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 -- 172.21.15.59:0/3155689063 shutdown_connections 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5645e20b5ca0 0x5645e20b80c0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5645e203b0f0 0x5645e2034e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5645e2030830 0x5645e2034930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa13dffb640 0 client.15588 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 --2- 172.21.15.59:0/3155689063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5645e202ff10 0x5645e20432e0 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 -- 172.21.15.59:0/3155689063 >> 172.21.15.59:0/3155689063 conn(0x5645e202c420 msgr2=0x5645e202acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 -- 172.21.15.59:0/3155689063 shutdown_connections 2024-05-08T05:30:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 1 -- 172.21.15.59:0/3155689063 wait complete. 2024-05-08T05:30:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15fa49740 20 client.15588 trim_cache size 0 max 16384 2024-05-08T05:30:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:30:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:30:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:30:30.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:30:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 -- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1400b2bb0 msgr2=0x7fa1400b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1400b2bb0 0x7fa1400b5070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa15000dfd0 tx=0x7fa150020000 comp rx=0 tx=0).stop 2024-05-08T05:30:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 -- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 msgr2=0x7fa1581995d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 0x7fa1581995d0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fa14c005cc0 tx=0x7fa14c02f650 comp rx=0 tx=0).stop 2024-05-08T05:30:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 -- 172.21.15.59:0/1758025377 shutdown_connections 2024-05-08T05:30:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1400b2bb0 0x7fa1400b5070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa15810d030 0x7fa15819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa158104580 0x7fa1581995d0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 --2- 172.21.15.59:0/1758025377 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa158103c60 0x7fa158199090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.259+0000 7fa15cd2b640 1 -- 172.21.15.59:0/1758025377 >> 172.21.15.59:0/1758025377 conn(0x7fa158077c30 msgr2=0x7fa1580fe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.260+0000 7fa15cd2b640 1 -- 172.21.15.59:0/1758025377 shutdown_connections 2024-05-08T05:30:30.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:30.260+0000 7fa15cd2b640 1 -- 172.21.15.59:0/1758025377 wait complete. 2024-05-08T05:30:31.361 INFO:tasks.quiescer.fs.[cephfs]:Total duration (38.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-08T05:30:31.361 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-05-08T05:30:31.362 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:30:31.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:31 smithi143 ceph-mon[34337]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:30:31.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:31 smithi096 ceph-mon[34253]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:30:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:31 smithi059 ceph-mon[30304]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:30:31.631 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:30:33.244 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:33 smithi059 ceph-mon[30304]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:30:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.490+0000 7ff669312700 1 -- 172.21.15.59:0/806604669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 msgr2=0x7ff6641022e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:33.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.490+0000 7ff669312700 1 --2- 172.21.15.59:0/806604669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 0x7ff6641022e0 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009990 tx=0x7ff64c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.490+0000 7ff669312700 1 -- 172.21.15.59:0/806604669 shutdown_connections 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.490+0000 7ff669312700 1 --2- 172.21.15.59:0/806604669 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff664102820 0x7ff664110290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.490+0000 7ff669312700 1 --2- 172.21.15.59:0/806604669 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff664108580 0x7ff664108960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.490+0000 7ff669312700 1 --2- 172.21.15.59:0/806604669 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 0x7ff6641022e0 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.490+0000 7ff669312700 1 -- 172.21.15.59:0/806604669 >> 172.21.15.59:0/806604669 conn(0x7ff6640776c0 msgr2=0x7ff664077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.490+0000 7ff669312700 1 -- 172.21.15.59:0/806604669 shutdown_connections 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.491+0000 7ff669312700 1 -- 172.21.15.59:0/806604669 wait complete. 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.491+0000 7ff669312700 1 Processor -- start 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.491+0000 7ff669312700 1 -- start start 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff669312700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 0x7ff66419f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:33.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff669312700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff664102820 0x7ff66419fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff669312700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff664108580 0x7ff6641998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff669312700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff6641136c0 con 0x7ff664101e60 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff669312700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff664113540 con 0x7ff664102820 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff669312700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff664113840 con 0x7ff664108580 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff662ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 0x7ff66419f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff662ffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 0x7ff66419f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43748/0 (socket says 172.21.15.59:43748) 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff662ffd700 1 -- 172.21.15.59:0/2106453713 learned_addr learned my addr 172.21.15.59:0/2106453713 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff6627fc700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff664102820 0x7ff66419fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff6637fe700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff664108580 0x7ff6641998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff662ffd700 1 -- 172.21.15.59:0/2106453713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff664108580 msgr2=0x7ff6641998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:33.495 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff662ffd700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff664108580 0x7ff6641998e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff662ffd700 1 -- 172.21.15.59:0/2106453713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff664102820 msgr2=0x7ff66419fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:33.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff662ffd700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff664102820 0x7ff66419fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff662ffd700 1 -- 172.21.15.59:0/2106453713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff66419a040 con 0x7ff664101e60 2024-05-08T05:30:33.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff662ffd700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 0x7ff66419f720 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff65400ada0 tx=0x7ff65400c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:33.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.492+0000 7ff65bfff700 1 -- 172.21.15.59:0/2106453713 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654017020 con 0x7ff664101e60 2024-05-08T05:30:33.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.493+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff66419a330 con 0x7ff664101e60 2024-05-08T05:30:33.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.493+0000 7ff65bfff700 1 -- 172.21.15.59:0/2106453713 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff65400c980 con 0x7ff664101e60 2024-05-08T05:30:33.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.493+0000 7ff65bfff700 1 -- 172.21.15.59:0/2106453713 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff654004280 con 0x7ff664101e60 2024-05-08T05:30:33.496 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.493+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff6641a65f0 con 0x7ff664101e60 2024-05-08T05:30:33.497 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:33 smithi143 ceph-mon[34337]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:30:33.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.494+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff66406a190 con 0x7ff664101e60 2024-05-08T05:30:33.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.496+0000 7ff65bfff700 1 -- 172.21.15.59:0/2106453713 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7ff654004ad0 con 0x7ff664101e60 2024-05-08T05:30:33.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.497+0000 7ff65bfff700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6500b2ba0 0x7ff6500b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:33.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.497+0000 7ff6627fc700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6500b2ba0 0x7ff6500b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:33.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.497+0000 7ff65bfff700 1 -- 172.21.15.59:0/2106453713 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ff6540e25e0 con 0x7ff664101e60 2024-05-08T05:30:33.498 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.497+0000 7ff6627fc700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6500b2ba0 0x7ff6500b5060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009750 tx=0x7ff64c005fd0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:33.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.500+0000 7ff65bfff700 1 -- 172.21.15.59:0/2106453713 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff654021020 con 0x7ff664101e60 2024-05-08T05:30:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:33 smithi096 ceph-mon[34253]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 9 op/s 2024-05-08T05:30:33.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.769+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff66402e510 con 0x7ff664101e60 2024-05-08T05:30:33.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.770+0000 7ff65bfff700 1 -- 172.21.15.59:0/2106453713 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7ff6540a9ae0 con 0x7ff664101e60 2024-05-08T05:30:33.771 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:30:33.771 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:29:31.333500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:30:33.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.772+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6500b2ba0 msgr2=0x7ff6500b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:33.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.772+0000 7ff669312700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6500b2ba0 0x7ff6500b5060 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff64c009750 tx=0x7ff64c005fd0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.772+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 msgr2=0x7ff66419f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:33.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.772+0000 7ff669312700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 0x7ff66419f720 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7ff65400ada0 tx=0x7ff65400c330 comp rx=0 tx=0).stop 2024-05-08T05:30:33.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.772+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 shutdown_connections 2024-05-08T05:30:33.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.772+0000 7ff669312700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6500b2ba0 0x7ff6500b5060 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.773+0000 7ff669312700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff664108580 0x7ff6641998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.773+0000 7ff669312700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff664101e60 0x7ff66419f720 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.773+0000 7ff669312700 1 --2- 172.21.15.59:0/2106453713 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff664102820 0x7ff66419fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:33.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.773+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 >> 172.21.15.59:0/2106453713 conn(0x7ff6640776c0 msgr2=0x7ff664100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:33.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.773+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 shutdown_connections 2024-05-08T05:30:33.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:33.773+0000 7ff669312700 1 -- 172.21.15.59:0/2106453713 wait complete. 2024-05-08T05:30:33.774 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 30 2024-05-08T05:30:34.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:34 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2106453713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:30:34.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:34 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2106453713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:30:34.580 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:34 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2106453713' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:30:34.617 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db / --timeout 60 --expiration 79.4 --await-for 42.0' 2024-05-08T05:30:34.617 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db / --timeout 60 --expiration 79.4 --await-for 42.0 2024-05-08T05:30:34.883 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:30:35.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:35 smithi059 ceph-mon[30304]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:30:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:35 smithi143 ceph-mon[34337]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:30:35.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:35 smithi096 ceph-mon[34253]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:30:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.554+0000 7f96fcf01700 1 -- 172.21.15.59:0/1727426627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 msgr2=0x7f96f8110390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.554+0000 7f96fcf01700 1 --2- 172.21.15.59:0/1727426627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 0x7f96f8110390 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f96f00099c0 tx=0x7f96f0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.554+0000 7f96fcf01700 1 -- 172.21.15.59:0/1727426627 shutdown_connections 2024-05-08T05:30:36.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.554+0000 7f96fcf01700 1 --2- 172.21.15.59:0/1727426627 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 0x7f96f8110390 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.554+0000 7f96fcf01700 1 --2- 172.21.15.59:0/1727426627 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f8108490 0x7f96f8108870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.554+0000 7f96fcf01700 1 --2- 172.21.15.59:0/1727426627 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81034f0 0x7f96f8103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.554+0000 7f96fcf01700 1 -- 172.21.15.59:0/1727426627 >> 172.21.15.59:0/1727426627 conn(0x7f96f807a650 msgr2=0x7f96f807aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.554+0000 7f96fcf01700 1 -- 172.21.15.59:0/1727426627 shutdown_connections 2024-05-08T05:30:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.554+0000 7f96fcf01700 1 -- 172.21.15.59:0/1727426627 wait complete. 2024-05-08T05:30:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.555+0000 7f96fcf01700 1 Processor -- start 2024-05-08T05:30:36.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.555+0000 7f96fcf01700 1 -- start start 2024-05-08T05:30:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.555+0000 7f96fcf01700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81034f0 0x7f96f81a1880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.555+0000 7f96fcf01700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 0x7f96f81a1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.555+0000 7f96fcf01700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f8108490 0x7f96f819ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.555+0000 7f96fcf01700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96f81136a0 con 0x7f96f8103eb0 2024-05-08T05:30:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.555+0000 7f96fcf01700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8113520 con 0x7f96f81034f0 2024-05-08T05:30:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.555+0000 7f96fcf01700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f96f8113820 con 0x7f96f8108490 2024-05-08T05:30:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f5d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 0x7f96f81a1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.558 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f5d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 0x7f96f81a1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43778/0 (socket says 172.21.15.59:43778) 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f5d9b700 1 -- 172.21.15.59:0/514849107 learned_addr learned my addr 172.21.15.59:0/514849107 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f659c700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81034f0 0x7f96f81a1880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f6d9d700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f8108490 0x7f96f819ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f5d9b700 1 -- 172.21.15.59:0/514849107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f8108490 msgr2=0x7f96f819ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f8108490 0x7f96f819ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f5d9b700 1 -- 172.21.15.59:0/514849107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81034f0 msgr2=0x7f96f81a1880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81034f0 0x7f96f81a1880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f5d9b700 1 -- 172.21.15.59:0/514849107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96f819c1a0 con 0x7f96f8103eb0 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f659c700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81034f0 0x7f96f81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 0x7f96f81a1dc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f96e000c920 tx=0x7f96e000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96ef7fe700 1 -- 172.21.15.59:0/514849107 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0007b00 con 0x7f96f8103eb0 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96fcf01700 1 -- 172.21.15.59:0/514849107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96f819c490 con 0x7f96f8103eb0 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96ef7fe700 1 -- 172.21.15.59:0/514849107 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96e0004ce0 con 0x7f96f8103eb0 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.556+0000 7f96ef7fe700 1 -- 172.21.15.59:0/514849107 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e0005550 con 0x7f96f8103eb0 2024-05-08T05:30:36.559 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.557+0000 7f96fcf01700 1 -- 172.21.15.59:0/514849107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96f81a8750 con 0x7f96f8103eb0 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.559+0000 7f96ef7fe700 1 -- 172.21.15.59:0/514849107 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f96e0007ca0 con 0x7f96f8103eb0 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.560+0000 7f96ef7fe700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f96e40b2c40 0x7f96e40b5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.560+0000 7f96f659c700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f96e40b2c40 0x7f96e40b5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.560+0000 7f96ef7fe700 1 -- 172.21.15.59:0/514849107 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f96e00e2b20 con 0x7f96f8103eb0 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.560+0000 7f9707794b80 1 Processor -- start 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.560+0000 7f96f659c700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f96e40b2c40 0x7f96e40b5100 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f96f819cb80 tx=0x7f96e80091b0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.560+0000 7f9707794b80 1 -- start start 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f9707794b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f9e850 0x5564f3f9ec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f9707794b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f98b00 0x5564f3f98f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f9707794b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f994c0 0x5564f3fa5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f9707794b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5564f3eeaad0 con 0x5564f3f994c0 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f9707794b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5564f3e4ff30 con 0x5564f3f98b00 2024-05-08T05:30:36.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f9707794b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5564f3ee6a90 con 0x5564f3f9e850 2024-05-08T05:30:36.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f994c0 0x5564f3fa5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f994c0 0x5564f3fa5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43782/0 (socket says 172.21.15.59:43782) 2024-05-08T05:30:36.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f6d9d700 1 -- 172.21.15.59:0/3871452382 learned_addr learned my addr 172.21.15.59:0/3871452382 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:36.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f98b00 0x5564f3f98f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f659c700 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f9e850 0x5564f3f9ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f6d9d700 1 -- 172.21.15.59:0/3871452382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f9e850 msgr2=0x5564f3f9ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:36.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f6d9d700 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f9e850 0x5564f3f9ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f6d9d700 1 -- 172.21.15.59:0/3871452382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f98b00 msgr2=0x5564f3f98f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:36.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f6d9d700 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f98b00 0x5564f3f98f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f6d9d700 1 -- 172.21.15.59:0/3871452382 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f3e37a60 con 0x5564f3f994c0 2024-05-08T05:30:36.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f98b00 0x5564f3f98f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:36.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96f6d9d700 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f994c0 0x5564f3fa5610 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f96f00099c0 tx=0x7f96f0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5cca472754e32db2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.561+0000 7f96ecff9700 1 -- 172.21.15.59:0/3871452382 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0027070 con 0x5564f3f994c0 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f96ecff9700 1 -- 172.21.15.59:0/3871452382 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f0005070 con 0x5564f3f994c0 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f96ecff9700 1 -- 172.21.15.59:0/3871452382 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000ba80 con 0x5564f3f994c0 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f9707794b80 1 -- 172.21.15.59:0/3871452382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f994c0 msgr2=0x5564f3fa5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f9707794b80 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f994c0 0x5564f3fa5610 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f96f00099c0 tx=0x7f96f0005c80 comp rx=0 tx=0).stop 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f9707794b80 1 -- 172.21.15.59:0/3871452382 shutdown_connections 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f9707794b80 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f994c0 0x5564f3fa5610 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f9707794b80 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f9e850 0x5564f3f9ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f9707794b80 1 --2- 172.21.15.59:0/3871452382 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f98b00 0x5564f3f98f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f9707794b80 1 -- 172.21.15.59:0/3871452382 >> 172.21.15.59:0/3871452382 conn(0x5564f3ec4d00 msgr2=0x5564f3f93880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f9707794b80 1 -- 172.21.15.59:0/3871452382 shutdown_connections 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.562+0000 7f9707794b80 1 -- 172.21.15.59:0/3871452382 wait complete. 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f9707794b80 1 -- start start 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f9707794b80 1 Processor -- start 2024-05-08T05:30:36.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f9707794b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 0x5564f3fa5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f9707794b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 0x5564f3fa2500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f9707794b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fa2a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f9707794b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5564f3fa5b50 con 0x5564f3f9e850 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f9707794b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5564f3ea4060 con 0x5564f3f994c0 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f9707794b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5564f3fa5cd0 con 0x5564f3f98b00 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f9707794b80 10 client.0 _ll_register_callbacks cb 0x5564f3e7e740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f96f6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fa2a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f96f5d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 0x5564f3fa2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f96f6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fa2a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43788/0 (socket says 172.21.15.59:43788) 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f96f6d9d700 1 -- 172.21.15.59:0/1010310397 learned_addr learned my addr 172.21.15.59:0/1010310397 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.563+0000 7f96f659c700 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 0x5564f3fa5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96f5d9b700 1 -- 172.21.15.59:0/1010310397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 msgr2=0x5564f3fa5160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 0x5564f3fa5160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96f5d9b700 1 -- 172.21.15.59:0/1010310397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 msgr2=0x5564f3fa2a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fa2a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96f5d9b700 1 -- 172.21.15.59:0/1010310397 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f3ea18f0 con 0x5564f3f994c0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96f6d9d700 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fa2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 0x5564f3fa2500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96e00dfcc0 tx=0x7f96e00a9730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a414068dad2a586 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96ddffb700 1 -- 172.21.15.59:0/1010310397 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e001d040 con 0x5564f3f994c0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f9707794b80 1 -- 172.21.15.59:0/1010310397 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f0005070 con 0x5564f3f994c0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96ddffb700 1 -- 172.21.15.59:0/1010310397 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f96e0004e90 con 0x5564f3f994c0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96ddffb700 1 -- 172.21.15.59:0/1010310397 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96e00a9850 con 0x5564f3f994c0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f9707794b80 10 client.25253 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f9707794b80 1 -- 172.21.15.59:0/1010310397 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5564f3fb0210 con 0x5564f3f994c0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.564+0000 7f96ddffb700 1 -- 172.21.15.59:0/1010310397 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f0005070 con 0x5564f3f994c0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f96ddffb700 1 -- 172.21.15.59:0/1010310397 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f96e006f080 con 0x5564f3f994c0 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f9707794b80 10 client.25253 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f9707794b80 10 client.25253 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:30:36.570 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f9707794b80 20 client.25253 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f9707794b80 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5564f3fb27a0 0x5564f3fb4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f9707794b80 4 client.25253 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f9707794b80 1 -- 172.21.15.59:0/1010310397 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5564f3ef84b0 con 0x5564f3fb27a0 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f96f659c700 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5564f3fb27a0 0x5564f3fb4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f96ddffb700 10 client.25253 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.565+0000 7f96f659c700 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5564f3fb27a0 0x5564f3fb4b90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4e75ae5e664019e5 server_cookie=9c82bc94244cec9 in_seq=0 out_seq=0 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f96ddffb700 1 -- 172.21.15.59:0/1010310397 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5564f3ef84b0 con 0x5564f3fb27a0 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f96ddffb700 10 client.25253 handle_command_reply: tid=0 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f9707794b80 1 client.25253 shutdown 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f9707794b80 1 -- 172.21.15.59:0/1010310397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 msgr2=0x5564f3fa2500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f9707794b80 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 0x5564f3fa2500 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f96e00dfcc0 tx=0x7f96e00a9730 comp rx=0 tx=0).stop 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f9707794b80 1 -- 172.21.15.59:0/1010310397 shutdown_connections 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f9707794b80 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5564f3fb27a0 0x5564f3fb4b90 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.571 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f96ddffb700 0 client.25253 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:36.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f9707794b80 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fa2a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f9707794b80 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 0x5564f3fa5160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f9707794b80 1 --2- 172.21.15.59:0/1010310397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 0x5564f3fa2500 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.566+0000 7f9707794b80 1 -- 172.21.15.59:0/1010310397 >> 172.21.15.59:0/1010310397 conn(0x5564f3ec4d00 msgr2=0x5564f3f9dbd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:36.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.567+0000 7f9707794b80 1 -- 172.21.15.59:0/1010310397 shutdown_connections 2024-05-08T05:30:36.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.567+0000 7f9707794b80 1 -- 172.21.15.59:0/1010310397 wait complete. 2024-05-08T05:30:36.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.567+0000 7f9707794b80 20 client.25253 trim_cache size 0 max 16384 2024-05-08T05:30:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f9707794b80 1 Processor -- start 2024-05-08T05:30:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f9707794b80 1 -- start start 2024-05-08T05:30:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f9707794b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 0x5564f3faeaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f9707794b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 0x5564f3f9da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f9707794b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fba3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f9707794b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96f81b1540 con 0x5564f3f9e850 2024-05-08T05:30:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f9707794b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96f81b3080 con 0x5564f3f994c0 2024-05-08T05:30:36.590 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f9707794b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f96f81b1c10 con 0x5564f3f98b00 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fba3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f6d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fba3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43794/0 (socket says 172.21.15.59:43794) 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f6d9d700 1 -- 172.21.15.59:0/1520626953 learned_addr learned my addr 172.21.15.59:0/1520626953 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f659c700 1 --2- 172.21.15.59:0/1520626953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 0x5564f3faeaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/1520626953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 0x5564f3f9da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f6d9d700 1 -- 172.21.15.59:0/1520626953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 msgr2=0x5564f3faeaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f6d9d700 1 --2- 172.21.15.59:0/1520626953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 0x5564f3faeaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f6d9d700 1 -- 172.21.15.59:0/1520626953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 msgr2=0x5564f3f9da30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f6d9d700 1 --2- 172.21.15.59:0/1520626953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 0x5564f3f9da30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.589+0000 7f96f6d9d700 1 -- 172.21.15.59:0/1520626953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5564f3fa0520 con 0x5564f3f9e850 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f96f6d9d700 1 --2- 172.21.15.59:0/1520626953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fba3f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006010 tx=0x7f96f0009fa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=286df07171d24214 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f96dffff700 1 -- 172.21.15.59:0/1520626953 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f0027030 con 0x5564f3f9e850 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f96dffff700 1 -- 172.21.15.59:0/1520626953 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f0028040 con 0x5564f3f9e850 2024-05-08T05:30:36.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f96dffff700 1 -- 172.21.15.59:0/1520626953 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96f000ba80 con 0x5564f3f9e850 2024-05-08T05:30:36.594 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f9707794b80 1 -- 172.21.15.59:0/1520626953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 msgr2=0x5564f3fba3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f9707794b80 1 --2- 172.21.15.59:0/1520626953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fba3f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f96f0006010 tx=0x7f96f0009fa0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f9707794b80 1 -- 172.21.15.59:0/1520626953 shutdown_connections 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f9707794b80 1 --2- 172.21.15.59:0/1520626953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f9e850 0x5564f3fba3f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f9707794b80 1 --2- 172.21.15.59:0/1520626953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f98b00 0x5564f3faeaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f9707794b80 1 --2- 172.21.15.59:0/1520626953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f994c0 0x5564f3f9da30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.590+0000 7f9707794b80 1 -- 172.21.15.59:0/1520626953 >> 172.21.15.59:0/1520626953 conn(0x5564f3ec4d00 msgr2=0x5564f3f9d320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 -- 172.21.15.59:0/1520626953 shutdown_connections 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 -- 172.21.15.59:0/1520626953 wait complete. 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 -- start start 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 Processor -- start 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f98b00 0x5564f3fab250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f994c0 0x5564f3f9a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.595 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f9e850 0x5564f3f9a6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f96f81b29b0 con 0x5564f3f98b00 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f96f81b22e0 con 0x5564f3f9e850 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f96f81af4c0 con 0x5564f3f994c0 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f9707794b80 10 client.0 _ll_register_callbacks cb 0x5564f3e7e740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f96f659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f98b00 0x5564f3fab250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f96f659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f98b00 0x5564f3fab250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43810/0 (socket says 172.21.15.59:43810) 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f96f659c700 1 -- 172.21.15.59:0/389858772 learned_addr learned my addr 172.21.15.59:0/389858772 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.591+0000 7f96f6d9d700 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f9e850 0x5564f3f9a6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f994c0 0x5564f3f9a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96f659c700 1 -- 172.21.15.59:0/389858772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f994c0 msgr2=0x5564f3f9a180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96f659c700 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f994c0 0x5564f3f9a180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96f659c700 1 -- 172.21.15.59:0/389858772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f9e850 msgr2=0x5564f3f9a6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96f659c700 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f9e850 0x5564f3f9a6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96f659c700 1 -- 172.21.15.59:0/389858772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5564f3f9ad70 con 0x5564f3f98b00 2024-05-08T05:30:36.596 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f994c0 0x5564f3f9a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96f659c700 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f98b00 0x5564f3fab250 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009550 tx=0x7f96e8004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7df5a22f1b2c0ea server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96ddffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96ddffb700 1 -- 172.21.15.59:0/389858772 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f96e801c050 con 0x5564f3f98b00 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f9707794b80 1 -- 172.21.15.59:0/389858772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f96f0028040 con 0x5564f3f98b00 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96ddffb700 1 -- 172.21.15.59:0/389858772 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f96e8004830 con 0x5564f3f98b00 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96ddffb700 1 -- 172.21.15.59:0/389858772 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f96e8003e50 con 0x5564f3f98b00 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f9707794b80 10 client.15612 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f9707794b80 1 -- 172.21.15.59:0/389858772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5564f401dfb0 con 0x5564f3f98b00 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.592+0000 7f96ddffb700 1 -- 172.21.15.59:0/389858772 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96f0028040 con 0x5564f3f98b00 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f96ddffb700 1 -- 172.21.15.59:0/389858772 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f96e802bce0 con 0x5564f3f98b00 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f9707794b80 10 client.15612 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f9707794b80 10 client.15612 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f9707794b80 20 client.15612 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f9707794b80 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5564f3fb27a0 0x5564f40207a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:36.597 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f9707794b80 4 client.15612 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await_for": 42.0}] 2024-05-08T05:30:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f9707794b80 1 -- 172.21.15.59:0/389858772 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await_for": 42.0}) -- 0x5564f3ef2550 con 0x5564f3fb27a0 2024-05-08T05:30:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5564f3fb27a0 0x5564f40207a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f96f5d9b700 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5564f3fb27a0 0x5564f40207a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abc0a2dab907a006 server_cookie=2c872a078994ada4 in_seq=0 out_seq=0 2024-05-08T05:30:36.598 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.593+0000 7f96ddffb700 10 client.15612 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:36.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f96ddffb700 1 -- 172.21.15.59:0/389858772 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5564f3ef2550 con 0x5564f3fb27a0 2024-05-08T05:30:36.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f96ddffb700 10 client.15612 handle_command_reply: tid=0 2024-05-08T05:30:36.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 client.15612 shutdown 2024-05-08T05:30:36.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 -- 172.21.15.59:0/389858772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f98b00 msgr2=0x5564f3fab250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:36.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f98b00 0x5564f3fab250 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f96e8009550 tx=0x7f96e8004b40 comp rx=0 tx=0).stop 2024-05-08T05:30:36.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 -- 172.21.15.59:0/389858772 shutdown_connections 2024-05-08T05:30:36.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5564f3fb27a0 0x5564f40207a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5564f3f9e850 0x5564f3f9a6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5564f3f98b00 0x5564f3fab250 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 --2- 172.21.15.59:0/389858772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5564f3f994c0 0x5564f3f9a180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f96ddffb700 0 client.15612 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 -- 172.21.15.59:0/389858772 >> 172.21.15.59:0/389858772 conn(0x5564f3ec4d00 msgr2=0x5564f3f9fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 -- 172.21.15.59:0/389858772 shutdown_connections 2024-05-08T05:30:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 1 -- 172.21.15.59:0/389858772 wait complete. 2024-05-08T05:30:36.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.603+0000 7f9707794b80 20 client.15612 trim_cache size 0 max 16384 2024-05-08T05:30:36.605 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 30, 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 2, 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "f4dea23a": { 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 2, 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 79.4, 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:30:36.606 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:30:36.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 -- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f96e40b2c40 msgr2=0x7f96e40b5100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f96e40b2c40 0x7f96e40b5100 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f96f819cb80 tx=0x7f96e80091b0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 -- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 msgr2=0x7f96f81a1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 0x7f96f81a1dc0 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f96e000c920 tx=0x7f96e000cc30 comp rx=0 tx=0).stop 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 -- 172.21.15.59:0/514849107 shutdown_connections 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f96e40b2c40 0x7f96e40b5100 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f96f8108490 0x7f96f819ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f96f81034f0 0x7f96f81a1880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 --2- 172.21.15.59:0/514849107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f96f8103eb0 0x7f96f81a1dc0 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 -- 172.21.15.59:0/514849107 >> 172.21.15.59:0/514849107 conn(0x7f96f807a650 msgr2=0x7f96f8101360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 -- 172.21.15.59:0/514849107 shutdown_connections 2024-05-08T05:30:36.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:36.604+0000 7f96fcf01700 1 -- 172.21.15.59:0/514849107 wait complete. 2024-05-08T05:30:37.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:37 smithi059 ceph-mon[30304]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-05-08T05:30:37.431 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'f4dea23a', quiesce took 6.1 seconds. Will release after: 19.4 2024-05-08T05:30:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:37 smithi143 ceph-mon[34337]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-05-08T05:30:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:37 smithi096 ceph-mon[34253]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-05-08T05:30:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:39 smithi143 ceph-mon[34337]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T05:30:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:30:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:39 smithi096 ceph-mon[34253]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T05:30:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:30:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:39 smithi059 ceph-mon[30304]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T05:30:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:30:40.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:40 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:30:40.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:40 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:30:40.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:40 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:30:40.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:40 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:30:40.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:40 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:30:40.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:40 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:30:40.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:40 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:30:40.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:40 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:30:40.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:40 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:30:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:41 smithi143 ceph-mon[34337]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-05-08T05:30:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:41 smithi096 ceph-mon[34253]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-05-08T05:30:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:41 smithi059 ceph-mon[30304]: pgmap v36: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 1.6 KiB/s wr, 13 op/s 2024-05-08T05:30:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.362+0000 7fe8fa179640 1 -- 172.21.15.59:0/21882215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 msgr2=0x7fe8f410ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.362+0000 7fe8fa179640 1 --2- 172.21.15.59:0/21882215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 0x7fe8f410ca60 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8009900 tx=0x7fe8e802a480 comp rx=0 tx=0).stop 2024-05-08T05:30:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.362+0000 7fe8fa179640 1 -- 172.21.15.59:0/21882215 shutdown_connections 2024-05-08T05:30:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.362+0000 7fe8fa179640 1 --2- 172.21.15.59:0/21882215 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f410d030 0x7fe8f410f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.362+0000 7fe8fa179640 1 --2- 172.21.15.59:0/21882215 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 0x7fe8f410ca60 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.362+0000 7fe8fa179640 1 --2- 172.21.15.59:0/21882215 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8f4103c60 0x7fe8f4104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.362+0000 7fe8fa179640 1 -- 172.21.15.59:0/21882215 >> 172.21.15.59:0/21882215 conn(0x7fe8f4077c30 msgr2=0x7fe8f4078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.362+0000 7fe8fa179640 1 -- 172.21.15.59:0/21882215 shutdown_connections 2024-05-08T05:30:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.362+0000 7fe8fa179640 1 -- 172.21.15.59:0/21882215 wait complete. 2024-05-08T05:30:43.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8fa179640 1 Processor -- start 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8fa179640 1 -- start start 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8fa179640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8f4103c60 0x7fe8f4199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8fa179640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 0x7fe8f4199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8fa179640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f410d030 0x7fe8f419d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8fa179640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4112c70 con 0x7fe8f4104580 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8fa179640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4112af0 con 0x7fe8f4103c60 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8fa179640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f4112df0 con 0x7fe8f410d030 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 0x7fe8f4199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 0x7fe8f4199650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59046/0 (socket says 172.21.15.59:59046) 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8f3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f410d030 0x7fe8f419d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8f2ffd640 1 -- 172.21.15.59:0/3300556694 learned_addr learned my addr 172.21.15.59:0/3300556694 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:43.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.363+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8f4103c60 0x7fe8f4199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8f2ffd640 1 -- 172.21.15.59:0/3300556694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f410d030 msgr2=0x7fe8f419d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f410d030 0x7fe8f419d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8f2ffd640 1 -- 172.21.15.59:0/3300556694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8f4103c60 msgr2=0x7fe8f4199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8f4103c60 0x7fe8f4199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8f2ffd640 1 -- 172.21.15.59:0/3300556694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8f419e100 con 0x7fe8f4104580 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 0x7fe8f4199650 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005940 tx=0x7fe8e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8f0ff9640 1 -- 172.21.15.59:0/3300556694 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e803f030 con 0x7fe8f4104580 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8fa179640 1 -- 172.21.15.59:0/3300556694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8f419e390 con 0x7fe8f4104580 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8f0ff9640 1 -- 172.21.15.59:0/3300556694 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e8003cb0 con 0x7fe8f4104580 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8f0ff9640 1 -- 172.21.15.59:0/3300556694 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8035070 con 0x7fe8f4104580 2024-05-08T05:30:43.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.364+0000 7fe8fa179640 1 -- 172.21.15.59:0/3300556694 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8f41a5c80 con 0x7fe8f4104580 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8fcf68740 1 Processor -- start 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8fcf68740 1 -- start start 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8f0ff9640 1 -- 172.21.15.59:0/3300556694 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7fe8e8042030 con 0x7fe8f4104580 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370df10 0x558d7370e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d73718bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d737190f0 0x558d7371b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558d736735d0 con 0x558d7370e830 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558d7362c890 con 0x558d7370df10 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558d7366f590 con 0x558d737190f0 2024-05-08T05:30:43.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d73718bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d73718bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59058/0 (socket says 172.21.15.59:59058) 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.367+0000 7fe8f37fe640 1 -- 172.21.15.59:0/1228980765 learned_addr learned my addr 172.21.15.59:0/1228980765 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370df10 0x558d7370e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d737190f0 0x558d7371b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f37fe640 1 -- 172.21.15.59:0/1228980765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d737190f0 msgr2=0x558d7371b4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d737190f0 0x558d7371b4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f37fe640 1 -- 172.21.15.59:0/1228980765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370df10 msgr2=0x558d7370e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370df10 0x558d7370e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f37fe640 1 -- 172.21.15.59:0/1228980765 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d73385210 con 0x558d7370e830 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370df10 0x558d7370e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d737190f0 0x558d7371b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d73718bb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400bd10 tx=0x7fe8e40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bded6fe766eaee5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f0ff9640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8dc0b2bb0 0x7fe8dc0b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8d9ffb640 1 -- 172.21.15.59:0/1228980765 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4004030 con 0x558d7370e830 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8dc0b2bb0 0x7fe8dc0b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8d9ffb640 1 -- 172.21.15.59:0/1228980765 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e40035f0 con 0x558d7370e830 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8d9ffb640 1 -- 172.21.15.59:0/1228980765 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e40117b0 con 0x558d7370e830 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.368+0000 7fe8f0ff9640 1 -- 172.21.15.59:0/3300556694 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fe8e81030c0 con 0x7fe8f4104580 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 -- 172.21.15.59:0/1228980765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 msgr2=0x558d73718bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d73718bb0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400bd10 tx=0x7fe8e40107a0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8dc0b2bb0 0x7fe8dc0b5070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe8f406c2c0 tx=0x7fe8ec009600 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 -- 172.21.15.59:0/1228980765 shutdown_connections 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d737190f0 0x558d7371b4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d73718bb0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/1228980765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370df10 0x558d7370e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 -- 172.21.15.59:0/1228980765 >> 172.21.15.59:0/1228980765 conn(0x558d7370a420 msgr2=0x558d7370a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 -- 172.21.15.59:0/1228980765 shutdown_connections 2024-05-08T05:30:43.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 -- 172.21.15.59:0/1228980765 wait complete. 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.369+0000 7fe8fcf68740 1 -- start start 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8fcf68740 1 Processor -- start 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370df10 0x558d73716050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370e830 0x558d73716590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 0x558d73716ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558d7371bd20 con 0x558d7370df10 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558d73630370 con 0x558d737190f0 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558d7371bea0 con 0x558d7370e830 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8fcf68740 10 client.0 _ll_register_callbacks cb 0x558d73384a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370df10 0x558d73716050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f37fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370df10 0x558d73716050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59072/0 (socket says 172.21.15.59:59072) 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f37fe640 1 -- 172.21.15.59:0/2857976272 learned_addr learned my addr 172.21.15.59:0/2857976272 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370e830 0x558d73716590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 0x558d73716ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f37fe640 1 -- 172.21.15.59:0/2857976272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370e830 msgr2=0x558d73716590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370e830 0x558d73716590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f37fe640 1 -- 172.21.15.59:0/2857976272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 msgr2=0x558d73716ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 0x558d73716ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f37fe640 1 -- 172.21.15.59:0/2857976272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d73451620 con 0x558d7370df10 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.370+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370e830 0x558d73716590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370df10 0x558d73716050 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400b720 tx=0x7fe8e400ba20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60d23e0c1287bf07 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/2857976272 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e4029060 con 0x558d7370df10 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/2857976272 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fe8e4003c20 con 0x558d7370df10 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2857976272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e40035f0 con 0x558d7370df10 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/2857976272 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8e4030050 con 0x558d7370df10 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8fcf68740 10 client.15624 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:43.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2857976272 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d73727b20 con 0x558d7370df10 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/2857976272 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e40035f0 con 0x558d7370df10 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/2857976272 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe8e40046c0 con 0x558d7370df10 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8fcf68740 10 client.15624 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8fcf68740 10 client.15624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8fcf68740 20 client.15624 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.371+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558d73728f20 0x558d7372b310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.372+0000 7fe8fcf68740 4 client.15624 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.372+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2857976272 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d736811e0 con 0x558d73728f20 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.372+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558d73728f20 0x558d7372b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.372+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558d73728f20 0x558d7372b310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=369ef12a9f5b311c server_cookie=a10905b52a89f7b1 in_seq=0 out_seq=0 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.372+0000 7fe8d2ffd640 10 client.15624 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/2857976272 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558d736811e0 con 0x558d73728f20 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8d2ffd640 10 client.15624 handle_command_reply: tid=0 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 client.15624 shutdown 2024-05-08T05:30:43.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2857976272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370df10 msgr2=0x558d73716050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370df10 0x558d73716050 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fe8e400b720 tx=0x7fe8e400ba20 comp rx=0 tx=0).stop 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2857976272 shutdown_connections 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558d73728f20 0x558d7372b310 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 0x558d73716ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370e830 0x558d73716590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2857976272 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370df10 0x558d73716050 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8d2ffd640 0 client.15624 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2857976272 >> 172.21.15.59:0/2857976272 conn(0x558d7370a420 msgr2=0x558d737101a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2857976272 shutdown_connections 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2857976272 wait complete. 2024-05-08T05:30:43.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.373+0000 7fe8fcf68740 20 client.15624 trim_cache size 0 max 16384 2024-05-08T05:30:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8fcf68740 1 Processor -- start 2024-05-08T05:30:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8fcf68740 1 -- start start 2024-05-08T05:30:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370df10 0x558d73709400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d737165a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 0x558d73716ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41afd60 con 0x558d7370e830 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b18a0 con 0x558d737190f0 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b0430 con 0x558d7370df10 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d737165a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8f2ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d737165a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59088/0 (socket says 172.21.15.59:59088) 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8f2ffd640 1 -- 172.21.15.59:0/2688317547 learned_addr learned my addr 172.21.15.59:0/2688317547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.389+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/2688317547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 0x558d73716ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8f2ffd640 1 -- 172.21.15.59:0/2688317547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370df10 msgr2=0x558d73709400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/2688317547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370df10 0x558d73709400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8f2ffd640 1 -- 172.21.15.59:0/2688317547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 msgr2=0x558d73716ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/2688317547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 0x558d73716ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8f2ffd640 1 -- 172.21.15.59:0/2688317547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d73711c20 con 0x558d7370e830 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/2688317547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 0x558d73716ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:43.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/2688317547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d737165a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe8e802a4c0 tx=0x7fe8e8032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa86268590814633 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8d97fa640 1 -- 172.21.15.59:0/2688317547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e8049040 con 0x558d7370e830 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8d97fa640 1 -- 172.21.15.59:0/2688317547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e80c9ea0 con 0x558d7370e830 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.390+0000 7fe8d97fa640 1 -- 172.21.15.59:0/2688317547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8e80ca100 con 0x558d7370e830 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2688317547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 msgr2=0x558d737165a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2688317547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d737165a0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fe8e802a4c0 tx=0x7fe8e8032f60 comp rx=0 tx=0).stop 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2688317547 shutdown_connections 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2688317547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d737190f0 0x558d73716ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2688317547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d7370e830 0x558d737165a0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/2688317547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370df10 0x558d73709400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2688317547 >> 172.21.15.59:0/2688317547 conn(0x558d7370a420 msgr2=0x558d73708cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2688317547 shutdown_connections 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 -- 172.21.15.59:0/2688317547 wait complete. 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 -- start start 2024-05-08T05:30:43.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 Processor -- start 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.391+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370df10 0x558d73721240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370e830 0x558d73721780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8fcf68740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d737190f0 0x558d7371c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b11d0 con 0x558d737190f0 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41b0b00 con 0x558d7370e830 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8fcf68740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8f41add20 con 0x558d7370df10 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8fcf68740 10 client.0 _ll_register_callbacks cb 0x7fe8e400bb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f3fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d737190f0 0x558d7371c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f3fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d737190f0 0x558d7371c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59098/0 (socket says 172.21.15.59:59098) 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f3fff640 1 -- 172.21.15.59:0/4287838550 learned_addr learned my addr 172.21.15.59:0/4287838550 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370df10 0x558d73721240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370e830 0x558d73721780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f3fff640 1 -- 172.21.15.59:0/4287838550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370df10 msgr2=0x558d73721240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370df10 0x558d73721240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f3fff640 1 -- 172.21.15.59:0/4287838550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370e830 msgr2=0x558d73721780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370e830 0x558d73721780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f3fff640 1 -- 172.21.15.59:0/4287838550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d7371d090 con 0x558d737190f0 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f2ffd640 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370e830 0x558d73721780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8f3fff640 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d737190f0 0x558d7371c860 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec01c580 tx=0x7fe8ec014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4995d983f3a0be3c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8d2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/4287838550 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8ec004b20 con 0x558d737190f0 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/4287838550 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fe8ec004cc0 con 0x558d737190f0 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8fcf68740 1 -- 172.21.15.59:0/4287838550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8e80c9ea0 con 0x558d737190f0 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/4287838550 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe8ec034050 con 0x558d737190f0 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.392+0000 7fe8fcf68740 10 client.15630 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8fcf68740 1 -- 172.21.15.59:0/4287838550 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x558d737930b0 con 0x558d737190f0 2024-05-08T05:30:43.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/4287838550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8e80c9ea0 con 0x558d737190f0 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/4287838550 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe8ec00f040 con 0x558d737190f0 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8fcf68740 10 client.15630 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8fcf68740 10 client.15630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8fcf68740 20 client.15630 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558d737938b0 0x558d73795cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8fcf68740 4 client.15630 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8fcf68740 1 -- 172.21.15.59:0/4287838550 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558d7367b050 con 0x558d737938b0 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.393+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558d737938b0 0x558d73795cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.394+0000 7fe8d2ffd640 10 client.15630 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:43.399 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.394+0000 7fe8f37fe640 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558d737938b0 0x558d73795cd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a96e762400654d64 server_cookie=416afe75a924f82a in_seq=0 out_seq=0 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8d2ffd640 1 -- 172.21.15.59:0/4287838550 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558d7367b050 con 0x558d737938b0 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8d2ffd640 10 client.15630 handle_command_reply: tid=0 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 client.15630 shutdown 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 -- 172.21.15.59:0/4287838550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d737190f0 msgr2=0x558d7371c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d737190f0 0x558d7371c860 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fe8ec01c580 tx=0x7fe8ec014040 comp rx=0 tx=0).stop 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 -- 172.21.15.59:0/4287838550 shutdown_connections 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558d737938b0 0x558d73795cd0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558d737190f0 0x558d7371c860 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558d7370e830 0x558d73721780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.402 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8d2ffd640 0 client.15630 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 --2- 172.21.15.59:0/4287838550 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558d7370df10 0x558d73721240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 -- 172.21.15.59:0/4287838550 >> 172.21.15.59:0/4287838550 conn(0x558d7370a420 msgr2=0x558d73709580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 -- 172.21.15.59:0/4287838550 shutdown_connections 2024-05-08T05:30:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.401+0000 7fe8fcf68740 1 -- 172.21.15.59:0/4287838550 wait complete. 2024-05-08T05:30:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fcf68740 20 client.15630 trim_cache size 0 max 16384 2024-05-08T05:30:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:30:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:30:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:30:43.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:30:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 -- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8dc0b2bb0 msgr2=0x7fe8dc0b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8dc0b2bb0 0x7fe8dc0b5070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fe8f406c2c0 tx=0x7fe8ec009600 comp rx=0 tx=0).stop 2024-05-08T05:30:43.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 -- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 msgr2=0x7fe8f4199650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 0x7fe8f4199650 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fe8e8005940 tx=0x7fe8e80042f0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 -- 172.21.15.59:0/3300556694 shutdown_connections 2024-05-08T05:30:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8dc0b2bb0 0x7fe8dc0b5070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe8f410d030 0x7fe8f419d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe8f4104580 0x7fe8f4199650 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 --2- 172.21.15.59:0/3300556694 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe8f4103c60 0x7fe8f4199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 -- 172.21.15.59:0/3300556694 >> 172.21.15.59:0/3300556694 conn(0x7fe8f4077c30 msgr2=0x7fe8f40fe780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 -- 172.21.15.59:0/3300556694 shutdown_connections 2024-05-08T05:30:43.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:43.402+0000 7fe8fa179640 1 -- 172.21.15.59:0/3300556694 wait complete. 2024-05-08T05:30:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:43 smithi143 ceph-mon[34337]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T05:30:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:43 smithi096 ceph-mon[34253]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T05:30:43.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:43 smithi059 ceph-mon[30304]: pgmap v37: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 10 op/s 2024-05-08T05:30:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:45 smithi143 ceph-mon[34337]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T05:30:45.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:45 smithi096 ceph-mon[34253]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T05:30:45.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:45 smithi059 ceph-mon[30304]: pgmap v38: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.6 KiB/s wr, 9 op/s 2024-05-08T05:30:47.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:47 smithi143 ceph-mon[34337]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-05-08T05:30:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:47 smithi096 ceph-mon[34253]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-05-08T05:30:47.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:47 smithi059 ceph-mon[30304]: pgmap v39: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.8 KiB/s wr, 15 op/s 2024-05-08T05:30:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:49 smithi143 ceph-mon[34337]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-08T05:30:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:49 smithi096 ceph-mon[34253]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-08T05:30:49.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:49 smithi059 ceph-mon[30304]: pgmap v40: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-08T05:30:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:51 smithi143 ceph-mon[34337]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-08T05:30:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:51 smithi096 ceph-mon[34253]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-08T05:30:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:51 smithi059 ceph-mon[30304]: pgmap v41: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-08T05:30:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:53 smithi143 ceph-mon[34337]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-08T05:30:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:53 smithi096 ceph-mon[34253]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-08T05:30:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:53 smithi059 ceph-mon[30304]: pgmap v42: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-08T05:30:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:55 smithi143 ceph-mon[34337]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-08T05:30:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:55 smithi096 ceph-mon[34253]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-08T05:30:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:55 smithi059 ceph-mon[30304]: pgmap v43: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-05-08T05:30:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.494+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2740498198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41022a0 msgr2=0x7fa1a410e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.494+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2740498198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41022a0 0x7fa1a410e760 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009cd0 tx=0x7fa19c02a8e0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.494+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2740498198 shutdown_connections 2024-05-08T05:30:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.494+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2740498198 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41022a0 0x7fa1a410e760 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.494+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2740498198 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41018e0 0x7fa1a4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.494+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2740498198 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41078a0 0x7fa1a4107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.507 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.494+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2740498198 >> 172.21.15.59:0/2740498198 conn(0x7fa1a4078a30 msgr2=0x7fa1a4078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.494+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2740498198 shutdown_connections 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.494+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2740498198 wait complete. 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1abb9d640 1 Processor -- start 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1abb9d640 1 -- start start 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1abb9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41018e0 0x7fa1a41991a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1abb9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41022a0 0x7fa1a41996e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1abb9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41078a0 0x7fa1a419da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1abb9d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4111b60 con 0x7fa1a41018e0 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1abb9d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41119e0 con 0x7fa1a41078a0 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1abb9d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4111ce0 con 0x7fa1a41022a0 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1a9939640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41018e0 0x7fa1a41991a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1a9939640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41018e0 0x7fa1a41991a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33940/0 (socket says 172.21.15.59:33940) 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1a9939640 1 -- 172.21.15.59:0/2831768034 learned_addr learned my addr 172.21.15.59:0/2831768034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:56.508 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41078a0 0x7fa1a419da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1a9138640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41022a0 0x7fa1a41996e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.495+0000 7fa1a9939640 1 -- 172.21.15.59:0/2831768034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41022a0 msgr2=0x7fa1a41996e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa1a9939640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41022a0 0x7fa1a41996e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa1a9939640 1 -- 172.21.15.59:0/2831768034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41078a0 msgr2=0x7fa1a419da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa1a9939640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41078a0 0x7fa1a419da60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa1a9939640 1 -- 172.21.15.59:0/2831768034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a419e190 con 0x7fa1a41018e0 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa1a9138640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41022a0 0x7fa1a41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa1a9939640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41018e0 0x7fa1a41991a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000d770 tx=0x7fa1a000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa19affd640 1 -- 172.21.15.59:0/2831768034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0017070 con 0x7fa1a41018e0 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa19affd640 1 -- 172.21.15.59:0/2831768034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a00054d0 con 0x7fa1a41018e0 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2831768034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a419e480 con 0x7fa1a41018e0 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa19affd640 1 -- 172.21.15.59:0/2831768034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a0004c30 con 0x7fa1a41018e0 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.496+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2831768034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41a5d10 con 0x7fa1a41018e0 2024-05-08T05:30:56.509 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.498+0000 7fa19affd640 1 -- 172.21.15.59:0/2831768034 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7fa1a0004df0 con 0x7fa1a41018e0 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1ae98e740 1 Processor -- start 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1ae98e740 1 -- start start 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bc01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bc0710 0x55a828bcefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bd18f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a828b255d0 con 0x55a828bbfdf0 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a828ade890 con 0x55a828bcf500 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a828b21590 con 0x55a828bc0710 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1aa13a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bc01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1aa13a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bc01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33946/0 (socket says 172.21.15.59:33946) 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.499+0000 7fa1aa13a640 1 -- 172.21.15.59:0/813046129 learned_addr learned my addr 172.21.15.59:0/813046129 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa1a9138640 1 --2- 172.21.15.59:0/813046129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bd18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa1a9939640 1 --2- 172.21.15.59:0/813046129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bc0710 0x55a828bcefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa1aa13a640 1 -- 172.21.15.59:0/813046129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bc0710 msgr2=0x55a828bcefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/813046129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bc0710 0x55a828bcefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa1aa13a640 1 -- 172.21.15.59:0/813046129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 msgr2=0x55a828bd18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/813046129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bd18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa1aa13a640 1 -- 172.21.15.59:0/813046129 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a828837210 con 0x55a828bbfdf0 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa19affd640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1900b2bb0 0x7fa1900b5070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/813046129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bc01d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0093e0 tx=0x7fa19c009a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c429c92c160da22 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa1a9939640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1900b2bb0 0x7fa1900b5070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa18ffff640 1 -- 172.21.15.59:0/813046129 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c031850 con 0x55a828bbfdf0 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa19affd640 1 -- 172.21.15.59:0/2831768034 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa1a00e2730 con 0x7fa1a41018e0 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa18ffff640 1 -- 172.21.15.59:0/813046129 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c036040 con 0x55a828bbfdf0 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.500+0000 7fa18ffff640 1 -- 172.21.15.59:0/813046129 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c0043e0 con 0x55a828bbfdf0 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1ae98e740 1 -- 172.21.15.59:0/813046129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 msgr2=0x55a828bc01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:56.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1a9939640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1900b2bb0 0x7fa1900b5070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00df930 tx=0x7fa1a0091040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/813046129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bc01d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fa19c0093e0 tx=0x7fa19c009a20 comp rx=0 tx=0).stop 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1ae98e740 1 -- 172.21.15.59:0/813046129 shutdown_connections 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/813046129 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bd18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/813046129 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bc0710 0x55a828bcefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/813046129 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bc01d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1ae98e740 1 -- 172.21.15.59:0/813046129 >> 172.21.15.59:0/813046129 conn(0x55a828bbc420 msgr2=0x55a828bbc800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1ae98e740 1 -- 172.21.15.59:0/813046129 shutdown_connections 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.501+0000 7fa1ae98e740 1 -- 172.21.15.59:0/813046129 wait complete. 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1ae98e740 1 -- start start 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1ae98e740 1 Processor -- start 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 0x55a828bcc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bcca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bccf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a828bd20d0 con 0x55a828bc0710 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a828ae2370 con 0x55a828bcf500 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a828bd2250 con 0x55a828bbfdf0 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1ae98e740 10 client.0 _ll_register_callbacks cb 0x55a828836a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1a9138640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bcca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1a9138640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bcca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33948/0 (socket says 172.21.15.59:33948) 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1a9138640 1 -- 172.21.15.59:0/4237484089 learned_addr learned my addr 172.21.15.59:0/4237484089 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 0x55a828bcc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.502+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bccf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1a9138640 1 -- 172.21.15.59:0/4237484089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 msgr2=0x55a828bcc4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1a9138640 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 0x55a828bcc4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1a9138640 1 -- 172.21.15.59:0/4237484089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 msgr2=0x55a828bccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1a9138640 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bccf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1a9138640 1 -- 172.21.15.59:0/4237484089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a828903620 con 0x55a828bc0710 2024-05-08T05:30:56.513 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 0x55a828bcc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1a9138640 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bcca00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa194013710 tx=0x7fa194013bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6545cfb5ff79dcbc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa18d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa18d7fa640 1 -- 172.21.15.59:0/4237484089 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194011dd0 con 0x55a828bc0710 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa18d7fa640 1 -- 172.21.15.59:0/4237484089 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa194004020 con 0x55a828bc0710 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1ae98e740 1 -- 172.21.15.59:0/4237484089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19c036040 con 0x55a828bc0710 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa18d7fa640 1 -- 172.21.15.59:0/4237484089 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa194004220 con 0x55a828bc0710 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1ae98e740 10 client.15642 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa1ae98e740 1 -- 172.21.15.59:0/4237484089 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a828bde0e0 con 0x55a828bc0710 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.503+0000 7fa18d7fa640 1 -- 172.21.15.59:0/4237484089 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c036040 con 0x55a828bc0710 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa18d7fa640 1 -- 172.21.15.59:0/4237484089 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa19401a070 con 0x55a828bc0710 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa1ae98e740 10 client.15642 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa1ae98e740 10 client.15642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa1ae98e740 20 client.15642 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:56.514 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a828bdf640 0x55a828be1a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa1ae98e740 4 client.15642 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa1ae98e740 1 -- 172.21.15.59:0/4237484089 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a828b331e0 con 0x55a828bdf640 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a828bdf640 0x55a828be1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a828bdf640 0x55a828be1a50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4874d5d93b352ca5 server_cookie=3f849b7237e14b0d in_seq=0 out_seq=0 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.504+0000 7fa18d7fa640 10 client.15642 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa18d7fa640 1 -- 172.21.15.59:0/4237484089 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a828b331e0 con 0x55a828bdf640 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa18d7fa640 10 client.15642 handle_command_reply: tid=0 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 client.15642 shutdown 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 -- 172.21.15.59:0/4237484089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 msgr2=0x55a828bcca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:56.515 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bcca00 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fa194013710 tx=0x7fa194013bd0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 -- 172.21.15.59:0/4237484089 shutdown_connections 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a828bdf640 0x55a828be1a50 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bccf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bcca00 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa18d7fa640 0 client.15642 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/4237484089 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 0x55a828bcc4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 -- 172.21.15.59:0/4237484089 >> 172.21.15.59:0/4237484089 conn(0x55a828bbc420 msgr2=0x55a828bc2210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 -- 172.21.15.59:0/4237484089 shutdown_connections 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 1 -- 172.21.15.59:0/4237484089 wait complete. 2024-05-08T05:30:56.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.505+0000 7fa1ae98e740 20 client.15642 trim_cache size 0 max 16384 2024-05-08T05:30:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1ae98e740 1 Processor -- start 2024-05-08T05:30:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1ae98e740 1 -- start start 2024-05-08T05:30:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bce400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bc0710 0x55a828be7320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bcf500 0x55a828be7860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41ada80 con 0x55a828bbfdf0 2024-05-08T05:30:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41af5c0 con 0x55a828bc0710 2024-05-08T05:30:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41ae150 con 0x55a828bcf500 2024-05-08T05:30:56.526 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1aa13a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bce400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1aa13a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bce400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33956/0 (socket says 172.21.15.59:33956) 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1aa13a640 1 -- 172.21.15.59:0/3033137502 learned_addr learned my addr 172.21.15.59:0/3033137502 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1a9138640 1 --2- 172.21.15.59:0/3033137502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bc0710 0x55a828be7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/3033137502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bcf500 0x55a828be7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1aa13a640 1 -- 172.21.15.59:0/3033137502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bcf500 msgr2=0x55a828be7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/3033137502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bcf500 0x55a828be7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1aa13a640 1 -- 172.21.15.59:0/3033137502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bc0710 msgr2=0x55a828be7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/3033137502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bc0710 0x55a828be7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.521+0000 7fa1aa13a640 1 -- 172.21.15.59:0/3033137502 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a828bb7a50 con 0x55a828bbfdf0 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/3033137502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bce400 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009410 tx=0x7fa19c02a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6063012f063b0fb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa18f7fe640 1 -- 172.21.15.59:0/3033137502 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c058070 con 0x55a828bbfdf0 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa18f7fe640 1 -- 172.21.15.59:0/3033137502 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c036040 con 0x55a828bbfdf0 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa18f7fe640 1 -- 172.21.15.59:0/3033137502 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19c053050 con 0x55a828bbfdf0 2024-05-08T05:30:56.527 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1ae98e740 1 -- 172.21.15.59:0/3033137502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 msgr2=0x55a828bce400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/3033137502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bce400 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7fa19c009410 tx=0x7fa19c02a8e0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1ae98e740 1 -- 172.21.15.59:0/3033137502 shutdown_connections 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/3033137502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bcf500 0x55a828be7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/3033137502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bc0710 0x55a828be7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/3033137502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bbfdf0 0x55a828bce400 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1ae98e740 1 -- 172.21.15.59:0/3033137502 >> 172.21.15.59:0/3033137502 conn(0x55a828bbc420 msgr2=0x55a828bcdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1ae98e740 1 -- 172.21.15.59:0/3033137502 shutdown_connections 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.522+0000 7fa1ae98e740 1 -- 172.21.15.59:0/3033137502 wait complete. 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1ae98e740 1 -- start start 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1ae98e740 1 Processor -- start 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 0x55a828bd77e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bd44e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1ae98e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bd4a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.528 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41aeef0 con 0x55a828bc0710 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41ae820 con 0x55a828bcf500 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1ae98e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41aba40 con 0x55a828bbfdf0 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1ae98e740 10 client.0 _ll_register_callbacks cb 0x7fa194005850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1a9138640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bd44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1a9138640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bd44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33972/0 (socket says 172.21.15.59:33972) 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1a9138640 1 -- 172.21.15.59:0/1894238910 learned_addr learned my addr 172.21.15.59:0/1894238910 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bd4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.523+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 0x55a828bd77e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa1a9138640 1 -- 172.21.15.59:0/1894238910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 msgr2=0x55a828bd77e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa1a9138640 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 0x55a828bd77e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa1a9138640 1 -- 172.21.15.59:0/1894238910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 msgr2=0x55a828bd4a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa1a9138640 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bd4a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa1a9138640 1 -- 172.21.15.59:0/1894238910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a828bd4f90 con 0x55a828bc0710 2024-05-08T05:30:56.529 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa1a9138640 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bd44e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa19401b530 tx=0x7fa194009c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25fa7bcaa7c1124a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa18d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa18d7fa640 1 -- 172.21.15.59:0/1894238910 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa194005450 con 0x55a828bc0710 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa18d7fa640 1 -- 172.21.15.59:0/1894238910 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fa19401b7a0 con 0x55a828bc0710 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa1ae98e740 1 -- 172.21.15.59:0/1894238910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa19c036040 con 0x55a828bc0710 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa18d7fa640 1 -- 172.21.15.59:0/1894238910 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fa1940055f0 con 0x55a828bc0710 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa1ae98e740 10 client.15648 fetch_fsmap learned FSMap version 30 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa1ae98e740 1 -- 172.21.15.59:0/1894238910 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a828c4ac70 con 0x55a828bc0710 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa18d7fa640 1 -- 172.21.15.59:0/1894238910 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa19c036040 con 0x55a828bc0710 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.524+0000 7fa18d7fa640 1 -- 172.21.15.59:0/1894238910 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fa19401a050 con 0x55a828bc0710 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.525+0000 7fa1ae98e740 10 client.15648 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.525+0000 7fa1ae98e740 10 client.15648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.525+0000 7fa1ae98e740 20 client.15648 populate_metadata read hostname 'smithi059' 2024-05-08T05:30:56.530 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.525+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a828c4b470 0x55a828c4d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.525+0000 7fa1ae98e740 4 client.15648 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:30:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.525+0000 7fa1ae98e740 1 -- 172.21.15.59:0/1894238910 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a828b2d050 con 0x55a828c4b470 2024-05-08T05:30:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.525+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a828c4b470 0x55a828c4d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.525+0000 7fa18d7fa640 10 client.15648 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:56.531 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.525+0000 7fa1aa13a640 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a828c4b470 0x55a828c4d890 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22a0e75158e1f73b server_cookie=f6878d9ea0ace79 in_seq=0 out_seq=0 2024-05-08T05:30:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa18d7fa640 1 -- 172.21.15.59:0/1894238910 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a828b2d050 con 0x55a828c4b470 2024-05-08T05:30:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa18d7fa640 10 client.15648 handle_command_reply: tid=0 2024-05-08T05:30:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 client.15648 shutdown 2024-05-08T05:30:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 -- 172.21.15.59:0/1894238910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 msgr2=0x55a828bd44e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bd44e0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7fa19401b530 tx=0x7fa194009c80 comp rx=0 tx=0).stop 2024-05-08T05:30:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 -- 172.21.15.59:0/1894238910 shutdown_connections 2024-05-08T05:30:56.532 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a828c4b470 0x55a828c4d890 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a828bcf500 0x55a828bd4a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a828bc0710 0x55a828bd44e0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 --2- 172.21.15.59:0/1894238910 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a828bbfdf0 0x55a828bd77e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa18d7fa640 0 client.15648 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 -- 172.21.15.59:0/1894238910 >> 172.21.15.59:0/1894238910 conn(0x55a828bbc420 msgr2=0x55a828bbb580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 -- 172.21.15.59:0/1894238910 shutdown_connections 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 1 -- 172.21.15.59:0/1894238910 wait complete. 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.529+0000 7fa1ae98e740 20 client.15648 trim_cache size 0 max 16384 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:30:56.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:30:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:30:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:30:56.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1900b2bb0 msgr2=0x7fa1900b5070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1900b2bb0 0x7fa1900b5070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fa1a00df930 tx=0x7fa1a0091040 comp rx=0 tx=0).stop 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41018e0 msgr2=0x7fa1a41991a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41018e0 0x7fa1a41991a0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fa1a000d770 tx=0x7fa1a000dc30 comp rx=0 tx=0).stop 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2831768034 shutdown_connections 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1900b2bb0 0x7fa1900b5070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41078a0 0x7fa1a419da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41022a0 0x7fa1a41996e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 --2- 172.21.15.59:0/2831768034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41018e0 0x7fa1a41991a0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2831768034 >> 172.21.15.59:0/2831768034 conn(0x7fa1a4078a30 msgr2=0x7fa1a406a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2831768034 shutdown_connections 2024-05-08T05:30:56.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:30:56.530+0000 7fa1abb9d640 1 -- 172.21.15.59:0/2831768034 wait complete. 2024-05-08T05:30:56.834 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:30:57.091 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:30:57.470 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:57 smithi059 ceph-mon[30304]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-08T05:30:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:57 smithi143 ceph-mon[34337]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-08T05:30:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:57 smithi096 ceph-mon[34253]: pgmap v44: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.2 KiB/s wr, 14 op/s 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.961+0000 7f954e7e9700 1 -- 172.21.15.59:0/2264210671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102fe0 msgr2=0x7f9548106fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.961+0000 7f954e7e9700 1 --2- 172.21.15.59:0/2264210671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102fe0 0x7f9548106fe0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f953c0099c0 tx=0x7f953c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.961+0000 7f954e7e9700 1 -- 172.21.15.59:0/2264210671 shutdown_connections 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.961+0000 7f954e7e9700 1 --2- 172.21.15.59:0/2264210671 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95481076c0 0x7f954810f290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.961+0000 7f954e7e9700 1 --2- 172.21.15.59:0/2264210671 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9548102630 0x7f9548102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.961+0000 7f954e7e9700 1 --2- 172.21.15.59:0/2264210671 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102fe0 0x7f9548106fe0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.961+0000 7f954e7e9700 1 -- 172.21.15.59:0/2264210671 >> 172.21.15.59:0/2264210671 conn(0x7f9548079350 msgr2=0x7f9548079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.961+0000 7f954e7e9700 1 -- 172.21.15.59:0/2264210671 shutdown_connections 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.961+0000 7f954e7e9700 1 -- 172.21.15.59:0/2264210671 wait complete. 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f954e7e9700 1 Processor -- start 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f954e7e9700 1 -- start start 2024-05-08T05:30:58.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f954e7e9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102630 0x7f9548104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f954e7e9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9548102fe0 0x7f95481045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f954e7e9700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95481076c0 0x7f9548104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f954e7e9700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9548112550 con 0x7f9548102630 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f954e7e9700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f95481123d0 con 0x7f9548102fe0 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f954e7e9700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f95481126d0 con 0x7f95481076c0 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f9547fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102630 0x7f9548104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f95477fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9548102fe0 0x7f95481045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f9547fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102630 0x7f9548104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50086/0 (socket says 172.21.15.59:50086) 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f9547fff700 1 -- 172.21.15.59:0/831751522 learned_addr learned my addr 172.21.15.59:0/831751522 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.962+0000 7f954cd86700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95481076c0 0x7f9548104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f95477fe700 1 -- 172.21.15.59:0/831751522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95481076c0 msgr2=0x7f9548104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f95477fe700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95481076c0 0x7f9548104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f95477fe700 1 -- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102630 msgr2=0x7f9548104080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:58.965 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f95477fe700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102630 0x7f9548104080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:58.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f95477fe700 1 -- 172.21.15.59:0/831751522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95481a8120 con 0x7f9548102fe0 2024-05-08T05:30:58.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f9547fff700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102630 0x7f9548104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:30:58.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f95477fe700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9548102fe0 0x7f95481045c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098e0 tx=0x7f953c0045e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:58.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f95457fa700 1 -- 172.21.15.59:0/831751522 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c026070 con 0x7f9548102fe0 2024-05-08T05:30:58.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95481a83b0 con 0x7f9548102fe0 2024-05-08T05:30:58.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95481a8860 con 0x7f9548102fe0 2024-05-08T05:30:58.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f95457fa700 1 -- 172.21.15.59:0/831751522 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f953c00f460 con 0x7f9548102fe0 2024-05-08T05:30:58.966 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.963+0000 7f95457fa700 1 -- 172.21.15.59:0/831751522 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f953c009e60 con 0x7f9548102fe0 2024-05-08T05:30:58.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.964+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f954806a190 con 0x7f9548102fe0 2024-05-08T05:30:58.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.966+0000 7f95457fa700 1 -- 172.21.15.59:0/831751522 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f953c029070 con 0x7f9548102fe0 2024-05-08T05:30:58.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.967+0000 7f95457fa700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f95300b2ba0 0x7f95300b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:30:58.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.967+0000 7f9547fff700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f95300b2ba0 0x7f95300b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:30:58.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.967+0000 7f95457fa700 1 -- 172.21.15.59:0/831751522 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f953c0e34d0 con 0x7f9548102fe0 2024-05-08T05:30:58.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.967+0000 7f9547fff700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f95300b2ba0 0x7f95300b5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9538011630 tx=0x7f9538009260 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:30:58.970 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:58.970+0000 7f95457fa700 1 -- 172.21.15.59:0/831751522 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f953c030080 con 0x7f9548102fe0 2024-05-08T05:30:59.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.240+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f954802e510 con 0x7f9548102fe0 2024-05-08T05:30:59.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.241+0000 7f95457fa700 1 -- 172.21.15.59:0/831751522 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f953c013070 con 0x7f9548102fe0 2024-05-08T05:30:59.242 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:30:59.242 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:29:31.333500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:30:59.244 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f95300b2ba0 msgr2=0x7f95300b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f95300b2ba0 0x7f95300b5060 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f9538011630 tx=0x7f9538009260 comp rx=0 tx=0).stop 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9548102fe0 msgr2=0x7f95481045c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9548102fe0 0x7f95481045c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f953c0098e0 tx=0x7f953c0045e0 comp rx=0 tx=0).stop 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 shutdown_connections 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f95300b2ba0 0x7f95300b5060 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f95481076c0 0x7f9548104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9548102630 0x7f9548104080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 --2- 172.21.15.59:0/831751522 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9548102fe0 0x7f95481045c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 >> 172.21.15.59:0/831751522 conn(0x7f9548079350 msgr2=0x7f954806bf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 shutdown_connections 2024-05-08T05:30:59.245 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:30:59.244+0000 7f954e7e9700 1 -- 172.21.15.59:0/831751522 wait complete. 2024-05-08T05:30:59.246 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 30 2024-05-08T05:30:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:30:59 smithi143 ceph-mon[34337]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:30:59.562 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:30:59 smithi096 ceph-mon[34253]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:30:59.562 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:30:59 smithi059 ceph-mon[30304]: pgmap v45: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:30:59.985 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db --set-id f4dea23a --release --await' 2024-05-08T05:30:59.985 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db --set-id f4dea23a --release --await 2024-05-08T05:31:00.245 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:31:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:00 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/831751522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:31:00.502 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:00 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/831751522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:31:00.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:00 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/831751522' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:31:01.487 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:01 smithi059 ceph-mon[30304]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:01 smithi143 ceph-mon[34337]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:01 smithi096 ceph-mon[34253]: pgmap v46: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:02.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.068+0000 7f58eb30b700 1 -- 172.21.15.59:0/1576949661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e41023a0 msgr2=0x7f58e4102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.068+0000 7f58eb30b700 1 --2- 172.21.15.59:0/1576949661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e41023a0 0x7f58e4102820 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009990 tx=0x7f58d4009ca0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.068+0000 7f58eb30b700 1 -- 172.21.15.59:0/1576949661 shutdown_connections 2024-05-08T05:31:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.068+0000 7f58eb30b700 1 --2- 172.21.15.59:0/1576949661 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4102d60 0x7f58e410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.068+0000 7f58eb30b700 1 --2- 172.21.15.59:0/1576949661 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58e4108410 0x7f58e41087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.068+0000 7f58eb30b700 1 --2- 172.21.15.59:0/1576949661 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e41023a0 0x7f58e4102820 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.068+0000 7f58eb30b700 1 -- 172.21.15.59:0/1576949661 >> 172.21.15.59:0/1576949661 conn(0x7f58e4079350 msgr2=0x7f58e4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.068+0000 7f58eb30b700 1 -- 172.21.15.59:0/1576949661 shutdown_connections 2024-05-08T05:31:02.069 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.068+0000 7f58eb30b700 1 -- 172.21.15.59:0/1576949661 wait complete. 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58eb30b700 1 Processor -- start 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58eb30b700 1 -- start start 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58eb30b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58e41023a0 0x7f58e419f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58eb30b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4102d60 0x7f58e419fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58eb30b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e4108410 0x7f58e41999e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58eb30b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41125b0 con 0x7f58e4108410 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58eb30b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4112430 con 0x7f58e4102d60 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58eb30b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f58e4112730 con 0x7f58e41023a0 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58e98a8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e4108410 0x7f58e41999e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58e98a8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e4108410 0x7f58e41999e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50110/0 (socket says 172.21.15.59:50110) 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58e98a8700 1 -- 172.21.15.59:0/850438255 learned_addr learned my addr 172.21.15.59:0/850438255 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:02.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58e88a6700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4102d60 0x7f58e419fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.069+0000 7f58e90a7700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58e41023a0 0x7f58e419f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58e98a8700 1 -- 172.21.15.59:0/850438255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58e41023a0 msgr2=0x7f58e419f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58e98a8700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58e41023a0 0x7f58e419f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58e98a8700 1 -- 172.21.15.59:0/850438255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4102d60 msgr2=0x7f58e419fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58e98a8700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4102d60 0x7f58e419fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58e98a8700 1 -- 172.21.15.59:0/850438255 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58e419a140 con 0x7f58e4108410 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58e98a8700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e4108410 0x7f58e41999e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f58dc007ee0 tx=0x7f58dc00d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58da7fc700 1 -- 172.21.15.59:0/850438255 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc004bb0 con 0x7f58e4108410 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58eb30b700 1 -- 172.21.15.59:0/850438255 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58e419a430 con 0x7f58e4108410 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58da7fc700 1 -- 172.21.15.59:0/850438255 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58dc004d50 con 0x7f58e4108410 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58da7fc700 1 -- 172.21.15.59:0/850438255 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58dc005920 con 0x7f58e4108410 2024-05-08T05:31:02.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.070+0000 7f58eb30b700 1 -- 172.21.15.59:0/850438255 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58e41a66f0 con 0x7f58e4108410 2024-05-08T05:31:02.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.073+0000 7f58da7fc700 1 -- 172.21.15.59:0/850438255 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f58dc01a020 con 0x7f58e4108410 2024-05-08T05:31:02.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.073+0000 7f58da7fc700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f58d00b2ba0 0x7f58d00b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.074+0000 7f58da7fc700 1 -- 172.21.15.59:0/850438255 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f58dc0e22a0 con 0x7f58e4108410 2024-05-08T05:31:02.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.074+0000 7f58e90a7700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f58d00b2ba0 0x7f58d00b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.074+0000 7f58f5b9eb80 1 Processor -- start 2024-05-08T05:31:02.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.074+0000 7f58f5b9eb80 1 -- start start 2024-05-08T05:31:02.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.074+0000 7f58e90a7700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f58d00b2ba0 0x7f58d00b5060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58e419ab20 tx=0x7f58e0009190 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:02.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb1fba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb19ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1a430 0x56080fb26580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56080fa6ba40 con 0x56080fb19a70 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56080f9d0ea0 con 0x56080fb1f7c0 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56080fa67a00 con 0x56080fb1a430 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e88a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb19ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e88a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb19ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50122/0 (socket says 172.21.15.59:50122) 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e88a6700 1 -- 172.21.15.59:0/28676507 learned_addr learned my addr 172.21.15.59:0/28676507 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e98a8700 1 --2- 172.21.15.59:0/28676507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1a430 0x56080fb26580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e90a7700 1 --2- 172.21.15.59:0/28676507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb1fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e88a6700 1 -- 172.21.15.59:0/28676507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1a430 msgr2=0x56080fb26580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e88a6700 1 --2- 172.21.15.59:0/28676507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1a430 0x56080fb26580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e88a6700 1 -- 172.21.15.59:0/28676507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 msgr2=0x56080fb1fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e88a6700 1 --2- 172.21.15.59:0/28676507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb1fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.075+0000 7f58e88a6700 1 -- 172.21.15.59:0/28676507 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080f9b89d0 con 0x56080fb19a70 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.076+0000 7f58e88a6700 1 --2- 172.21.15.59:0/28676507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb19ef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009990 tx=0x7f58d4005c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7920b0f2be39f01 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:02.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.076+0000 7f58cf7fe700 1 -- 172.21.15.59:0/28676507 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4003a60 con 0x56080fb19a70 2024-05-08T05:31:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.076+0000 7f58cf7fe700 1 -- 172.21.15.59:0/28676507 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d4026070 con 0x56080fb19a70 2024-05-08T05:31:02.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.076+0000 7f58cf7fe700 1 -- 172.21.15.59:0/28676507 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d400ba30 con 0x56080fb19a70 2024-05-08T05:31:02.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/28676507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 msgr2=0x56080fb19ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:02.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/28676507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb19ef0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009990 tx=0x7f58d4005c70 comp rx=0 tx=0).stop 2024-05-08T05:31:02.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/28676507 shutdown_connections 2024-05-08T05:31:02.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/28676507 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1a430 0x56080fb26580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/28676507 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb1fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/28676507 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb19ef0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/28676507 >> 172.21.15.59:0/28676507 conn(0x56080fa45c70 msgr2=0x56080fb147f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/28676507 shutdown_connections 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/28676507 wait complete. 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 -- start start 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.077+0000 7f58f5b9eb80 1 Processor -- start 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb19a70 0x56080fb260e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb23480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb239c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56080fb2aed0 con 0x56080fb1a430 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56080fa24fd0 con 0x56080fb1f7c0 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56080fb2b050 con 0x56080fb19a70 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58f5b9eb80 10 client.0 _ll_register_callbacks cb 0x56080f9ff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e88a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb23480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e88a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb23480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50126/0 (socket says 172.21.15.59:50126) 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e88a6700 1 -- 172.21.15.59:0/3340534953 learned_addr learned my addr 172.21.15.59:0/3340534953 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e98a8700 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb239c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e88a6700 1 -- 172.21.15.59:0/3340534953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb19a70 msgr2=0x56080fb260e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb19a70 0x56080fb260e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e88a6700 1 -- 172.21.15.59:0/3340534953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 msgr2=0x56080fb239c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb239c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e88a6700 1 -- 172.21.15.59:0/3340534953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080fa22860 con 0x56080fb1a430 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.078+0000 7f58e98a8700 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb239c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.079+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb23480 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009350 tx=0x7f58d400ba80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ab776165a766588 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.079+0000 7f58ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.079+0000 7f58ccff9700 1 -- 172.21.15.59:0/3340534953 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4005270 con 0x56080fb1a430 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.079+0000 7f58ccff9700 1 -- 172.21.15.59:0/3340534953 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f58d4005410 con 0x56080fb1a430 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.079+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3340534953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d4026070 con 0x56080fb1a430 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.079+0000 7f58ccff9700 1 -- 172.21.15.59:0/3340534953 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58d4003680 con 0x56080fb1a430 2024-05-08T05:31:02.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.079+0000 7f58f5b9eb80 10 client.15666 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.079+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3340534953 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56080fb35560 con 0x56080fb1a430 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.079+0000 7f58ccff9700 1 -- 172.21.15.59:0/3340534953 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d4026070 con 0x56080fb1a430 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.080+0000 7f58ccff9700 1 -- 172.21.15.59:0/3340534953 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f58d4032400 con 0x56080fb1a430 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.080+0000 7f58f5b9eb80 10 client.15666 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.080+0000 7f58f5b9eb80 10 client.15666 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.080+0000 7f58f5b9eb80 20 client.15666 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.080+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56080fb37a30 0x56080fb39e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.080+0000 7f58f5b9eb80 4 client.15666 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.080+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3340534953 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56080fa79420 con 0x56080fb37a30 2024-05-08T05:31:02.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.080+0000 7f58e90a7700 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56080fb37a30 0x56080fb39e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.081+0000 7f58e90a7700 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56080fb37a30 0x56080fb39e20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=495e0a34d4692287 server_cookie=a0973600cc5a8f8f in_seq=0 out_seq=0 2024-05-08T05:31:02.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.081+0000 7f58ccff9700 10 client.15666 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:02.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.081+0000 7f58ccff9700 1 -- 172.21.15.59:0/3340534953 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56080fa79420 con 0x56080fb37a30 2024-05-08T05:31:02.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.081+0000 7f58ccff9700 10 client.15666 handle_command_reply: tid=0 2024-05-08T05:31:02.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 client.15666 shutdown 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3340534953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 msgr2=0x56080fb23480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb23480 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009350 tx=0x7f58d400ba80 comp rx=0 tx=0).stop 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3340534953 shutdown_connections 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56080fb37a30 0x56080fb39e20 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb239c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb19a70 0x56080fb260e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3340534953 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb23480 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3340534953 >> 172.21.15.59:0/3340534953 conn(0x56080fa45c70 msgr2=0x56080fb1eb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58ccff9700 0 client.15666 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3340534953 shutdown_connections 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3340534953 wait complete. 2024-05-08T05:31:02.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.082+0000 7f58f5b9eb80 20 client.15666 trim_cache size 0 max 16384 2024-05-08T05:31:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58f5b9eb80 1 Processor -- start 2024-05-08T05:31:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58f5b9eb80 1 -- start start 2024-05-08T05:31:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb19a70 0x56080fb0e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb3ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb3d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41ad110 con 0x56080fb1a430 2024-05-08T05:31:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41aec50 con 0x56080fb1f7c0 2024-05-08T05:31:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41ad7e0 con 0x56080fb19a70 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e88a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb3ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e88a6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb3ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50138/0 (socket says 172.21.15.59:50138) 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e88a6700 1 -- 172.21.15.59:0/3050484635 learned_addr learned my addr 172.21.15.59:0/3050484635 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e98a8700 1 --2- 172.21.15.59:0/3050484635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb3d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e90a7700 1 --2- 172.21.15.59:0/3050484635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb19a70 0x56080fb0e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e88a6700 1 -- 172.21.15.59:0/3050484635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb19a70 msgr2=0x56080fb0e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3050484635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb19a70 0x56080fb0e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e88a6700 1 -- 172.21.15.59:0/3050484635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 msgr2=0x56080fb3d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3050484635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb3d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.103+0000 7f58e88a6700 1 -- 172.21.15.59:0/3050484635 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56080fb3d8c0 con 0x56080fb1a430 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.104+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3050484635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb3ce40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009ce0 tx=0x7f58d4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd4c8c1f63d87590 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.104+0000 7f58ceffd700 1 -- 172.21.15.59:0/3050484635 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d40050d0 con 0x56080fb1a430 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.104+0000 7f58ceffd700 1 -- 172.21.15.59:0/3050484635 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d4026040 con 0x56080fb1a430 2024-05-08T05:31:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.104+0000 7f58ceffd700 1 -- 172.21.15.59:0/3050484635 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58d4005410 con 0x56080fb1a430 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3050484635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 msgr2=0x56080fb3ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3050484635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb3ce40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f58d4009ce0 tx=0x7f58d4005e80 comp rx=0 tx=0).stop 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3050484635 shutdown_connections 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3050484635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1f7c0 0x56080fb3d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3050484635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb19a70 0x56080fb0e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3050484635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb1a430 0x56080fb3ce40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3050484635 >> 172.21.15.59:0/3050484635 conn(0x56080fa45c70 msgr2=0x56080fb0de90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3050484635 shutdown_connections 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3050484635 wait complete. 2024-05-08T05:31:02.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 -- start start 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.105+0000 7f58f5b9eb80 1 Processor -- start 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb3aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1a430 0x56080fb3b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58f5b9eb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1f7c0 0x56080fba0f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e90a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb3aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41ae580 con 0x56080fb19a70 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41adeb0 con 0x56080fb1a430 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e90a7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb3aed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50148/0 (socket says 172.21.15.59:50148) 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58f5b9eb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f58e41ab090 con 0x56080fb1f7c0 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e90a7700 1 -- 172.21.15.59:0/3130498691 learned_addr learned my addr 172.21.15.59:0/3130498691 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58f5b9eb80 10 client.0 _ll_register_callbacks cb 0x56080f9ff6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1a430 0x56080fb3b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e98a8700 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1f7c0 0x56080fba0f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e90a7700 1 -- 172.21.15.59:0/3130498691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1f7c0 msgr2=0x56080fba0f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:02.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e90a7700 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1f7c0 0x56080fba0f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e90a7700 1 -- 172.21.15.59:0/3130498691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1a430 msgr2=0x56080fb3b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e90a7700 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1a430 0x56080fb3b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e90a7700 1 -- 172.21.15.59:0/3130498691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56080fba1490 con 0x56080fb19a70 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1a430 0x56080fb3b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58e90a7700 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb3aed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x56080fb2b140 tx=0x7f58e0004860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6981e39bf7065390 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.106+0000 7f58ccff9700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58ccff9700 1 -- 172.21.15.59:0/3130498691 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58e001b040 con 0x56080fb19a70 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58ccff9700 1 -- 172.21.15.59:0/3130498691 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f58e0027240 con 0x56080fb19a70 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3130498691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f58d4026040 con 0x56080fb19a70 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58ccff9700 1 -- 172.21.15.59:0/3130498691 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f58e0037050 con 0x56080fb19a70 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58f5b9eb80 10 client.15672 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3130498691 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56080fba27d0 con 0x56080fb19a70 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58ccff9700 1 -- 172.21.15.59:0/3130498691 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58d4026040 con 0x56080fb19a70 2024-05-08T05:31:02.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58ccff9700 1 -- 172.21.15.59:0/3130498691 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f58e0006500 con 0x56080fb19a70 2024-05-08T05:31:02.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58f5b9eb80 10 client.15672 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:02.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58f5b9eb80 10 client.15672 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:31:02.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58f5b9eb80 20 client.15672 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:02.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56080fba2ee0 0x56080fba5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:02.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58f5b9eb80 4 client.15672 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "f4dea23a", "release": true, "await": true}] 2024-05-08T05:31:02.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.107+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3130498691 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "f4dea23a", "release": true, "await": true}) -- 0x56080fa734c0 con 0x56080fba2ee0 2024-05-08T05:31:02.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.108+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56080fba2ee0 0x56080fba5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:02.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.108+0000 7f58e88a6700 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56080fba2ee0 0x56080fba5300 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1e389f15b5f37f8 server_cookie=c2d1da93b3860176 in_seq=0 out_seq=0 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.108+0000 7f58ccff9700 10 client.15672 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58ccff9700 1 -- 172.21.15.59:0/3130498691 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x56080fa734c0 con 0x56080fba2ee0 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58ccff9700 10 client.15672 handle_command_reply: tid=0 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 client.15672 shutdown 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3130498691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 msgr2=0x56080fb3aed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb3aed0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x56080fb2b140 tx=0x7f58e0004860 comp rx=0 tx=0).stop 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3130498691 shutdown_connections 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56080fba2ee0 0x56080fba5300 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56080fb1f7c0 0x56080fba0f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56080fb19a70 0x56080fb3aed0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58ccff9700 0 client.15672 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:02.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 --2- 172.21.15.59:0/3130498691 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56080fb1a430 0x56080fb3b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3130498691 >> 172.21.15.59:0/3130498691 conn(0x56080fa45c70 msgr2=0x56080fb20480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:02.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3130498691 shutdown_connections 2024-05-08T05:31:02.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.111+0000 7f58f5b9eb80 1 -- 172.21.15.59:0/3130498691 wait complete. 2024-05-08T05:31:02.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58f5b9eb80 20 client.15672 trim_cache size 0 max 16384 2024-05-08T05:31:02.114 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:31:02.114 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 30, 2024-05-08T05:31:02.114 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:31:02.114 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 4, 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "f4dea23a": { 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 4, 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 79.4, 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:31:02.115 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:31:02.116 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:31:02.116 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:31:02.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:02.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:02.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:02.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:02.116 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:02.116 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:31:02.116 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 -- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f58d00b2ba0 msgr2=0x7f58d00b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f58d00b2ba0 0x7f58d00b5060 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f58e419ab20 tx=0x7f58e0009190 comp rx=0 tx=0).stop 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 -- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e4108410 msgr2=0x7f58e41999e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e4108410 0x7f58e41999e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f58dc007ee0 tx=0x7f58dc00d3b0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 -- 172.21.15.59:0/850438255 shutdown_connections 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f58d00b2ba0 0x7f58d00b5060 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f58e4108410 0x7f58e41999e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f58e41023a0 0x7f58e419f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 --2- 172.21.15.59:0/850438255 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f58e4102d60 0x7f58e419fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 -- 172.21.15.59:0/850438255 >> 172.21.15.59:0/850438255 conn(0x7f58e4079350 msgr2=0x7f58e41016b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 -- 172.21.15.59:0/850438255 shutdown_connections 2024-05-08T05:31:02.117 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:02.112+0000 7f58eb30b700 1 -- 172.21.15.59:0/850438255 wait complete. 2024-05-08T05:31:02.900 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'f4dea23a', seconds elapsed: 31.5 2024-05-08T05:31:02.900 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 19.4 2024-05-08T05:31:02.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:31:03.171 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:31:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:03 smithi143 ceph-mon[34337]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:03 smithi096 ceph-mon[34253]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:03.552 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:03 smithi059 ceph-mon[30304]: pgmap v47: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.003+0000 7f86a79f6700 1 -- 172.21.15.59:0/1848344792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a01023a0 msgr2=0x7f86a0102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.003+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1848344792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a01023a0 0x7f86a0102820 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f8690009990 tx=0x7f8690009ca0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.003+0000 7f86a79f6700 1 -- 172.21.15.59:0/1848344792 shutdown_connections 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.003+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1848344792 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a0102d60 0x7f86a010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.003+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1848344792 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86a0108410 0x7f86a01087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.003+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1848344792 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a01023a0 0x7f86a0102820 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.003+0000 7f86a79f6700 1 -- 172.21.15.59:0/1848344792 >> 172.21.15.59:0/1848344792 conn(0x7f86a0079350 msgr2=0x7f86a0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.003+0000 7f86a79f6700 1 -- 172.21.15.59:0/1848344792 shutdown_connections 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.003+0000 7f86a79f6700 1 -- 172.21.15.59:0/1848344792 wait complete. 2024-05-08T05:31:05.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.004+0000 7f86a79f6700 1 Processor -- start 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.004+0000 7f86a79f6700 1 -- start start 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.004+0000 7f86a79f6700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a01023a0 0x7f86a019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.004+0000 7f86a79f6700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0102d60 0x7f86a019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.004+0000 7f86a79f6700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86a0108410 0x7f86a0199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.004+0000 7f86a79f6700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f86a0112560 con 0x7f86a0102d60 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.004+0000 7f86a79f6700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01123e0 con 0x7f86a0108410 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.004+0000 7f86a79f6700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f86a01126e0 con 0x7f86a01023a0 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.004+0000 7f86a4f91700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0102d60 0x7f86a019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a4f91700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0102d60 0x7f86a019fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50160/0 (socket says 172.21.15.59:50160) 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a4f91700 1 -- 172.21.15.59:0/1712477386 learned_addr learned my addr 172.21.15.59:0/1712477386 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a5792700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a01023a0 0x7f86a019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a5f93700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86a0108410 0x7f86a0199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a4f91700 1 -- 172.21.15.59:0/1712477386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a01023a0 msgr2=0x7f86a019f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a4f91700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a01023a0 0x7f86a019f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a4f91700 1 -- 172.21.15.59:0/1712477386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86a0108410 msgr2=0x7f86a0199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a4f91700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86a0108410 0x7f86a0199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a4f91700 1 -- 172.21.15.59:0/1712477386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86a019a0b0 con 0x7f86a0102d60 2024-05-08T05:31:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a4f91700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0102d60 0x7f86a019fcd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8690000c00 tx=0x7f86900047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86967fc700 1 -- 172.21.15.59:0/1712477386 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690027070 con 0x7f86a0102d60 2024-05-08T05:31:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86a019a340 con 0x7f86a0102d60 2024-05-08T05:31:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86967fc700 1 -- 172.21.15.59:0/1712477386 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8690003f20 con 0x7f86a0102d60 2024-05-08T05:31:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86967fc700 1 -- 172.21.15.59:0/1712477386 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8690022ca0 con 0x7f86a0102d60 2024-05-08T05:31:05.008 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.005+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86a01a6660 con 0x7f86a0102d60 2024-05-08T05:31:05.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.006+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f86a006a190 con 0x7f86a0102d60 2024-05-08T05:31:05.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.008+0000 7f86967fc700 1 -- 172.21.15.59:0/1712477386 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 25) ==== 149936+0+0 (secure 0 0 0) 0x7f869002a070 con 0x7f86a0102d60 2024-05-08T05:31:05.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.009+0000 7f86967fc700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f868c0b2ba0 0x7f868c0b5060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:05.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.009+0000 7f86a5792700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f868c0b2ba0 0x7f868c0b5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:05.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.009+0000 7f86967fc700 1 -- 172.21.15.59:0/1712477386 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f86900e4170 con 0x7f86a0102d60 2024-05-08T05:31:05.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.010+0000 7f86a5792700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f868c0b2ba0 0x7f868c0b5060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f869c009550 tx=0x7f869c0091b0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:05.017 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.017+0000 7f86967fc700 1 -- 172.21.15.59:0/1712477386 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8690031070 con 0x7f86a0102d60 2024-05-08T05:31:05.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.291+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f86a019ae80 con 0x7f86a0102d60 2024-05-08T05:31:05.292 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:05 smithi059 ceph-mon[30304]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:05.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.292+0000 7f86967fc700 1 -- 172.21.15.59:0/1712477386 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f8690013070 con 0x7f86a0102d60 2024-05-08T05:31:05.292 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:31:05.293 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:29:31.333500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:31:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f868c0b2ba0 msgr2=0x7f868c0b5060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:05.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f868c0b2ba0 0x7f868c0b5060 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f869c009550 tx=0x7f869c0091b0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0102d60 msgr2=0x7f86a019fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0102d60 0x7f86a019fcd0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8690000c00 tx=0x7f86900047b0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 shutdown_connections 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f868c0b2ba0 0x7f868c0b5060 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f86a0108410 0x7f86a0199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f86a01023a0 0x7f86a019f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 --2- 172.21.15.59:0/1712477386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f86a0102d60 0x7f86a019fcd0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 >> 172.21.15.59:0/1712477386 conn(0x7f86a0079350 msgr2=0x7f86a0101690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 shutdown_connections 2024-05-08T05:31:05.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:05.296+0000 7f86a79f6700 1 -- 172.21.15.59:0/1712477386 wait complete. 2024-05-08T05:31:05.298 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 30 2024-05-08T05:31:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:05 smithi143 ceph-mon[34337]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:05 smithi096 ceph-mon[34253]: pgmap v48: 129 pgs: 129 active+clean; 1.4 MiB data, 353 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:05.954 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db / --timeout 60 --expiration 79.4 --await-for 42.0' 2024-05-08T05:31:05.955 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db / --timeout 60 --expiration 79.4 --await-for 42.0 2024-05-08T05:31:06.237 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:31:06.478 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:06 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1712477386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:31:06.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:06 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1712477386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:31:06.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:06 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1712477386' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:31:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:07 smithi143 ceph-mon[34337]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-08T05:31:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:07 smithi143 ceph-mon[34337]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-08T05:31:07.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:07 smithi059 ceph-mon[30304]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-08T05:31:07.521 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:07 smithi059 ceph-mon[30304]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-08T05:31:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:07 smithi096 ceph-mon[34253]: pgmap v49: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 23 KiB/s wr, 29 op/s 2024-05-08T05:31:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:07 smithi096 ceph-mon[34253]: mgrmap e26: x(active, since 92s), standbys: y, z 2024-05-08T05:31:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.060+0000 7f0ae8632700 1 -- 172.21.15.59:0/472775766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae0103580 msgr2=0x7f0ae0103a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.060+0000 7f0ae8632700 1 --2- 172.21.15.59:0/472775766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae0103580 0x7f0ae0103a00 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00099c0 tx=0x7f0ad0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.061+0000 7f0ae8632700 1 -- 172.21.15.59:0/472775766 shutdown_connections 2024-05-08T05:31:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.061+0000 7f0ae8632700 1 --2- 172.21.15.59:0/472775766 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ae0103f40 0x7f0ae0110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.061+0000 7f0ae8632700 1 --2- 172.21.15.59:0/472775766 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ae01095c0 0x7f0ae01099a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.061+0000 7f0ae8632700 1 --2- 172.21.15.59:0/472775766 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae0103580 0x7f0ae0103a00 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.061+0000 7f0ae8632700 1 -- 172.21.15.59:0/472775766 >> 172.21.15.59:0/472775766 conn(0x7f0ae007a690 msgr2=0x7f0ae007aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.061+0000 7f0ae8632700 1 -- 172.21.15.59:0/472775766 shutdown_connections 2024-05-08T05:31:08.063 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.061+0000 7f0ae8632700 1 -- 172.21.15.59:0/472775766 wait complete. 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.061+0000 7f0ae8632700 1 Processor -- start 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.061+0000 7f0ae8632700 1 -- start start 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae8632700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ae0103580 0x7f0ae00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae8632700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ae0103f40 0x7f0ae0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae8632700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae01095c0 0x7f0ae0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae8632700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01136c0 con 0x7f0ae01095c0 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae8632700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0113540 con 0x7f0ae0103f40 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae8632700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae0113840 con 0x7f0ae0103580 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae6bcf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae01095c0 0x7f0ae0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae6bcf700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae01095c0 0x7f0ae0076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50172/0 (socket says 172.21.15.59:50172) 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae6bcf700 1 -- 172.21.15.59:0/2564064850 learned_addr learned my addr 172.21.15.59:0/2564064850 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ae0103580 0x7f0ae00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ae0103f40 0x7f0ae0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.064 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae6bcf700 1 -- 172.21.15.59:0/2564064850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ae0103580 msgr2=0x7f0ae00757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae6bcf700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ae0103580 0x7f0ae00757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae6bcf700 1 -- 172.21.15.59:0/2564064850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ae0103f40 msgr2=0x7f0ae0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae6bcf700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ae0103f40 0x7f0ae0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae6bcf700 1 -- 172.21.15.59:0/2564064850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0ae01a7f40 con 0x7f0ae01095c0 2024-05-08T05:31:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ae6bcf700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae01095c0 0x7f0ae0076220 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800efc0 tx=0x7f0ad800c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.062+0000 7f0ad77fe700 1 -- 172.21.15.59:0/2564064850 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad80092e0 con 0x7f0ae01095c0 2024-05-08T05:31:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.063+0000 7f0ae8632700 1 -- 172.21.15.59:0/2564064850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0ae01a81d0 con 0x7f0ae01095c0 2024-05-08T05:31:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.063+0000 7f0ad77fe700 1 -- 172.21.15.59:0/2564064850 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad8004bb0 con 0x7f0ae01095c0 2024-05-08T05:31:08.065 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.063+0000 7f0ad77fe700 1 -- 172.21.15.59:0/2564064850 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad8004020 con 0x7f0ae01095c0 2024-05-08T05:31:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.063+0000 7f0ae8632700 1 -- 172.21.15.59:0/2564064850 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0ae01a8690 con 0x7f0ae01095c0 2024-05-08T05:31:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.065+0000 7f0ad77fe700 1 -- 172.21.15.59:0/2564064850 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0ad801b030 con 0x7f0ae01095c0 2024-05-08T05:31:08.066 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.066+0000 7f0ad77fe700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0acc0b2aa0 0x7f0acc0b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0acc0b2aa0 0x7f0acc0b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.067 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0ad77fe700 1 -- 172.21.15.59:0/2564064850 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0ad80e2440 con 0x7f0ae01095c0 2024-05-08T05:31:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0af2ec5b80 1 Processor -- start 2024-05-08T05:31:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0acc0b2aa0 0x7f0acc0b4f60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0adc005950 tx=0x7f0adc00b410 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0af2ec5b80 1 -- start start 2024-05-08T05:31:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c965c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c95fb00 0x56336c95ff80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c9604c0 0x56336c96c610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56336c8b1ad0 con 0x56336c95fb00 2024-05-08T05:31:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56336c816f30 con 0x56336c965850 2024-05-08T05:31:08.068 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56336c8ada90 con 0x56336c9604c0 2024-05-08T05:31:08.070 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.067+0000 7f0ae5bcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c95fb00 0x56336c95ff80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae5bcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c95fb00 0x56336c95ff80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50184/0 (socket says 172.21.15.59:50184) 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/2744473120 learned_addr learned my addr 172.21.15.59:0/2744473120 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c965c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae6bcf700 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c9604c0 0x56336c96c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/2744473120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c9604c0 msgr2=0x56336c96c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c9604c0 0x56336c96c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/2744473120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 msgr2=0x56336c965c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c965c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/2744473120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56336c7fea60 con 0x56336c95fb00 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae6bcf700 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c9604c0 0x56336c96c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c95fb00 0x56336c95ff80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006010 tx=0x7f0ad0005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50d0fe9cceb549e1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ad4ff9700 1 -- 172.21.15.59:0/2744473120 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0027070 con 0x56336c95fb00 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ad4ff9700 1 -- 172.21.15.59:0/2744473120 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad0009590 con 0x56336c95fb00 2024-05-08T05:31:08.071 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.068+0000 7f0ad4ff9700 1 -- 172.21.15.59:0/2744473120 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000b9f0 con 0x56336c95fb00 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.069+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/2744473120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c95fb00 msgr2=0x56336c95ff80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.069+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c95fb00 0x56336c95ff80 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006010 tx=0x7f0ad0005c80 comp rx=0 tx=0).stop 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.069+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/2744473120 shutdown_connections 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.069+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c9604c0 0x56336c96c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.069+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c965c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.069+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/2744473120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c95fb00 0x56336c95ff80 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.069+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/2744473120 >> 172.21.15.59:0/2744473120 conn(0x56336c88bd00 msgr2=0x56336c95a880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.069+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/2744473120 shutdown_connections 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.069+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/2744473120 wait complete. 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0af2ec5b80 1 -- start start 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0af2ec5b80 1 Processor -- start 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 0x56336c96c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c969560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.072 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c969aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56336c96cb50 con 0x56336c9604c0 2024-05-08T05:31:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56336c86b060 con 0x56336c965850 2024-05-08T05:31:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56336c96ccd0 con 0x56336c95fb00 2024-05-08T05:31:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0ae5bcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c969560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0ae5bcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c969560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50198/0 (socket says 172.21.15.59:50198) 2024-05-08T05:31:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/1678364875 learned_addr learned my addr 172.21.15.59:0/1678364875 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0af2ec5b80 10 client.0 _ll_register_callbacks cb 0x56336c845740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 0x56336c96c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.073 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.070+0000 7f0ae6bcf700 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c969aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/1678364875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 msgr2=0x56336c96c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 0x56336c96c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/1678364875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 msgr2=0x56336c969aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c969aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/1678364875 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56336c8688f0 con 0x56336c9604c0 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c969560 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00092c0 tx=0x7f0ad0005280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49865dd682dd3585 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ac9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1678364875 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0027040 con 0x56336c9604c0 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1678364875 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0ad000b790 con 0x56336c9604c0 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1678364875 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad0009590 con 0x56336c9604c0 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1678364875 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ad0039050 con 0x56336c9604c0 2024-05-08T05:31:08.076 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0af2ec5b80 10 client.15690 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.071+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1678364875 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56336c977170 con 0x56336c9604c0 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.072+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1678364875 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad0009590 con 0x56336c9604c0 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.072+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1678364875 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0ad002a070 con 0x56336c9604c0 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.072+0000 7f0af2ec5b80 10 client.15690 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.072+0000 7f0af2ec5b80 10 client.15690 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.072+0000 7f0af2ec5b80 20 client.15690 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.072+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56336c979730 0x56336c97bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.072+0000 7f0af2ec5b80 4 client.15690 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.072+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1678364875 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56336c8bf4b0 con 0x56336c979730 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.072+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56336c979730 0x56336c97bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.073+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56336c979730 0x56336c97bb20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc270da95f4a7daf server_cookie=dce7310a28efff9e in_seq=0 out_seq=0 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.073+0000 7f0ac9ffb700 10 client.15690 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.073+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1678364875 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56336c8bf4b0 con 0x56336c979730 2024-05-08T05:31:08.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.073+0000 7f0ac9ffb700 10 client.15690 handle_command_reply: tid=0 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.073+0000 7f0af2ec5b80 1 client.15690 shutdown 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.073+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1678364875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 msgr2=0x56336c969560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.073+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c969560 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00092c0 tx=0x7f0ad0005280 comp rx=0 tx=0).stop 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.073+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1678364875 shutdown_connections 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.073+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56336c979730 0x56336c97bb20 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.074+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c969aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.074+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 0x56336c96c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.074+0000 7f0ac9ffb700 0 client.15690 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.074+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1678364875 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c969560 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.074+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1678364875 >> 172.21.15.59:0/1678364875 conn(0x56336c88bd00 msgr2=0x56336c964bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.074+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1678364875 shutdown_connections 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.074+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1678364875 wait complete. 2024-05-08T05:31:08.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.074+0000 7f0af2ec5b80 20 client.15690 trim_cache size 0 max 16384 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.094+0000 7f0af2ec5b80 1 Processor -- start 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.094+0000 7f0af2ec5b80 1 -- start start 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 0x56336c975900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c95f200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c9813e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ad0d0 con 0x56336c9604c0 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01aec10 con 0x56336c965850 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ad7a0 con 0x56336c95fb00 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae5bcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c95f200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.097 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae5bcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c95f200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50210/0 (socket says 172.21.15.59:50210) 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/3964178396 learned_addr learned my addr 172.21.15.59:0/3964178396 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/3964178396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 0x56336c975900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae6bcf700 1 --2- 172.21.15.59:0/3964178396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c9813e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/3964178396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 msgr2=0x56336c975900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/3964178396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 0x56336c975900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/3964178396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 msgr2=0x56336c9813e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/3964178396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c9813e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/3964178396 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56336c975e40 con 0x56336c9604c0 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.095+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/3964178396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c95f200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006010 tx=0x7f0ad000fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1002a341fa7c31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0acbfff700 1 -- 172.21.15.59:0/3964178396 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0027070 con 0x56336c9604c0 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0acbfff700 1 -- 172.21.15.59:0/3964178396 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad0030070 con 0x56336c9604c0 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0acbfff700 1 -- 172.21.15.59:0/3964178396 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad000b9f0 con 0x56336c9604c0 2024-05-08T05:31:08.098 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/3964178396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 msgr2=0x56336c95f200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/3964178396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c95f200 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f0ad0006010 tx=0x7f0ad000fba0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/3964178396 shutdown_connections 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/3964178396 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c965850 0x56336c9813e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/3964178396 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c95fb00 0x56336c975900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/3964178396 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c95f200 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/3964178396 >> 172.21.15.59:0/3964178396 conn(0x56336c88bd00 msgr2=0x56336c95eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/3964178396 shutdown_connections 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.096+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/3964178396 wait complete. 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0af2ec5b80 1 -- start start 2024-05-08T05:31:08.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0af2ec5b80 1 Processor -- start 2024-05-08T05:31:08.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c95fb00 0x56336c971d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c972260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0af2ec5b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c965850 0x56336c961180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ae540 con 0x56336c9604c0 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ade70 con 0x56336c95fb00 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0af2ec5b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0ae01ab070 con 0x56336c965850 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0af2ec5b80 10 client.0 _ll_register_callbacks cb 0x56336c845740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae5bcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c972260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae5bcd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c972260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50222/0 (socket says 172.21.15.59:50222) 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/1844544063 learned_addr learned my addr 172.21.15.59:0/1844544063 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c95fb00 0x56336c971d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae6bcf700 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c965850 0x56336c961180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/1844544063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c965850 msgr2=0x56336c961180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c965850 0x56336c961180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/1844544063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c95fb00 msgr2=0x56336c971d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c95fb00 0x56336c971d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.097+0000 7f0ae5bcd700 1 -- 172.21.15.59:0/1844544063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56336c9618a0 con 0x56336c9604c0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0ae5bcd700 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c972260 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00092e0 tx=0x7f0ad0009cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53aa8c1674fd5e6c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0ac9ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1844544063 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ad0044030 con 0x56336c9604c0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1844544063 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0ad0005470 con 0x56336c9604c0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1844544063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ad0030070 con 0x56336c9604c0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1844544063 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0ad0027040 con 0x56336c9604c0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0af2ec5b80 10 client.15696 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1844544063 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x56336c962b80 con 0x56336c9604c0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1844544063 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ad0030070 con 0x56336c9604c0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1844544063 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0ad002a040 con 0x56336c9604c0 2024-05-08T05:31:08.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0af2ec5b80 10 client.15696 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:08.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0af2ec5b80 10 client.15696 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:31:08.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0af2ec5b80 20 client.15696 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:08.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56336c979730 0x56336c9e7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:08.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0af2ec5b80 4 client.15696 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await_for": 42.0}] 2024-05-08T05:31:08.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.098+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1844544063 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 79.4, "await_for": 42.0}) -- 0x56336c8b9550 con 0x56336c979730 2024-05-08T05:31:08.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.099+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56336c979730 0x56336c9e7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:08.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.099+0000 7f0ae63ce700 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56336c979730 0x56336c9e7070 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=756c0cf3bbde5b39 server_cookie=ddede1bee14b7258 in_seq=0 out_seq=0 2024-05-08T05:31:08.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.099+0000 7f0ac9ffb700 10 client.15696 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:08.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0ac9ffb700 1 -- 172.21.15.59:0/1844544063 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x56336c8b9550 con 0x56336c979730 2024-05-08T05:31:08.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0ac9ffb700 10 client.15696 handle_command_reply: tid=0 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 client.15696 shutdown 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1844544063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 msgr2=0x56336c972260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c972260 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f0ad00092e0 tx=0x7f0ad0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1844544063 shutdown_connections 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56336c979730 0x56336c9e7070 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56336c965850 0x56336c961180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56336c95fb00 0x56336c971d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 --2- 172.21.15.59:0/1844544063 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56336c9604c0 0x56336c972260 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1844544063 >> 172.21.15.59:0/1844544063 conn(0x56336c88bd00 msgr2=0x56336c95f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0ac9ffb700 0 client.15696 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1844544063 shutdown_connections 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 1 -- 172.21.15.59:0/1844544063 wait complete. 2024-05-08T05:31:08.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.107+0000 7f0af2ec5b80 20 client.15696 trim_cache size 0 max 16384 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 30, 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 6, 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "deb8584": { 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 6, 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:31:08.109 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 79.4, 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:08.110 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:08.111 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:31:08.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 -- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0acc0b2aa0 msgr2=0x7f0acc0b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:08.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0acc0b2aa0 0x7f0acc0b4f60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0adc005950 tx=0x7f0adc00b410 comp rx=0 tx=0).stop 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 -- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae01095c0 msgr2=0x7f0ae0076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae01095c0 0x7f0ae0076220 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f0ad800efc0 tx=0x7f0ad800c3a0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 -- 172.21.15.59:0/2564064850 shutdown_connections 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f0acc0b2aa0 0x7f0acc0b4f60 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0ae01095c0 0x7f0ae0076220 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f0ae0103580 0x7f0ae00757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 --2- 172.21.15.59:0/2564064850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0ae0103f40 0x7f0ae0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 -- 172.21.15.59:0/2564064850 >> 172.21.15.59:0/2564064850 conn(0x7f0ae007a690 msgr2=0x7f0ae0101ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 -- 172.21.15.59:0/2564064850 shutdown_connections 2024-05-08T05:31:08.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:08.108+0000 7f0ae8632700 1 -- 172.21.15.59:0/2564064850 wait complete. 2024-05-08T05:31:08.883 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'deb8584', quiesce took 6.0 seconds. Will release after: 19.4 2024-05-08T05:31:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:09 smithi143 ceph-mon[34337]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-08T05:31:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:09 smithi096 ceph-mon[34253]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-08T05:31:09.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:09 smithi059 ceph-mon[30304]: pgmap v50: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 25 op/s 2024-05-08T05:31:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.623+0000 7fe89c6c8640 1 -- 172.21.15.59:0/2394308822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894103c40 msgr2=0x7fe894104020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.623+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/2394308822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894103c40 0x7fe894104020 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fe890009900 tx=0x7fe89002a510 comp rx=0 tx=0).stop 2024-05-08T05:31:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.623+0000 7fe89c6c8640 1 -- 172.21.15.59:0/2394308822 shutdown_connections 2024-05-08T05:31:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.623+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/2394308822 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe89410cf80 0x7fe89410f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.625 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.623+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/2394308822 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894104560 0x7fe89410ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.623+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/2394308822 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894103c40 0x7fe894104020 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.623+0000 7fe89c6c8640 1 -- 172.21.15.59:0/2394308822 >> 172.21.15.59:0/2394308822 conn(0x7fe894078a30 msgr2=0x7fe894078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.623+0000 7fe89c6c8640 1 -- 172.21.15.59:0/2394308822 shutdown_connections 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.623+0000 7fe89c6c8640 1 -- 172.21.15.59:0/2394308822 wait complete. 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89c6c8640 1 Processor -- start 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89c6c8640 1 -- start start 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89c6c8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894103c40 0x7fe89419b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89c6c8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894104560 0x7fe89419b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89c6c8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe89410cf80 0x7fe89419fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89c6c8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe894112c30 con 0x7fe894104560 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89c6c8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe894112ab0 con 0x7fe894103c40 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89c6c8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe894112db0 con 0x7fe89410cf80 2024-05-08T05:31:09.626 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe899c63640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894104560 0x7fe89419b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe899c63640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894104560 0x7fe89419b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46098/0 (socket says 172.21.15.59:46098) 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe899c63640 1 -- 172.21.15.59:0/21954818 learned_addr learned my addr 172.21.15.59:0/21954818 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89ac65640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe89410cf80 0x7fe89419fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe89a464640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894103c40 0x7fe89419b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe899c63640 1 -- 172.21.15.59:0/21954818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe89410cf80 msgr2=0x7fe89419fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe899c63640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe89410cf80 0x7fe89419fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe899c63640 1 -- 172.21.15.59:0/21954818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894103c40 msgr2=0x7fe89419b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe899c63640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894103c40 0x7fe89419b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.624+0000 7fe899c63640 1 -- 172.21.15.59:0/21954818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8941a0350 con 0x7fe894104560 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.625+0000 7fe899c63640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894104560 0x7fe89419b8f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c890 tx=0x7fe88400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.625+0000 7fe88b7fe640 1 -- 172.21.15.59:0/21954818 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe884004220 con 0x7fe894104560 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.625+0000 7fe89c6c8640 1 -- 172.21.15.59:0/21954818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8941a0640 con 0x7fe894104560 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.625+0000 7fe88b7fe640 1 -- 172.21.15.59:0/21954818 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8840043c0 con 0x7fe894104560 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.625+0000 7fe89c6c8640 1 -- 172.21.15.59:0/21954818 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8941a7ed0 con 0x7fe894104560 2024-05-08T05:31:09.627 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.625+0000 7fe88b7fe640 1 -- 172.21.15.59:0/21954818 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88401d060 con 0x7fe894104560 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.627+0000 7fe89f3e8740 1 Processor -- start 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89f3e8740 1 -- start start 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 0x5582d91402f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d914abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d914d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5582d90a55d0 con 0x5582d9140830 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5582d905e890 con 0x5582d913ff10 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5582d90a1590 con 0x5582d914b0f0 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89a464640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d914abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89a464640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d914abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46110/0 (socket says 172.21.15.59:46110) 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89a464640 1 -- 172.21.15.59:0/2847429216 learned_addr learned my addr 172.21.15.59:0/2847429216 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:09.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89ac65640 1 --2- 172.21.15.59:0/2847429216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 0x5582d91402f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe88b7fe640 1 -- 172.21.15.59:0/21954818 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe8840079b0 con 0x7fe894104560 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89a464640 1 -- 172.21.15.59:0/2847429216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 msgr2=0x5582d914d4e0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89a464640 1 --2- 172.21.15.59:0/2847429216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d914d4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.628+0000 7fe89a464640 1 -- 172.21.15.59:0/2847429216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 msgr2=0x5582d91402f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.629+0000 7fe89a464640 1 --2- 172.21.15.59:0/2847429216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 0x5582d91402f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.629+0000 7fe89a464640 1 -- 172.21.15.59:0/2847429216 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d8db7210 con 0x5582d9140830 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.629+0000 7fe89a464640 1 --2- 172.21.15.59:0/2847429216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d914abb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe89002aa30 tx=0x7fe890031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a69d35649bcb740 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.629+0000 7fe888ff9640 1 -- 172.21.15.59:0/2847429216 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89003f070 con 0x5582d9140830 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.629+0000 7fe888ff9640 1 -- 172.21.15.59:0/2847429216 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8900316a0 con 0x5582d9140830 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.629+0000 7fe888ff9640 1 -- 172.21.15.59:0/2847429216 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe890035070 con 0x5582d9140830 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.629+0000 7fe88b7fe640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8800b2a10 0x7fe8800b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89ac65640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8800b2a10 0x7fe8800b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe88b7fe640 1 -- 172.21.15.59:0/21954818 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fe8840e2a70 con 0x7fe894104560 2024-05-08T05:31:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2847429216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 msgr2=0x5582d914abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2847429216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d914abb0 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fe89002aa30 tx=0x7fe890031cd0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2847429216 shutdown_connections 2024-05-08T05:31:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2847429216 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d914d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2847429216 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d914abb0 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2847429216 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 0x5582d91402f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2847429216 >> 172.21.15.59:0/2847429216 conn(0x5582d913c420 msgr2=0x5582d913c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:09.636 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89ac65640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8800b2a10 0x7fe8800b4ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe88c007f00 tx=0x7fe88c007520 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2847429216 shutdown_connections 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.630+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2847429216 wait complete. 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89f3e8740 1 -- start start 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89f3e8740 1 Processor -- start 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9147ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d9140830 0x5582d9148530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d914b0f0 0x5582d9148a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5582d914dcc0 con 0x5582d913ff10 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5582d9062370 con 0x5582d914b0f0 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5582d914de40 con 0x5582d9140830 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89f3e8740 10 client.0 _ll_register_callbacks cb 0x5582d8db6a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89a464640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9147ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89a464640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9147ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46126/0 (socket says 172.21.15.59:46126) 2024-05-08T05:31:09.637 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89a464640 1 -- 172.21.15.59:0/1443152437 learned_addr learned my addr 172.21.15.59:0/1443152437 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89ac65640 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d914b0f0 0x5582d9148a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe899c63640 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d9140830 0x5582d9148530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89a464640 1 -- 172.21.15.59:0/1443152437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d9140830 msgr2=0x5582d9148530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89a464640 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d9140830 0x5582d9148530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89a464640 1 -- 172.21.15.59:0/1443152437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d914b0f0 msgr2=0x5582d9148a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89a464640 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d914b0f0 0x5582d9148a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.631+0000 7fe89a464640 1 -- 172.21.15.59:0/1443152437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d8e83620 con 0x5582d913ff10 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89a464640 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9147ff0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe890031440 tx=0x7fe890032ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac87c3ce5874035b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe87dffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe87dffb640 1 -- 172.21.15.59:0/1443152437 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89004a070 con 0x5582d913ff10 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe87dffb640 1 -- 172.21.15.59:0/1443152437 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fe8900049b0 con 0x5582d913ff10 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1443152437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8900316a0 con 0x5582d913ff10 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe87dffb640 1 -- 172.21.15.59:0/1443152437 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe890056050 con 0x5582d913ff10 2024-05-08T05:31:09.638 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89f3e8740 10 client.15708 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1443152437 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582d9159af0 con 0x5582d913ff10 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe87dffb640 1 -- 172.21.15.59:0/1443152437 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8900316a0 con 0x5582d913ff10 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe87dffb640 1 -- 172.21.15.59:0/1443152437 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe890043400 con 0x5582d913ff10 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89f3e8740 10 client.15708 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89f3e8740 10 client.15708 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89f3e8740 20 client.15708 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5582d915b130 0x5582d915d520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89f3e8740 4 client.15708 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.632+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1443152437 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582d90b31e0 con 0x5582d915b130 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.633+0000 7fe899c63640 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5582d915b130 0x5582d915d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.633+0000 7fe87dffb640 10 client.15708 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.633+0000 7fe899c63640 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5582d915b130 0x5582d915d520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfce195728905b7 server_cookie=fd20ee3985701ee7 in_seq=0 out_seq=0 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe87dffb640 1 -- 172.21.15.59:0/1443152437 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5582d90b31e0 con 0x5582d915b130 2024-05-08T05:31:09.639 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe87dffb640 10 client.15708 handle_command_reply: tid=0 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 client.15708 shutdown 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1443152437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 msgr2=0x5582d9147ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9147ff0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7fe890031440 tx=0x7fe890032ed0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1443152437 shutdown_connections 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5582d915b130 0x5582d915d520 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d914b0f0 0x5582d9148a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d9140830 0x5582d9148530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1443152437 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9147ff0 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1443152437 >> 172.21.15.59:0/1443152437 conn(0x5582d913c420 msgr2=0x5582d91421a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe87dffb640 0 client.15708 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1443152437 shutdown_connections 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1443152437 wait complete. 2024-05-08T05:31:09.640 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.634+0000 7fe89f3e8740 20 client.15708 trim_cache size 0 max 16384 2024-05-08T05:31:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.649+0000 7fe89f3e8740 1 Processor -- start 2024-05-08T05:31:09.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.649+0000 7fe89f3e8740 1 -- start start 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 0x5582d9142840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d9148890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d9162e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8941b1de0 con 0x5582d9140830 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8941b3920 con 0x5582d913ff10 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8941b24b0 con 0x5582d914b0f0 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe899c63640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d9148890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe899c63640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d9148890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46138/0 (socket says 172.21.15.59:46138) 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe899c63640 1 -- 172.21.15.59:0/1770114609 learned_addr learned my addr 172.21.15.59:0/1770114609 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe89ac65640 1 --2- 172.21.15.59:0/1770114609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d9162e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe89a464640 1 --2- 172.21.15.59:0/1770114609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 0x5582d9142840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe899c63640 1 -- 172.21.15.59:0/1770114609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 msgr2=0x5582d9162e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe899c63640 1 --2- 172.21.15.59:0/1770114609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d9162e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe899c63640 1 -- 172.21.15.59:0/1770114609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 msgr2=0x5582d9142840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe899c63640 1 --2- 172.21.15.59:0/1770114609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 0x5582d9142840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe899c63640 1 -- 172.21.15.59:0/1770114609 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582d913c250 con 0x5582d9140830 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe899c63640 1 --2- 172.21.15.59:0/1770114609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d9148890 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe8840dfeb0 tx=0x7fe884004570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3e0a449533d81f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.650+0000 7fe87ffff640 1 -- 172.21.15.59:0/1770114609 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8840aa0e0 con 0x5582d9140830 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe87ffff640 1 -- 172.21.15.59:0/1770114609 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8840a99e0 con 0x5582d9140830 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe87ffff640 1 -- 172.21.15.59:0/1770114609 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8840e00d0 con 0x5582d9140830 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1770114609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 msgr2=0x5582d9148890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1770114609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d9148890 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7fe8840dfeb0 tx=0x7fe884004570 comp rx=0 tx=0).stop 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1770114609 shutdown_connections 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1770114609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d9162e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1770114609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d9140830 0x5582d9148890 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/1770114609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d913ff10 0x5582d9142840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1770114609 >> 172.21.15.59:0/1770114609 conn(0x5582d913c420 msgr2=0x5582d914a0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:09.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1770114609 shutdown_connections 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.651+0000 7fe89f3e8740 1 -- 172.21.15.59:0/1770114609 wait complete. 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89f3e8740 1 -- start start 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89f3e8740 1 Processor -- start 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9153360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d9140830 0x5582d91448e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89f3e8740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d9144e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe8941b3250 con 0x5582d913ff10 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe8941b2b80 con 0x5582d9140830 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89f3e8740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe8941afda0 con 0x5582d914b0f0 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89f3e8740 10 client.0 _ll_register_callbacks cb 0x7fe88c007860 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89a464640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89a464640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9153360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46150/0 (socket says 172.21.15.59:46150) 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89a464640 1 -- 172.21.15.59:0/2028018319 learned_addr learned my addr 172.21.15.59:0/2028018319 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:09.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe899c63640 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d9140830 0x5582d91448e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89ac65640 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d9144e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89a464640 1 -- 172.21.15.59:0/2028018319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 msgr2=0x5582d9144e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89a464640 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d9144e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89a464640 1 -- 172.21.15.59:0/2028018319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d9140830 msgr2=0x5582d91448e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89a464640 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d9140830 0x5582d91448e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89a464640 1 -- 172.21.15.59:0/2028018319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582d9142d80 con 0x5582d913ff10 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.652+0000 7fe89ac65640 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d9144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89a464640 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9153360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe890001eb0 tx=0x7fe8900436b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9882d8530abf920 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe87dffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe87dffb640 1 -- 172.21.15.59:0/2028018319 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89004a040 con 0x5582d913ff10 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe87dffb640 1 -- 172.21.15.59:0/2028018319 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fe8900314a0 con 0x5582d913ff10 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2028018319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe8840a99e0 con 0x5582d913ff10 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe87dffb640 1 -- 172.21.15.59:0/2028018319 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fe890063050 con 0x5582d913ff10 2024-05-08T05:31:09.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89f3e8740 10 client.15714 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2028018319 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x5582d91c55b0 con 0x5582d913ff10 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe87dffb640 1 -- 172.21.15.59:0/2028018319 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe8840a99e0 con 0x5582d913ff10 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe87dffb640 1 -- 172.21.15.59:0/2028018319 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fe89004f520 con 0x5582d913ff10 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89f3e8740 10 client.15714 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89f3e8740 10 client.15714 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89f3e8740 20 client.15714 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5582d91c5cf0 0x5582d91c8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89f3e8740 4 client.15714 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.653+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2028018319 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582d90ad050 con 0x5582d91c5cf0 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.654+0000 7fe899c63640 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5582d91c5cf0 0x5582d91c8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.654+0000 7fe899c63640 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5582d91c5cf0 0x5582d91c8110 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e42095b31e099ae1 server_cookie=71bf58b5cbb641f in_seq=0 out_seq=0 2024-05-08T05:31:09.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.654+0000 7fe87dffb640 10 client.15714 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.660+0000 7fe87dffb640 1 -- 172.21.15.59:0/2028018319 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582d90ad050 con 0x5582d91c5cf0 2024-05-08T05:31:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.660+0000 7fe87dffb640 10 client.15714 handle_command_reply: tid=0 2024-05-08T05:31:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.660+0000 7fe89f3e8740 1 client.15714 shutdown 2024-05-08T05:31:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2028018319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 msgr2=0x5582d9153360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9153360 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7fe890001eb0 tx=0x7fe8900436b0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2028018319 shutdown_connections 2024-05-08T05:31:09.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5582d91c5cf0 0x5582d91c8110 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5582d914b0f0 0x5582d9144e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5582d9140830 0x5582d91448e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe87dffb640 0 client.15714 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 --2- 172.21.15.59:0/2028018319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5582d913ff10 0x5582d9153360 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2028018319 >> 172.21.15.59:0/2028018319 conn(0x5582d913c420 msgr2=0x5582d9149820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2028018319 shutdown_connections 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 1 -- 172.21.15.59:0/2028018319 wait complete. 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89f3e8740 20 client.15714 trim_cache size 0 max 16384 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:31:09.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:31:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:31:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:31:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89c6c8640 1 -- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8800b2a10 msgr2=0x7fe8800b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8800b2a10 0x7fe8800b4ed0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe88c007f00 tx=0x7fe88c007520 comp rx=0 tx=0).stop 2024-05-08T05:31:09.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89c6c8640 1 -- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894104560 msgr2=0x7fe89419b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894104560 0x7fe89419b8f0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fe88400c890 tx=0x7fe88400cd50 comp rx=0 tx=0).stop 2024-05-08T05:31:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.661+0000 7fe89c6c8640 1 -- 172.21.15.59:0/21954818 shutdown_connections 2024-05-08T05:31:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.662+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe8800b2a10 0x7fe8800b4ed0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.662+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe89410cf80 0x7fe89419fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.662+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe894104560 0x7fe89419b8f0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.662+0000 7fe89c6c8640 1 --2- 172.21.15.59:0/21954818 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe894103c40 0x7fe89419b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.662+0000 7fe89c6c8640 1 -- 172.21.15.59:0/21954818 >> 172.21.15.59:0/21954818 conn(0x7fe894078a30 msgr2=0x7fe894108f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.662+0000 7fe89c6c8640 1 -- 172.21.15.59:0/21954818 shutdown_connections 2024-05-08T05:31:09.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:09.662+0000 7fe89c6c8640 1 -- 172.21.15.59:0/21954818 wait complete. 2024-05-08T05:31:10.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:10 smithi143 ceph-mon[34337]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-08T05:31:10.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:10 smithi096 ceph-mon[34253]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-08T05:31:10.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:10 smithi059 ceph-mon[30304]: pgmap v51: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-08T05:31:13.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:12 smithi143 ceph-mon[34337]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-08T05:31:13.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:12 smithi096 ceph-mon[34253]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-08T05:31:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:12 smithi059 ceph-mon[30304]: pgmap v52: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-08T05:31:15.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:14 smithi143 ceph-mon[34337]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-08T05:31:15.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:14 smithi096 ceph-mon[34253]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-08T05:31:15.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:14 smithi059 ceph-mon[30304]: pgmap v53: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 24 KiB/s wr, 25 op/s 2024-05-08T05:31:17.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:16 smithi143 ceph-mon[34337]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-08T05:31:17.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:16 smithi096 ceph-mon[34253]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-08T05:31:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:16 smithi059 ceph-mon[30304]: pgmap v54: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 24 KiB/s wr, 30 op/s 2024-05-08T05:31:19.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:18 smithi143 ceph-mon[34337]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-08T05:31:19.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:18 smithi096 ceph-mon[34253]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-08T05:31:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:18 smithi059 ceph-mon[30304]: pgmap v55: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-05-08T05:31:21.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:20 smithi143 ceph-mon[34337]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-08T05:31:21.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:20 smithi096 ceph-mon[34253]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-08T05:31:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:20 smithi059 ceph-mon[30304]: pgmap v56: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 13 op/s 2024-05-08T05:31:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.756+0000 7fc4c759e640 1 -- 172.21.15.59:0/3305897985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 msgr2=0x7fc4c810c770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.756+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3305897985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 0x7fc4c810c770 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0009900 tx=0x7fc4b002a480 comp rx=0 tx=0).stop 2024-05-08T05:31:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.756+0000 7fc4c759e640 1 -- 172.21.15.59:0/3305897985 shutdown_connections 2024-05-08T05:31:22.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.756+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3305897985 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4c810cd90 0x7fc4c810f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.756+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3305897985 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 0x7fc4c810c770 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.756+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3305897985 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c8103970 0x7fc4c8103d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.756+0000 7fc4c759e640 1 -- 172.21.15.59:0/3305897985 >> 172.21.15.59:0/3305897985 conn(0x7fc4c80fd1d0 msgr2=0x7fc4c80ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.756+0000 7fc4c759e640 1 -- 172.21.15.59:0/3305897985 shutdown_connections 2024-05-08T05:31:22.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.756+0000 7fc4c759e640 1 -- 172.21.15.59:0/3305897985 wait complete. 2024-05-08T05:31:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.757+0000 7fc4c759e640 1 Processor -- start 2024-05-08T05:31:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.757+0000 7fc4c759e640 1 -- start start 2024-05-08T05:31:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4c759e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4c8103970 0x7fc4c81990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4c759e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 0x7fc4c8199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 0x7fc4c8199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 0x7fc4c8199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52896/0 (socket says 172.21.15.59:52896) 2024-05-08T05:31:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4bffff640 1 -- 172.21.15.59:0/3411547239 learned_addr learned my addr 172.21.15.59:0/3411547239 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:22.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c810cd90 0x7fc4c819d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8112a40 con 0x7fc4c8104290 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81128c0 con 0x7fc4c810cd90 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c8112bc0 con 0x7fc4c8103970 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4c659c640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4c8103970 0x7fc4c81990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c810cd90 0x7fc4c819d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4bffff640 1 -- 172.21.15.59:0/3411547239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4c8103970 msgr2=0x7fc4c81990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4bffff640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4c8103970 0x7fc4c81990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4bffff640 1 -- 172.21.15.59:0/3411547239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c810cd90 msgr2=0x7fc4c819d950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4bffff640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c810cd90 0x7fc4c819d950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4bffff640 1 -- 172.21.15.59:0/3411547239 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4c819e080 con 0x7fc4c8104290 2024-05-08T05:31:22.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.758+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c810cd90 0x7fc4c819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:31:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.759+0000 7fc4bffff640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 0x7fc4c8199620 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0005940 tx=0x7fc4b0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.759+0000 7fc4bf7fe640 1 -- 172.21.15.59:0/3411547239 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b003f030 con 0x7fc4c8104290 2024-05-08T05:31:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.759+0000 7fc4bf7fe640 1 -- 172.21.15.59:0/3411547239 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b0003aa0 con 0x7fc4c8104290 2024-05-08T05:31:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.759+0000 7fc4bf7fe640 1 -- 172.21.15.59:0/3411547239 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0035070 con 0x7fc4c8104290 2024-05-08T05:31:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.759+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4c819e310 con 0x7fc4c8104290 2024-05-08T05:31:22.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.760+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4c81a5c00 con 0x7fc4c8104290 2024-05-08T05:31:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.764+0000 7fc4cfdfc740 1 Processor -- start 2024-05-08T05:31:22.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.764+0000 7fc4bf7fe640 1 -- 172.21.15.59:0/3411547239 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc4b0042030 con 0x7fc4c8104290 2024-05-08T05:31:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.764+0000 7fc4cfdfc740 1 -- start start 2024-05-08T05:31:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649ac4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 0x5637649b6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4cfdfc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5637649115d0 con 0x5637649b72c0 2024-05-08T05:31:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4cfdfc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5637648ca890 con 0x5637649aca00 2024-05-08T05:31:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4cfdfc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56376490d590 con 0x5637649ac0e0 2024-05-08T05:31:22.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4c6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649ac4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4c6d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649ac4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:33134/0 (socket says 172.21.15.59:33134) 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4c6d9d640 1 -- 172.21.15.59:0/3629603814 learned_addr learned my addr 172.21.15.59:0/3629603814 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4c659c640 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 0x5637649b6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4c6d9d640 1 -- 172.21.15.59:0/3629603814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 msgr2=0x5637649b6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 0x5637649b6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4c6d9d640 1 -- 172.21.15.59:0/3629603814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 msgr2=0x5637649b96b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4bffff640 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.765+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b96b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.766+0000 7fc4c6d9d640 1 -- 172.21.15.59:0/3629603814 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563764623210 con 0x5637649ac0e0 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.766+0000 7fc4bffff640 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:31:22.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.766+0000 7fc4bf7fe640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc4b40b2a50 0x7fc4b40b4f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.766+0000 7fc4c659c640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc4b40b2a50 0x7fc4b40b4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.766+0000 7fc4bf7fe640 1 -- 172.21.15.59:0/3411547239 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc4b0102c00 con 0x7fc4c8104290 2024-05-08T05:31:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.766+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649ac4c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00099b0 tx=0x7fc4c0009e70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bb575c015b582a4a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.766+0000 7fc4c659c640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc4b40b2a50 0x7fc4b40b4f10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5637649ad680 tx=0x7fc4b800b390 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.767+0000 7fc4bcff9640 1 -- 172.21.15.59:0/3629603814 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0024070 con 0x5637649ac0e0 2024-05-08T05:31:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.767+0000 7fc4bcff9640 1 -- 172.21.15.59:0/3629603814 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4c000c970 con 0x5637649ac0e0 2024-05-08T05:31:22.769 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.767+0000 7fc4bcff9640 1 -- 172.21.15.59:0/3629603814 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c00167a0 con 0x5637649ac0e0 2024-05-08T05:31:22.774 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3629603814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 msgr2=0x5637649ac4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649ac4c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc4c00099b0 tx=0x7fc4c0009e70 comp rx=0 tx=0).stop 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3629603814 shutdown_connections 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b96b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 0x5637649b6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/3629603814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649ac4c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3629603814 >> 172.21.15.59:0/3629603814 conn(0x5637649a85f0 msgr2=0x5637649a89d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3629603814 shutdown_connections 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3629603814 wait complete. 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.768+0000 7fc4cfdfc740 1 -- start start 2024-05-08T05:31:22.775 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4cfdfc740 1 Processor -- start 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 0x5637649b0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b4d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4c659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4c659c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649b04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:33140/0 (socket says 172.21.15.59:33140) 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4c659c640 1 -- 172.21.15.59:0/2112504632 learned_addr learned my addr 172.21.15.59:0/2112504632 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4bffff640 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 0x5637649b0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.769+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5637649b9e30 con 0x5637649b72c0 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5637648ce370 con 0x5637649aca00 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5637649b9fb0 con 0x5637649ac0e0 2024-05-08T05:31:22.776 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4c6d9d640 1 -- 172.21.15.59:0/2112504632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 msgr2=0x5637649b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4c6d9d640 1 -- 172.21.15.59:0/2112504632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 msgr2=0x5637649b0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 0x5637649b0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4c6d9d640 1 -- 172.21.15.59:0/2112504632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5637646ef620 con 0x5637649b72c0 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4cfdfc740 10 client.0 _ll_register_callbacks cb 0x563764622a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b4d30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009980 tx=0x7fc4c000cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bc2b7ba4a4df9f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4adffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.770+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc4c000c970 con 0x5637649b72c0 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.771+0000 7fc4adffb640 1 -- 172.21.15.59:0/2112504632 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4c0024070 con 0x5637649b72c0 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.771+0000 7fc4adffb640 1 -- 172.21.15.59:0/2112504632 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc4c0015f00 con 0x5637649b72c0 2024-05-08T05:31:22.777 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.771+0000 7fc4adffb640 1 -- 172.21.15.59:0/2112504632 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4c0016380 con 0x5637649b72c0 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.771+0000 7fc4cfdfc740 10 client.15723 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.771+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({fsmap=30}) -- 0x5637649c5930 con 0x5637649b72c0 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.772+0000 7fc4adffb640 1 -- 172.21.15.59:0/2112504632 <== mon.0 v2:172.21.15.59:3300/0 4 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc4c0032030 con 0x5637649b72c0 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.772+0000 7fc4cfdfc740 10 client.15723 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.772+0000 7fc4cfdfc740 10 client.15723 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.772+0000 7fc4cfdfc740 20 client.15723 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.772+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5637649c6c80 0x5637649c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.772+0000 7fc4cfdfc740 4 client.15723 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.772+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56376491f1e0 con 0x5637649c6c80 2024-05-08T05:31:22.778 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.772+0000 7fc4c659c640 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5637649c6c80 0x5637649c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.773+0000 7fc4adffb640 10 client.15723 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.773+0000 7fc4c659c640 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5637649c6c80 0x5637649c9070 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d9ddae936a7d890 server_cookie=2bc488f9b8e7f4d0 in_seq=0 out_seq=0 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.773+0000 7fc4adffb640 1 -- 172.21.15.59:0/2112504632 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56376491f1e0 con 0x5637649c6c80 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.773+0000 7fc4adffb640 10 client.15723 handle_command_reply: tid=0 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.773+0000 7fc4cfdfc740 1 client.15723 shutdown 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 msgr2=0x5637649b4d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b4d30 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0009980 tx=0x7fc4c000cea0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 shutdown_connections 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5637649c6c80 0x5637649c9070 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4adffb640 0 client.15723 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649b72c0 0x5637649b4d30 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649aca00 0x5637649b0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/2112504632 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649ac0e0 0x5637649b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 >> 172.21.15.59:0/2112504632 conn(0x5637649a85f0 msgr2=0x5637649ae320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 shutdown_connections 2024-05-08T05:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/2112504632 wait complete. 2024-05-08T05:31:22.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.774+0000 7fc4cfdfc740 20 client.15723 trim_cache size 0 max 16384 2024-05-08T05:31:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.805+0000 7fc4cfdfc740 1 Processor -- start 2024-05-08T05:31:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.805+0000 7fc4cfdfc740 1 -- start start 2024-05-08T05:31:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.805+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649ac0e0 0x5637649a75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.805+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649aca00 0x5637649b6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.805+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649b72c0 0x5637649ce720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649aca00 0x5637649b6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649aca00 0x5637649b6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52918/0 (socket says 172.21.15.59:52918) 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4bffff640 1 -- 172.21.15.59:0/3269298164 learned_addr learned my addr 172.21.15.59:0/3269298164 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3269298164 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81ad990 con 0x5637649aca00 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3269298164 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81af4d0 con 0x5637649ac0e0 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3269298164 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81ae060 con 0x5637649b72c0 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4c659c640 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649ac0e0 0x5637649a75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649b72c0 0x5637649ce720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4bffff640 1 -- 172.21.15.59:0/3269298164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649b72c0 msgr2=0x5637649ce720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4bffff640 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649b72c0 0x5637649ce720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4bffff640 1 -- 172.21.15.59:0/3269298164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649ac0e0 msgr2=0x5637649a75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4bffff640 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649ac0e0 0x5637649a75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4bffff640 1 -- 172.21.15.59:0/3269298164 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5637649a8f70 con 0x5637649aca00 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649b72c0 0x5637649ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4bffff640 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649aca00 0x5637649b6830 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00c9ac0 tx=0x7fc4b00c9540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50ff8d10af0f91f3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:22.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.806+0000 7fc4affff640 1 -- 172.21.15.59:0/3269298164 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b00cae60 con 0x5637649aca00 2024-05-08T05:31:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.807+0000 7fc4affff640 1 -- 172.21.15.59:0/3269298164 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b00d22d0 con 0x5637649aca00 2024-05-08T05:31:22.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.807+0000 7fc4affff640 1 -- 172.21.15.59:0/3269298164 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b0049040 con 0x5637649aca00 2024-05-08T05:31:22.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.807+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3269298164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649aca00 msgr2=0x5637649b6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.807+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649aca00 0x5637649b6830 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7fc4b00c9ac0 tx=0x7fc4b00c9540 comp rx=0 tx=0).stop 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.808+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3269298164 shutdown_connections 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.808+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649b72c0 0x5637649ce720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.808+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649aca00 0x5637649b6830 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.808+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/3269298164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649ac0e0 0x5637649a75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.808+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3269298164 >> 172.21.15.59:0/3269298164 conn(0x5637649a85f0 msgr2=0x5637649a6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.808+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3269298164 shutdown_connections 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.808+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/3269298164 wait complete. 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.808+0000 7fc4cfdfc740 1 -- start start 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4cfdfc740 1 Processor -- start 2024-05-08T05:31:22.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649ac0e0 0x5637649bf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4cfdfc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649aca00 0x5637649bfd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4c659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649ac0e0 0x5637649bf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4c659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649ac0e0 0x5637649bf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52932/0 (socket says 172.21.15.59:52932) 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4c659c640 1 -- 172.21.15.59:0/583285182 learned_addr learned my addr 172.21.15.59:0/583285182 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649b72c0 0x563764a29290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81aee00 con 0x5637649ac0e0 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81ae730 con 0x5637649b72c0 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.809+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc4c81ab950 con 0x5637649aca00 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4cfdfc740 10 client.0 _ll_register_callbacks cb 0x7fc4c0022070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4bffff640 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649aca00 0x5637649bfd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649b72c0 0x563764a29290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4c659c640 1 -- 172.21.15.59:0/583285182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649aca00 msgr2=0x5637649bfd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:22.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4c659c640 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649aca00 0x5637649bfd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4c659c640 1 -- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649b72c0 msgr2=0x563764a29290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4c659c640 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649b72c0 0x563764a29290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4c659c640 1 -- 172.21.15.59:0/583285182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563764a29ac0 con 0x5637649ac0e0 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4c6d9d640 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649b72c0 0x563764a29290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4c659c640 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649ac0e0 0x5637649bf840 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x5637649c9690 tx=0x7fc4b8014040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78aba4fbb70c68de server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4adffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4adffb640 1 -- 172.21.15.59:0/583285182 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4b802c020 con 0x5637649ac0e0 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4adffb640 1 -- 172.21.15.59:0/583285182 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc4b8005230 con 0x5637649ac0e0 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.810+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x563764919050 con 0x5637649ac0e0 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.811+0000 7fc4adffb640 1 -- 172.21.15.59:0/583285182 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7fc4b8037050 con 0x5637649ac0e0 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.811+0000 7fc4cfdfc740 10 client.15729 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.811+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563764a31df0 con 0x5637649ac0e0 2024-05-08T05:31:22.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.811+0000 7fc4adffb640 1 -- 172.21.15.59:0/583285182 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4b8004c50 con 0x5637649ac0e0 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.811+0000 7fc4adffb640 1 -- 172.21.15.59:0/583285182 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7fc4b8019040 con 0x5637649ac0e0 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.811+0000 7fc4cfdfc740 10 client.15729 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.811+0000 7fc4cfdfc740 10 client.15729 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.811+0000 7fc4cfdfc740 20 client.15729 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.812+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563764a32590 0x563764a349b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.812+0000 7fc4cfdfc740 4 client.15729 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.812+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563764919050 con 0x563764a32590 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.812+0000 7fc4bffff640 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563764a32590 0x563764a349b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.812+0000 7fc4bffff640 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563764a32590 0x563764a349b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=20d467382e382329 server_cookie=e1c41f219ea2dde6 in_seq=0 out_seq=0 2024-05-08T05:31:22.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.812+0000 7fc4adffb640 10 client.15729 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.816+0000 7fc4adffb640 1 -- 172.21.15.59:0/583285182 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563764919050 con 0x563764a32590 2024-05-08T05:31:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.816+0000 7fc4adffb640 10 client.15729 handle_command_reply: tid=0 2024-05-08T05:31:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.816+0000 7fc4cfdfc740 1 client.15729 shutdown 2024-05-08T05:31:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.817+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649ac0e0 msgr2=0x5637649bf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.817+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649ac0e0 0x5637649bf840 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x5637649c9690 tx=0x7fc4b8014040 comp rx=0 tx=0).stop 2024-05-08T05:31:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.817+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 shutdown_connections 2024-05-08T05:31:22.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.817+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563764a32590 0x563764a349b0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.817+0000 7fc4adffb640 0 client.15729 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.817+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5637649b72c0 0x563764a29290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.817+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5637649aca00 0x5637649bfd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.817+0000 7fc4cfdfc740 1 --2- 172.21.15.59:0/583285182 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5637649ac0e0 0x5637649bf840 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.817+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 >> 172.21.15.59:0/583285182 conn(0x5637649a85f0 msgr2=0x5637649a7750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.818+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 shutdown_connections 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.818+0000 7fc4cfdfc740 1 -- 172.21.15.59:0/583285182 wait complete. 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.818+0000 7fc4cfdfc740 20 client.15729 trim_cache size 0 max 16384 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:31:22.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:31:22.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.818+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc4b40b2a50 msgr2=0x7fc4b40b4f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc4b40b2a50 0x7fc4b40b4f10 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x5637649ad680 tx=0x7fc4b800b390 comp rx=0 tx=0).stop 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 msgr2=0x7fc4c8199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 0x7fc4c8199620 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0005940 tx=0x7fc4b0004270 comp rx=0 tx=0).stop 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 shutdown_connections 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc4b40b2a50 0x7fc4b40b4f10 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc4c810cd90 0x7fc4c819d950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc4c8104290 0x7fc4c8199620 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 --2- 172.21.15.59:0/3411547239 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc4c8103970 0x7fc4c81990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 >> 172.21.15.59:0/3411547239 conn(0x7fc4c80fd1d0 msgr2=0x7fc4c80feca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 shutdown_connections 2024-05-08T05:31:22.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:22.819+0000 7fc4c759e640 1 -- 172.21.15.59:0/3411547239 wait complete. 2024-05-08T05:31:23.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:22 smithi143 ceph-mon[34337]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:31:23.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:22 smithi096 ceph-mon[34253]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:31:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:22 smithi059 ceph-mon[30304]: pgmap v57: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:31:25.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:24 smithi143 ceph-mon[34337]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:31:25.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:24 smithi096 ceph-mon[34253]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:31:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:24 smithi059 ceph-mon[30304]: pgmap v58: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:31:27.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:26 smithi143 ceph-mon[34337]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:27.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:26 smithi096 ceph-mon[34253]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:27.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:26 smithi059 ceph-mon[30304]: pgmap v59: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:28.284 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:31:28.575 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:31:28.977 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:28 smithi059 ceph-mon[30304]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:29.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:28 smithi143 ceph-mon[34337]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:29.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:28 smithi096 ceph-mon[34253]: pgmap v60: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:30.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.545+0000 7f8bbe48b700 1 -- 172.21.15.59:0/2800695971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb8109510 msgr2=0x7f8bb81098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:30.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.545+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/2800695971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb8109510 0x7f8bb81098f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f8ba8009990 tx=0x7f8ba8009ca0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.545+0000 7f8bbe48b700 1 -- 172.21.15.59:0/2800695971 shutdown_connections 2024-05-08T05:31:30.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.545+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/2800695971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb806bb40 0x7f8bb8110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.545+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/2800695971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb8109510 0x7f8bb81098f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.545+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/2800695971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb806b180 0x7f8bb806b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.545+0000 7f8bbe48b700 1 -- 172.21.15.59:0/2800695971 >> 172.21.15.59:0/2800695971 conn(0x7f8bb8079350 msgr2=0x7f8bb8079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:30.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.545+0000 7f8bbe48b700 1 -- 172.21.15.59:0/2800695971 shutdown_connections 2024-05-08T05:31:30.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.545+0000 7f8bbe48b700 1 -- 172.21.15.59:0/2800695971 wait complete. 2024-05-08T05:31:30.551 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.546+0000 7f8bbe48b700 1 Processor -- start 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.546+0000 7f8bbe48b700 1 -- start start 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bbe48b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb806b180 0x7f8bb819bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bbe48b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb806bb40 0x7f8bb819c280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bbe48b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb8109510 0x7f8bb81a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bbe48b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8113620 con 0x7f8bb806b180 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bbe48b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81134a0 con 0x7f8bb806bb40 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bbe48b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81137a0 con 0x7f8bb8109510 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bb7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb806b180 0x7f8bb819bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bb7fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb806b180 0x7f8bb819bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33190/0 (socket says 172.21.15.59:33190) 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bb7fff700 1 -- 172.21.15.59:0/1058588881 learned_addr learned my addr 172.21.15.59:0/1058588881 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bbca28700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb8109510 0x7f8bb81a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.547+0000 7f8bb77fe700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb806bb40 0x7f8bb819c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bb7fff700 1 -- 172.21.15.59:0/1058588881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb8109510 msgr2=0x7f8bb81a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:30.552 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bb7fff700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb8109510 0x7f8bb81a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bb7fff700 1 -- 172.21.15.59:0/1058588881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb806bb40 msgr2=0x7f8bb819c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bb7fff700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb806bb40 0x7f8bb819c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bb7fff700 1 -- 172.21.15.59:0/1058588881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb81a0d50 con 0x7f8bb806b180 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bb7fff700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb806b180 0x7f8bb819bd40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80098e0 tx=0x7f8ba800f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bb57fa700 1 -- 172.21.15.59:0/1058588881 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8027070 con 0x7f8bb806b180 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bb57fa700 1 -- 172.21.15.59:0/1058588881 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba800be70 con 0x7f8bb806b180 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb81a0fe0 con 0x7f8bb806b180 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.548+0000 7f8bb57fa700 1 -- 172.21.15.59:0/1058588881 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba80173f0 con 0x7f8bb806b180 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.549+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb81a88c0 con 0x7f8bb806b180 2024-05-08T05:31:30.553 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.550+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8bb806a190 con 0x7f8bb806b180 2024-05-08T05:31:30.554 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.552+0000 7f8bb57fa700 1 -- 172.21.15.59:0/1058588881 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8ba802a070 con 0x7f8bb806b180 2024-05-08T05:31:30.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.554+0000 7f8bb57fa700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8ba00b2a70 0x7f8ba00b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:30.555 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.554+0000 7f8bb77fe700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8ba00b2a70 0x7f8ba00b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:30.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.554+0000 7f8bb57fa700 1 -- 172.21.15.59:0/1058588881 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f8ba80e3fb0 con 0x7f8bb806b180 2024-05-08T05:31:30.556 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.555+0000 7f8bb77fe700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8ba00b2a70 0x7f8ba00b4f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bb81044d0 tx=0x7f8bac009260 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:30.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.560+0000 7f8bb57fa700 1 -- 172.21.15.59:0/1058588881 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f8ba8031070 con 0x7f8bb806b180 2024-05-08T05:31:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.921+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8bb81a8af0 con 0x7f8bb806b180 2024-05-08T05:31:30.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.922+0000 7f8bb57fa700 1 -- 172.21.15.59:0/1058588881 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) ==== 94+0+11929 (secure 0 0 0) 0x7f8ba8013070 con 0x7f8bb806b180 2024-05-08T05:31:30.924 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:31:30.924 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":30,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":30,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:29:31.333500+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:31:30.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8ba00b2a70 msgr2=0x7f8ba00b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:30.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8ba00b2a70 0x7f8ba00b4f30 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f8bb81044d0 tx=0x7f8bac009260 comp rx=0 tx=0).stop 2024-05-08T05:31:30.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb806b180 msgr2=0x7f8bb819bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:30.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb806b180 0x7f8bb819bd40 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f8ba80098e0 tx=0x7f8ba800f6e0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 shutdown_connections 2024-05-08T05:31:30.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8ba00b2a70 0x7f8ba00b4f30 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8bb8109510 0x7f8bb81a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8bb806b180 0x7f8bb819bd40 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 --2- 172.21.15.59:0/1058588881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8bb806bb40 0x7f8bb819c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:30.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 >> 172.21.15.59:0/1058588881 conn(0x7f8bb8079350 msgr2=0x7f8bb810e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:30.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 shutdown_connections 2024-05-08T05:31:30.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:30.925+0000 7f8bbe48b700 1 -- 172.21.15.59:0/1058588881 wait complete. 2024-05-08T05:31:30.927 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 30 2024-05-08T05:31:31.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:30 smithi059 ceph-mon[30304]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:31.216 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:30 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1058588881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:31:31.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:30 smithi143 ceph-mon[34337]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:31.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:30 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1058588881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:31:31.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:30 smithi096 ceph-mon[34253]: pgmap v61: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:31.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:30 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1058588881' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:31:31.773 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db --set-id deb8584 --release --await' 2024-05-08T05:31:31.773 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db --set-id deb8584 --release --await 2024-05-08T05:31:32.120 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:31:33.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:32 smithi143 ceph-mon[34337]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:33.264 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:32 smithi059 ceph-mon[30304]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:33.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:32 smithi096 ceph-mon[34253]: pgmap v62: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:34.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.018+0000 7f0351919700 1 -- 172.21.15.59:0/1990775468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f034c102e00 msgr2=0x7f034c1031e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.018+0000 7f0351919700 1 --2- 172.21.15.59:0/1990775468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f034c102e00 0x7f034c1031e0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f033c0099c0 tx=0x7f033c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.018+0000 7f0351919700 1 -- 172.21.15.59:0/1990775468 shutdown_connections 2024-05-08T05:31:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.018+0000 7f0351919700 1 --2- 172.21.15.59:0/1990775468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034c107760 0x7f034c10f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.018+0000 7f0351919700 1 --2- 172.21.15.59:0/1990775468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f034c102e00 0x7f034c1031e0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.018+0000 7f0351919700 1 --2- 172.21.15.59:0/1990775468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f034c1037b0 0x7f034c107030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.018+0000 7f0351919700 1 -- 172.21.15.59:0/1990775468 >> 172.21.15.59:0/1990775468 conn(0x7f034c078a10 msgr2=0x7f034c0ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.018+0000 7f0351919700 1 -- 172.21.15.59:0/1990775468 shutdown_connections 2024-05-08T05:31:34.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.018+0000 7f0351919700 1 -- 172.21.15.59:0/1990775468 wait complete. 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.019+0000 7f0351919700 1 Processor -- start 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.019+0000 7f0351919700 1 -- start start 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.019+0000 7f0351919700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034c102e00 0x7f034c109380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.019+0000 7f0351919700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f034c1037b0 0x7f034c1098c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.019+0000 7f0351919700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f034c107760 0x7f034c109e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.019+0000 7f0351919700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f034c112550 con 0x7f034c107760 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.019+0000 7f0351919700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f034c1123d0 con 0x7f034c1037b0 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.019+0000 7f0351919700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f034c1126d0 con 0x7f034c102e00 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034a7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f034c1037b0 0x7f034c1098c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034a7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f034c1037b0 0x7f034c1098c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:44022/0 (socket says 172.21.15.59:44022) 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034a7fc700 1 -- 172.21.15.59:0/705971971 learned_addr learned my addr 172.21.15.59:0/705971971 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034b7fe700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f034c107760 0x7f034c109e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034affd700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034c102e00 0x7f034c109380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034a7fc700 1 -- 172.21.15.59:0/705971971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034c102e00 msgr2=0x7f034c109380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034a7fc700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034c102e00 0x7f034c109380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034a7fc700 1 -- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f034c107760 msgr2=0x7f034c109e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034a7fc700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f034c107760 0x7f034c109e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034a7fc700 1 -- 172.21.15.59:0/705971971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f034c1a8050 con 0x7f034c1037b0 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034affd700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034c102e00 0x7f034c109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.020+0000 7f034a7fc700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f034c1037b0 0x7f034c1098c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f034000e9a0 tx=0x7f034000ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.021+0000 7f0350917700 1 -- 172.21.15.59:0/705971971 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034000cbd0 con 0x7f034c1037b0 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.021+0000 7f0350917700 1 -- 172.21.15.59:0/705971971 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034000cd70 con 0x7f034c1037b0 2024-05-08T05:31:34.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.021+0000 7f0350917700 1 -- 172.21.15.59:0/705971971 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0340018680 con 0x7f034c1037b0 2024-05-08T05:31:34.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.021+0000 7f0351919700 1 -- 172.21.15.59:0/705971971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f034c1a8340 con 0x7f034c1037b0 2024-05-08T05:31:34.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.022+0000 7f0351919700 1 -- 172.21.15.59:0/705971971 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f034c1a8740 con 0x7f034c1037b0 2024-05-08T05:31:34.026 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.025+0000 7f0350917700 1 -- 172.21.15.59:0/705971971 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f0340010430 con 0x7f034c1037b0 2024-05-08T05:31:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.025+0000 7f035c1acb80 1 Processor -- start 2024-05-08T05:31:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.025+0000 7f035c1acb80 1 -- start start 2024-05-08T05:31:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb731b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb72cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb72d5d0 0x563edb72da30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f035c1acb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563edb676960 con 0x563edb7320d0 2024-05-08T05:31:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f035c1acb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563edb6158c0 con 0x563edb72d5d0 2024-05-08T05:31:34.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f035c1acb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563edb672920 con 0x563edb7317b0 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f034b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb731b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f034a7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb72d5d0 0x563edb72da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f034b7fe700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb731b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:56540/0 (socket says 172.21.15.59:56540) 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f034b7fe700 1 -- 172.21.15.59:0/733944604 learned_addr learned my addr 172.21.15.59:0/733944604 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.026+0000 7f034affd700 1 --2- 172.21.15.59:0/733944604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb72cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f034a7fc700 1 -- 172.21.15.59:0/733944604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 msgr2=0x563edb731b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f034a7fc700 1 --2- 172.21.15.59:0/733944604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb731b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f034a7fc700 1 -- 172.21.15.59:0/733944604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 msgr2=0x563edb72cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f034a7fc700 1 --2- 172.21.15.59:0/733944604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb72cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f034a7fc700 1 -- 172.21.15.59:0/733944604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563edb5c39d0 con 0x563edb72d5d0 2024-05-08T05:31:34.029 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f034affd700 1 --2- 172.21.15.59:0/733944604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb72cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:31:34.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f0350917700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03340b2a70 0x7f03340b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f0350917700 1 -- 172.21.15.59:0/705971971 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f03400e2cf0 con 0x7f034c1037b0 2024-05-08T05:31:34.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f034a7fc700 1 --2- 172.21.15.59:0/733944604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb72d5d0 0x563edb72da30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03400e0090 tx=0x7f03400e6e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d1719c39adf5ca server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:34.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f034b7fe700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03340b2a70 0x7f03340b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.027+0000 7f0331ffb700 1 -- 172.21.15.59:0/733944604 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400a5ae0 con 0x563edb72d5d0 2024-05-08T05:31:34.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f0331ffb700 1 -- 172.21.15.59:0/733944604 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400a9730 con 0x563edb72d5d0 2024-05-08T05:31:34.030 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f0331ffb700 1 -- 172.21.15.59:0/733944604 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400e0520 con 0x563edb72d5d0 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f035c1acb80 1 -- 172.21.15.59:0/733944604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb72d5d0 msgr2=0x563edb72da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f035c1acb80 1 --2- 172.21.15.59:0/733944604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb72d5d0 0x563edb72da30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f03400e0090 tx=0x7f03400e6e60 comp rx=0 tx=0).stop 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f034b7fe700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03340b2a70 0x7f03340b4f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03440075c0 tx=0x7f034400b040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f035c1acb80 1 -- 172.21.15.59:0/733944604 shutdown_connections 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f035c1acb80 1 --2- 172.21.15.59:0/733944604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb72d5d0 0x563edb72da30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f035c1acb80 1 --2- 172.21.15.59:0/733944604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb731b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f035c1acb80 1 --2- 172.21.15.59:0/733944604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb72cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.028+0000 7f035c1acb80 1 -- 172.21.15.59:0/733944604 >> 172.21.15.59:0/733944604 conn(0x563edb650200 msgr2=0x563edb71efe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.029+0000 7f035c1acb80 1 -- 172.21.15.59:0/733944604 shutdown_connections 2024-05-08T05:31:34.031 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.029+0000 7f035c1acb80 1 -- 172.21.15.59:0/733944604 wait complete. 2024-05-08T05:31:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.029+0000 7f035c1acb80 1 -- start start 2024-05-08T05:31:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.029+0000 7f035c1acb80 1 Processor -- start 2024-05-08T05:31:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 0x563edb738e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 0x563edb736230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb736770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f035c1acb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563edb739ca0 con 0x563edb7320d0 2024-05-08T05:31:34.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f035c1acb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563edb5dcd30 con 0x563edb7317b0 2024-05-08T05:31:34.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f035c1acb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563edb739e20 con 0x563edb72d5d0 2024-05-08T05:31:34.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f035c1acb80 10 client.0 _ll_register_callbacks cb 0x563edb60ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:34.033 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f034b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb736770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f034b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb736770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33234/0 (socket says 172.21.15.59:33234) 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f034b7fe700 1 -- 172.21.15.59:0/3103316040 learned_addr learned my addr 172.21.15.59:0/3103316040 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f034affd700 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 0x563edb738e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f034b7fe700 1 -- 172.21.15.59:0/3103316040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 msgr2=0x563edb738e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f034b7fe700 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 0x563edb738e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f034b7fe700 1 -- 172.21.15.59:0/3103316040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 msgr2=0x563edb736230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f034b7fe700 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 0x563edb736230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.030+0000 7f034b7fe700 1 -- 172.21.15.59:0/3103316040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563edb612790 con 0x563edb7320d0 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.031+0000 7f034b7fe700 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb736770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0344016e00 tx=0x7f0344004b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a68f40a373a0268 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.031+0000 7f032affd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.031+0000 7f032affd700 1 -- 172.21.15.59:0/3103316040 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344027020 con 0x563edb7320d0 2024-05-08T05:31:34.034 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.031+0000 7f032affd700 1 -- 172.21.15.59:0/3103316040 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0344023450 con 0x563edb7320d0 2024-05-08T05:31:34.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.031+0000 7f035c1acb80 1 -- 172.21.15.59:0/3103316040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03400a9730 con 0x563edb7320d0 2024-05-08T05:31:34.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.031+0000 7f032affd700 1 -- 172.21.15.59:0/3103316040 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0344032050 con 0x563edb7320d0 2024-05-08T05:31:34.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.031+0000 7f035c1acb80 10 client.15747 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:34.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.031+0000 7f035c1acb80 1 -- 172.21.15.59:0/3103316040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563edb744520 con 0x563edb7320d0 2024-05-08T05:31:34.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.032+0000 7f032affd700 1 -- 172.21.15.59:0/3103316040 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0344009290 con 0x563edb7320d0 2024-05-08T05:31:34.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.032+0000 7f032affd700 1 -- 172.21.15.59:0/3103316040 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f0344022670 con 0x563edb7320d0 2024-05-08T05:31:34.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.032+0000 7f035c1acb80 10 client.15747 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:34.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.032+0000 7f035c1acb80 10 client.15747 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:31:34.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.032+0000 7f035c1acb80 20 client.15747 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:34.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.032+0000 7f035c1acb80 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563edb746bb0 0x563edb748fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.032+0000 7f035c1acb80 4 client.15747 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:31:34.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.032+0000 7f035c1acb80 1 -- 172.21.15.59:0/3103316040 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f03400a9730 con 0x563edb746bb0 2024-05-08T05:31:34.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.033+0000 7f034affd700 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563edb746bb0 0x563edb748fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.033+0000 7f032affd700 10 client.15747 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:34.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.033+0000 7f034affd700 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563edb746bb0 0x563edb748fc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f77caa42d52d505d server_cookie=73a4aaad30f19bdf in_seq=0 out_seq=0 2024-05-08T05:31:34.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.034+0000 7f032affd700 1 -- 172.21.15.59:0/3103316040 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f03400a9730 con 0x563edb746bb0 2024-05-08T05:31:34.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.034+0000 7f032affd700 10 client.15747 handle_command_reply: tid=0 2024-05-08T05:31:34.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.034+0000 7f035c1acb80 1 client.15747 shutdown 2024-05-08T05:31:34.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 -- 172.21.15.59:0/3103316040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 msgr2=0x563edb736770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:34.037 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb736770 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f0344016e00 tx=0x7f0344004b40 comp rx=0 tx=0).stop 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 -- 172.21.15.59:0/3103316040 shutdown_connections 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563edb746bb0 0x563edb748fc0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb736770 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 0x563edb738e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f032affd700 0 client.15747 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 --2- 172.21.15.59:0/3103316040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 0x563edb736230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 -- 172.21.15.59:0/3103316040 >> 172.21.15.59:0/3103316040 conn(0x563edb650200 msgr2=0x563edb732e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 -- 172.21.15.59:0/3103316040 shutdown_connections 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 1 -- 172.21.15.59:0/3103316040 wait complete. 2024-05-08T05:31:34.038 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.035+0000 7f035c1acb80 20 client.15747 trim_cache size 0 max 16384 2024-05-08T05:31:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.075+0000 7f035c1acb80 1 Processor -- start 2024-05-08T05:31:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.075+0000 7f035c1acb80 1 -- start start 2024-05-08T05:31:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.075+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb72d5d0 0x563edb742d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.075+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb72ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.075+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7320d0 0x563edb72cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.075+0000 7f035c1acb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f034c1ad320 con 0x563edb72d5d0 2024-05-08T05:31:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.075+0000 7f035c1acb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f034c1aee60 con 0x563edb7320d0 2024-05-08T05:31:34.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.075+0000 7f035c1acb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f034c1ad9f0 con 0x563edb7317b0 2024-05-08T05:31:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb72ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034b7fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7320d0 0x563edb72cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb72d5d0 0x563edb742d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034a7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb72ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:56552/0 (socket says 172.21.15.59:56552) 2024-05-08T05:31:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034a7fc700 1 -- 172.21.15.59:0/4269148894 learned_addr learned my addr 172.21.15.59:0/4269148894 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:34.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034a7fc700 1 -- 172.21.15.59:0/4269148894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7320d0 msgr2=0x563edb72cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034a7fc700 1 --2- 172.21.15.59:0/4269148894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7320d0 0x563edb72cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034a7fc700 1 -- 172.21.15.59:0/4269148894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb72d5d0 msgr2=0x563edb742d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034a7fc700 1 --2- 172.21.15.59:0/4269148894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb72d5d0 0x563edb742d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034a7fc700 1 -- 172.21.15.59:0/4269148894 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563edb749640 con 0x563edb7317b0 2024-05-08T05:31:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.076+0000 7f034a7fc700 1 --2- 172.21.15.59:0/4269148894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb72ca40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0340000c00 tx=0x7f03400e6810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7f95101628578c4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.077+0000 7f03317fa700 1 -- 172.21.15.59:0/4269148894 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400a9390 con 0x563edb7317b0 2024-05-08T05:31:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.077+0000 7f03317fa700 1 -- 172.21.15.59:0/4269148894 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400e5040 con 0x563edb7317b0 2024-05-08T05:31:34.079 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.077+0000 7f03317fa700 1 -- 172.21.15.59:0/4269148894 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f03400a9570 con 0x563edb7317b0 2024-05-08T05:31:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 -- 172.21.15.59:0/4269148894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 msgr2=0x563edb72ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 --2- 172.21.15.59:0/4269148894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb72ca40 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f0340000c00 tx=0x7f03400e6810 comp rx=0 tx=0).stop 2024-05-08T05:31:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 -- 172.21.15.59:0/4269148894 shutdown_connections 2024-05-08T05:31:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 --2- 172.21.15.59:0/4269148894 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7320d0 0x563edb72cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 --2- 172.21.15.59:0/4269148894 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb72d5d0 0x563edb742d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 --2- 172.21.15.59:0/4269148894 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb7317b0 0x563edb72ca40 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 -- 172.21.15.59:0/4269148894 >> 172.21.15.59:0/4269148894 conn(0x563edb650200 msgr2=0x563edb72c330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 -- 172.21.15.59:0/4269148894 shutdown_connections 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 -- 172.21.15.59:0/4269148894 wait complete. 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.078+0000 7f035c1acb80 1 -- start start 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f035c1acb80 1 Processor -- start 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 0x563edb73f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 0x563edb73a0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f035c1acb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb73a610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f035c1acb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f034c1ae790 con 0x563edb7320d0 2024-05-08T05:31:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f034a7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 0x563edb73a0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f034a7fc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 0x563edb73a0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:44060/0 (socket says 172.21.15.59:44060) 2024-05-08T05:31:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f034b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb73a610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f034a7fc700 1 -- 172.21.15.59:0/1083614191 learned_addr learned my addr 172.21.15.59:0/1083614191 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f034affd700 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 0x563edb73f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f034c1ae0c0 con 0x563edb7317b0 2024-05-08T05:31:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.079+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f034c1ab2c0 con 0x563edb72d5d0 2024-05-08T05:31:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f035c1acb80 10 client.0 _ll_register_callbacks cb 0x563edb60ea20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:34.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f034b7fe700 1 -- 172.21.15.59:0/1083614191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 msgr2=0x563edb73f4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f034b7fe700 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 0x563edb73f4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f034b7fe700 1 -- 172.21.15.59:0/1083614191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 msgr2=0x563edb73a0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f034b7fe700 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 0x563edb73a0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f034b7fe700 1 -- 172.21.15.59:0/1083614191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563edb73ac50 con 0x563edb7320d0 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f034affd700 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 0x563edb73f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f034a7fc700 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 0x563edb73a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f034b7fe700 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb73a610 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0344009140 tx=0x7f03440094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82f68878031b2c91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f032affd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f032affd700 1 -- 172.21.15.59:0/1083614191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0344027070 con 0x563edb7320d0 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f032affd700 1 -- 172.21.15.59:0/1083614191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f0344004bb0 con 0x563edb7320d0 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.080+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03400e5040 con 0x563edb7320d0 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f032affd700 1 -- 172.21.15.59:0/1083614191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f0344005e60 con 0x563edb7320d0 2024-05-08T05:31:34.083 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f035c1acb80 10 client.15753 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:34.084 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x563edb7b21f0 con 0x563edb7320d0 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f032affd700 1 -- 172.21.15.59:0/1083614191 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03400e5040 con 0x563edb7320d0 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f032affd700 1 -- 172.21.15.59:0/1083614191 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f034403c020 con 0x563edb7320d0 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f035c1acb80 10 client.15753 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f035c1acb80 10 client.15753 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f035c1acb80 20 client.15753 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f035c1acb80 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563edb7b28f0 0x563edb7b4d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f035c1acb80 4 client.15753 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "deb8584", "release": true, "await": true}] 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.081+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "deb8584", "release": true, "await": true}) -- 0x563edb6f0750 con 0x563edb7b28f0 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.082+0000 7f034affd700 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563edb7b28f0 0x563edb7b4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.082+0000 7f034affd700 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563edb7b28f0 0x563edb7b4d00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b23ea19192dc9e6 server_cookie=94188a585de5aa89 in_seq=0 out_seq=0 2024-05-08T05:31:34.085 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.082+0000 7f032affd700 10 client.15753 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:34.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.084+0000 7f032affd700 1 -- 172.21.15.59:0/1083614191 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+587 (crc 0 0 0) 0x563edb6f0750 con 0x563edb7b28f0 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.084+0000 7f032affd700 10 client.15753 handle_command_reply: tid=0 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.084+0000 7f035c1acb80 1 client.15753 shutdown 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 msgr2=0x563edb73a610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb73a610 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f0344009140 tx=0x7f03440094c0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 shutdown_connections 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563edb7b28f0 0x563edb7b4d00 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563edb7320d0 0x563edb73a610 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563edb72d5d0 0x563edb73f4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 --2- 172.21.15.59:0/1083614191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563edb7317b0 0x563edb73a0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f032affd700 0 client.15753 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 >> 172.21.15.59:0/1083614191 conn(0x563edb650200 msgr2=0x563edb72cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 shutdown_connections 2024-05-08T05:31:34.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 1 -- 172.21.15.59:0/1083614191 wait complete. 2024-05-08T05:31:34.088 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.085+0000 7f035c1acb80 20 client.15753 trim_cache size 0 max 16384 2024-05-08T05:31:34.088 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:31:34.088 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 30, 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 8, 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "deb8584": { 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 8, 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 79.4, 2024-05-08T05:31:34.089 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:31:34.090 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:31:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 -- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03340b2a70 msgr2=0x7f03340b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03340b2a70 0x7f03340b4f30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f03440075c0 tx=0x7f034400b040 comp rx=0 tx=0).stop 2024-05-08T05:31:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 -- 172.21.15.59:0/705971971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f034c1037b0 msgr2=0x7f034c1098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f034c1037b0 0x7f034c1098c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f034000e9a0 tx=0x7f034000ecb0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 -- 172.21.15.59:0/705971971 shutdown_connections 2024-05-08T05:31:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03340b2a70 0x7f03340b4f30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f034c107760 0x7f034c109e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f034c102e00 0x7f034c109380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 --2- 172.21.15.59:0/705971971 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f034c1037b0 0x7f034c1098c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.086+0000 7f0351919700 1 -- 172.21.15.59:0/705971971 >> 172.21.15.59:0/705971971 conn(0x7f034c078a10 msgr2=0x7f034c104dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.087+0000 7f0351919700 1 -- 172.21.15.59:0/705971971 shutdown_connections 2024-05-08T05:31:34.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:31:34.087+0000 7f0351919700 1 -- 172.21.15.59:0/705971971 wait complete. 2024-05-08T05:31:34.919 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'deb8584', seconds elapsed: 32.0 2024-05-08T05:31:35.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:34 smithi143 ceph-mon[34337]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:35.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:34 smithi096 ceph-mon[34253]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:35.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:34 smithi059 ceph-mon[30304]: pgmap v63: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.962+0000 7f622c380640 1 -- 172.21.15.59:0/2548780304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241078a0 msgr2=0x7f6224107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.962+0000 7f622c380640 1 --2- 172.21.15.59:0/2548780304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241078a0 0x7f6224107c80 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f6220009930 tx=0x7f622002a420 comp rx=0 tx=0).stop 2024-05-08T05:31:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.962+0000 7f622c380640 1 -- 172.21.15.59:0/2548780304 shutdown_connections 2024-05-08T05:31:35.968 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.962+0000 7f622c380640 1 --2- 172.21.15.59:0/2548780304 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62241022a0 0x7f622410e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.962+0000 7f622c380640 1 --2- 172.21.15.59:0/2548780304 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62241018e0 0x7f6224101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.962+0000 7f622c380640 1 --2- 172.21.15.59:0/2548780304 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241078a0 0x7f6224107c80 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.962+0000 7f622c380640 1 -- 172.21.15.59:0/2548780304 >> 172.21.15.59:0/2548780304 conn(0x7f6224078a30 msgr2=0x7f6224078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.963+0000 7f622c380640 1 -- 172.21.15.59:0/2548780304 shutdown_connections 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.963+0000 7f622c380640 1 -- 172.21.15.59:0/2548780304 wait complete. 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.963+0000 7f622c380640 1 Processor -- start 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622c380640 1 -- start start 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622c380640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241018e0 0x7f6224198030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622c380640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62241022a0 0x7f6224198570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622c380640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62241078a0 0x7f622419c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622c380640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6224111bb0 con 0x7f62241018e0 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622c380640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6224111a30 con 0x7f62241022a0 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622c380640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6224111d30 con 0x7f62241078a0 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622a11c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241018e0 0x7f6224198030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.969 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622a11c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241018e0 0x7f6224198030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33268/0 (socket says 172.21.15.59:33268) 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622a11c640 1 -- 172.21.15.59:0/637940286 learned_addr learned my addr 172.21.15.59:0/637940286 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.964+0000 7f622991b640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62241022a0 0x7f6224198570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.965+0000 7f622a11c640 1 -- 172.21.15.59:0/637940286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62241078a0 msgr2=0x7f622419c8a0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.965+0000 7f622a11c640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62241078a0 0x7f622419c8a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.965+0000 7f622a11c640 1 -- 172.21.15.59:0/637940286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62241022a0 msgr2=0x7f6224198570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.965+0000 7f622a11c640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62241022a0 0x7f6224198570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.965+0000 7f622a11c640 1 -- 172.21.15.59:0/637940286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f622419cfd0 con 0x7f62241018e0 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.965+0000 7f622a11c640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241018e0 0x7f6224198030 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6220005ab0 tx=0x7f6220003b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.965+0000 7f621b7fe640 1 -- 172.21.15.59:0/637940286 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f622003f070 con 0x7f62241018e0 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.965+0000 7f621b7fe640 1 -- 172.21.15.59:0/637940286 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6220031b50 con 0x7f62241018e0 2024-05-08T05:31:35.970 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.965+0000 7f622c380640 1 -- 172.21.15.59:0/637940286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f622419d260 con 0x7f62241018e0 2024-05-08T05:31:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.966+0000 7f621b7fe640 1 -- 172.21.15.59:0/637940286 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6220035030 con 0x7f62241018e0 2024-05-08T05:31:35.971 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.966+0000 7f622c380640 1 -- 172.21.15.59:0/637940286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62241a4b50 con 0x7f62241018e0 2024-05-08T05:31:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.969+0000 7f622f170740 1 Processor -- start 2024-05-08T05:31:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.969+0000 7f622f170740 1 -- start start 2024-05-08T05:31:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622f170740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d1f10 0x555eba3d22f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622f170740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3dcbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622f170740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3df4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622f170740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555eba3375d0 con 0x555eba3d2830 2024-05-08T05:31:35.973 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622f170740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555eba2f0890 con 0x555eba3dd0f0 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622f170740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555eba333590 con 0x555eba3d1f10 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f621b7fe640 1 -- 172.21.15.59:0/637940286 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f6220042070 con 0x7f62241018e0 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622a11c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3dcbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622991b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3df4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622991b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3df4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:44080/0 (socket says 172.21.15.59:44080) 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622991b640 1 -- 172.21.15.59:0/2626676883 learned_addr learned my addr 172.21.15.59:0/2626676883 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.970+0000 7f622a91d640 1 --2- 172.21.15.59:0/2626676883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d1f10 0x555eba3d22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.971+0000 7f622991b640 1 -- 172.21.15.59:0/2626676883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d1f10 msgr2=0x555eba3d22f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.971+0000 7f622991b640 1 --2- 172.21.15.59:0/2626676883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d1f10 0x555eba3d22f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.971+0000 7f622991b640 1 -- 172.21.15.59:0/2626676883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 msgr2=0x555eba3dcbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.971+0000 7f622991b640 1 --2- 172.21.15.59:0/2626676883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3dcbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.974 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.971+0000 7f622991b640 1 -- 172.21.15.59:0/2626676883 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eba049210 con 0x555eba3dd0f0 2024-05-08T05:31:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.971+0000 7f622a11c640 1 --2- 172.21.15.59:0/2626676883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:31:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.971+0000 7f622991b640 1 --2- 172.21.15.59:0/2626676883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3df4e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6214009c90 tx=0x7f6214011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3ce8282a67d46d9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.971+0000 7f621b7fe640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62100b2b10 0x7f62100b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.972+0000 7f6218bfe640 1 -- 172.21.15.59:0/2626676883 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621401e070 con 0x555eba3dd0f0 2024-05-08T05:31:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.972+0000 7f6218bfe640 1 -- 172.21.15.59:0/2626676883 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6214019ae0 con 0x555eba3dd0f0 2024-05-08T05:31:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.972+0000 7f6218bfe640 1 -- 172.21.15.59:0/2626676883 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621400ea40 con 0x555eba3dd0f0 2024-05-08T05:31:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.972+0000 7f622a91d640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62100b2b10 0x7f62100b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.975 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.972+0000 7f621b7fe640 1 -- 172.21.15.59:0/637940286 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f6220102d10 con 0x7f62241018e0 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.973+0000 7f622a91d640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62100b2b10 0x7f62100b4fd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f621c005fd0 tx=0x7f621c0094e0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.973+0000 7f622f170740 1 -- 172.21.15.59:0/2626676883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 msgr2=0x555eba3df4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.973+0000 7f622f170740 1 --2- 172.21.15.59:0/2626676883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3df4e0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f6214009c90 tx=0x7f6214011810 comp rx=0 tx=0).stop 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.973+0000 7f622f170740 1 -- 172.21.15.59:0/2626676883 shutdown_connections 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.973+0000 7f622f170740 1 --2- 172.21.15.59:0/2626676883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3df4e0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.974+0000 7f622f170740 1 --2- 172.21.15.59:0/2626676883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3dcbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.974+0000 7f622f170740 1 --2- 172.21.15.59:0/2626676883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d1f10 0x555eba3d22f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.974+0000 7f622f170740 1 -- 172.21.15.59:0/2626676883 >> 172.21.15.59:0/2626676883 conn(0x555eba3ce420 msgr2=0x555eba3ce800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.974+0000 7f622f170740 1 -- 172.21.15.59:0/2626676883 shutdown_connections 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.974+0000 7f622f170740 1 -- 172.21.15.59:0/2626676883 wait complete. 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.974+0000 7f622f170740 1 -- start start 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.975+0000 7f622f170740 1 Processor -- start 2024-05-08T05:31:35.977 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.975+0000 7f622f170740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.978 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.976+0000 7f622a11c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.976+0000 7f622f170740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d2830 0x555eba3da530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.976+0000 7f622f170740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3daa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.976+0000 7f622f170740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555eba3dfd20 con 0x555eba3d1f10 2024-05-08T05:31:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.976+0000 7f622a11c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33284/0 (socket says 172.21.15.59:33284) 2024-05-08T05:31:35.979 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.976+0000 7f622a11c640 1 -- 172.21.15.59:0/2988815006 learned_addr learned my addr 172.21.15.59:0/2988815006 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.976+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555eba2f4370 con 0x555eba3dd0f0 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.976+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555eba3dfea0 con 0x555eba3d2830 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.976+0000 7f622f170740 10 client.0 _ll_register_callbacks cb 0x555eba048a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f622a91d640 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3daa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f622991b640 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d2830 0x555eba3da530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f622a91d640 1 -- 172.21.15.59:0/2988815006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d2830 msgr2=0x555eba3da530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f622a91d640 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d2830 0x555eba3da530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f622a91d640 1 -- 172.21.15.59:0/2988815006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 msgr2=0x555eba3d9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f622a91d640 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f622a91d640 1 -- 172.21.15.59:0/2988815006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eba115620 con 0x555eba3dd0f0 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f622a91d640 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3daa70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f621c0126d0 tx=0x7f621c004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46f3935626cee83c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f620dffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.977+0000 7f620dffb640 1 -- 172.21.15.59:0/2988815006 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621c01f8f0 con 0x555eba3dd0f0 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.978+0000 7f620dffb640 1 -- 172.21.15.59:0/2988815006 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f621c01fa90 con 0x555eba3dd0f0 2024-05-08T05:31:35.980 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.978+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6214019ae0 con 0x555eba3dd0f0 2024-05-08T05:31:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.978+0000 7f620dffb640 1 -- 172.21.15.59:0/2988815006 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f621c030050 con 0x555eba3dd0f0 2024-05-08T05:31:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.978+0000 7f622f170740 10 client.25397 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.978+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555eba3ebbf0 con 0x555eba3dd0f0 2024-05-08T05:31:35.981 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.978+0000 7f620dffb640 1 -- 172.21.15.59:0/2988815006 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6214019ae0 con 0x555eba3dd0f0 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.979+0000 7f620dffb640 1 -- 172.21.15.59:0/2988815006 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f621c00f040 con 0x555eba3dd0f0 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.979+0000 7f622f170740 10 client.25397 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.979+0000 7f622f170740 10 client.25397 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.979+0000 7f622f170740 20 client.25397 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.979+0000 7f622f170740 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555eba3ed0b0 0x555eba3ef4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.979+0000 7f622f170740 4 client.25397 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.979+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555eba3451e0 con 0x555eba3ed0b0 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.980+0000 7f622a11c640 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555eba3ed0b0 0x555eba3ef4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.980+0000 7f620dffb640 10 client.25397 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.980+0000 7f622a11c640 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555eba3ed0b0 0x555eba3ef4a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f2de2ee7e661343 server_cookie=b9e095ea27a52124 in_seq=0 out_seq=0 2024-05-08T05:31:35.983 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f620dffb640 1 -- 172.21.15.59:0/2988815006 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555eba3451e0 con 0x555eba3ed0b0 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f620dffb640 10 client.25397 handle_command_reply: tid=0 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 client.25397 shutdown 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 msgr2=0x555eba3daa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3daa70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f621c0126d0 tx=0x7f621c004ab0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 shutdown_connections 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555eba3ed0b0 0x555eba3ef4a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3dd0f0 0x555eba3daa70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3d2830 0x555eba3da530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 --2- 172.21.15.59:0/2988815006 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 >> 172.21.15.59:0/2988815006 conn(0x555eba3ce420 msgr2=0x555eba3d41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f620dffb640 0 client.25397 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 shutdown_connections 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 1 -- 172.21.15.59:0/2988815006 wait complete. 2024-05-08T05:31:35.984 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:35.981+0000 7f622f170740 20 client.25397 trim_cache size 0 max 16384 2024-05-08T05:31:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.012+0000 7f622f170740 1 Processor -- start 2024-05-08T05:31:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.012+0000 7f622f170740 1 -- start start 2024-05-08T05:31:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.012+0000 7f622f170740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d4920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.012+0000 7f622f170740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d2830 0x555eba3cd400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.012+0000 7f622f170740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3dd0f0 0x555eba3f4db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.012+0000 7f622f170740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62241b1ce0 con 0x555eba3d1f10 2024-05-08T05:31:36.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.012+0000 7f622f170740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62241b3820 con 0x555eba3d2830 2024-05-08T05:31:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.012+0000 7f622f170740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f62241b23b0 con 0x555eba3dd0f0 2024-05-08T05:31:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.012+0000 7f622a11c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622991b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d2830 0x555eba3cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622a11c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d4920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33292/0 (socket says 172.21.15.59:33292) 2024-05-08T05:31:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622a11c640 1 -- 172.21.15.59:0/1170674067 learned_addr learned my addr 172.21.15.59:0/1170674067 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622991b640 1 -- 172.21.15.59:0/1170674067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3dd0f0 msgr2=0x555eba3f4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622991b640 1 --2- 172.21.15.59:0/1170674067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3dd0f0 0x555eba3f4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622991b640 1 -- 172.21.15.59:0/1170674067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 msgr2=0x555eba3d4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622991b640 1 --2- 172.21.15.59:0/1170674067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622991b640 1 -- 172.21.15.59:0/1170674067 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555eba3ce250 con 0x555eba3d2830 2024-05-08T05:31:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622a11c640 1 --2- 172.21.15.59:0/1170674067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:31:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f622991b640 1 --2- 172.21.15.59:0/1170674067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d2830 0x555eba3cd400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6214005890 tx=0x7f621400e5a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=47d2b6d41678a58d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.013+0000 7f620ffff640 1 -- 172.21.15.59:0/1170674067 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621401e040 con 0x555eba3d2830 2024-05-08T05:31:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.014+0000 7f620ffff640 1 -- 172.21.15.59:0/1170674067 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f621400e3b0 con 0x555eba3d2830 2024-05-08T05:31:36.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.014+0000 7f620ffff640 1 -- 172.21.15.59:0/1170674067 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62140035f0 con 0x555eba3d2830 2024-05-08T05:31:36.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.014+0000 7f622f170740 1 -- 172.21.15.59:0/1170674067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d2830 msgr2=0x555eba3cd400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.014+0000 7f622f170740 1 --2- 172.21.15.59:0/1170674067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d2830 0x555eba3cd400 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6214005890 tx=0x7f621400e5a0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 -- 172.21.15.59:0/1170674067 shutdown_connections 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 --2- 172.21.15.59:0/1170674067 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3dd0f0 0x555eba3f4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 --2- 172.21.15.59:0/1170674067 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d2830 0x555eba3cd400 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 --2- 172.21.15.59:0/1170674067 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d1f10 0x555eba3d4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 -- 172.21.15.59:0/1170674067 >> 172.21.15.59:0/1170674067 conn(0x555eba3ce420 msgr2=0x555eba3cccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 -- 172.21.15.59:0/1170674067 shutdown_connections 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 -- 172.21.15.59:0/1170674067 wait complete. 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 -- start start 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 Processor -- start 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d1f10 0x555eba3e5450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.015+0000 7f622f170740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3e07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:36.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622f170740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3dd0f0 0x555eba3e0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622991b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3e07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622991b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3e07f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33304/0 (socket says 172.21.15.59:33304) 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622991b640 1 -- 172.21.15.59:0/101517966 learned_addr learned my addr 172.21.15.59:0/101517966 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f62241b3150 con 0x555eba3d2830 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f62241b2a80 con 0x555eba3d1f10 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f62241afd70 con 0x555eba3dd0f0 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622a11c640 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d1f10 0x555eba3e5450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622f170740 10 client.0 _ll_register_callbacks cb 0x7f621c011f70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622991b640 1 -- 172.21.15.59:0/101517966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3dd0f0 msgr2=0x555eba3e0d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622991b640 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3dd0f0 0x555eba3e0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622991b640 1 -- 172.21.15.59:0/101517966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d1f10 msgr2=0x555eba3e5450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622991b640 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d1f10 0x555eba3e5450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622991b640 1 -- 172.21.15.59:0/101517966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555eba3cde40 con 0x555eba3d2830 2024-05-08T05:31:36.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622a11c640 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d1f10 0x555eba3e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622991b640 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3e07f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6214009be0 tx=0x7f62140099b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d0f03cff54bdfc4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f620dffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f620dffb640 1 -- 172.21.15.59:0/101517966 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f621401e040 con 0x555eba3d2830 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f620dffb640 1 -- 172.21.15.59:0/101517966 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f6214005090 con 0x555eba3d2830 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.016+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f621400e3b0 con 0x555eba3d2830 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f620dffb640 1 -- 172.21.15.59:0/101517966 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f621403d050 con 0x555eba3d2830 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f622f170740 10 client.15771 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x555eba4575b0 con 0x555eba3d2830 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f620dffb640 1 -- 172.21.15.59:0/101517966 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6214030030 con 0x555eba3d2830 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f620dffb640 1 -- 172.21.15.59:0/101517966 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f62140374e0 con 0x555eba3d2830 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f622f170740 10 client.15771 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f622f170740 10 client.15771 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f622f170740 20 client.15771 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:36.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f622f170740 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555eba4579e0 0x555eba459e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f622f170740 4 client.15771 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:31:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.017+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f621400e3b0 con 0x555eba4579e0 2024-05-08T05:31:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.018+0000 7f622a11c640 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555eba4579e0 0x555eba459e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.018+0000 7f622a11c640 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555eba4579e0 0x555eba459e00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dba1f0e7db19ccd7 server_cookie=9ee7a2b4f73a15dc in_seq=0 out_seq=0 2024-05-08T05:31:36.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.018+0000 7f620dffb640 10 client.15771 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.022+0000 7f620dffb640 1 -- 172.21.15.59:0/101517966 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f621400e3b0 con 0x555eba4579e0 2024-05-08T05:31:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.022+0000 7f620dffb640 10 client.15771 handle_command_reply: tid=0 2024-05-08T05:31:36.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.022+0000 7f622f170740 1 client.15771 shutdown 2024-05-08T05:31:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.022+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 msgr2=0x555eba3e07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.022+0000 7f622f170740 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3e07f0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f6214009be0 tx=0x7f62140099b0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.022+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 shutdown_connections 2024-05-08T05:31:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.022+0000 7f622f170740 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555eba4579e0 0x555eba459e00 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.023+0000 7f622f170740 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555eba3dd0f0 0x555eba3e0d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.023+0000 7f620dffb640 0 client.15771 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.023+0000 7f622f170740 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555eba3d2830 0x555eba3e07f0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.023+0000 7f622f170740 1 --2- 172.21.15.59:0/101517966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555eba3d1f10 0x555eba3e5450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.023+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 >> 172.21.15.59:0/101517966 conn(0x555eba3ce420 msgr2=0x555eba3cd580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.023+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 shutdown_connections 2024-05-08T05:31:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.023+0000 7f622f170740 1 -- 172.21.15.59:0/101517966 wait complete. 2024-05-08T05:31:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.023+0000 7f622f170740 20 client.15771 trim_cache size 0 max 16384 2024-05-08T05:31:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:31:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:31:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:31:36.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:31:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 -- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62100b2b10 msgr2=0x7f62100b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62100b2b10 0x7f62100b4fd0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f621c005fd0 tx=0x7f621c0094e0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 -- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241018e0 msgr2=0x7f6224198030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241018e0 0x7f6224198030 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f6220005ab0 tx=0x7f6220003b90 comp rx=0 tx=0).stop 2024-05-08T05:31:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 -- 172.21.15.59:0/637940286 shutdown_connections 2024-05-08T05:31:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f62100b2b10 0x7f62100b4fd0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f62241078a0 0x7f622419c8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f62241022a0 0x7f6224198570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 --2- 172.21.15.59:0/637940286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f62241018e0 0x7f6224198030 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 -- 172.21.15.59:0/637940286 >> 172.21.15.59:0/637940286 conn(0x7f6224078a30 msgr2=0x7f6224104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 -- 172.21.15.59:0/637940286 shutdown_connections 2024-05-08T05:31:36.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:36.024+0000 7f622c380640 1 -- 172.21.15.59:0/637940286 wait complete. 2024-05-08T05:31:37.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:36 smithi143 ceph-mon[34337]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:37.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:36 smithi096 ceph-mon[34253]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:37.312 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:36 smithi059 ceph-mon[30304]: pgmap v64: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:39.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:38 smithi143 ceph-mon[34337]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:31:39.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:38 smithi096 ceph-mon[34253]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:31:39.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:38 smithi059 ceph-mon[30304]: pgmap v65: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:31:40.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:39 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:31:40.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:39 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:31:40.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:39 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:31:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:40 smithi143 ceph-mon[34337]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:40 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:31:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:40 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:31:41.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:40 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:31:41.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:40 smithi096 ceph-mon[34253]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:41.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:40 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:31:41.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:40 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:31:41.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:40 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:31:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:40 smithi059 ceph-mon[30304]: pgmap v66: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:40 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:31:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:40 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:31:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:40 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:31:43.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:42 smithi143 ceph-mon[34337]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:43.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:42 smithi096 ceph-mon[34253]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:42 smithi059 ceph-mon[30304]: pgmap v67: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:45.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:44 smithi143 ceph-mon[34337]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:45.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:44 smithi096 ceph-mon[34253]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:44 smithi059 ceph-mon[30304]: pgmap v68: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:47.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:46 smithi143 ceph-mon[34337]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:47.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:46 smithi096 ceph-mon[34253]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:46 smithi059 ceph-mon[30304]: pgmap v69: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 255 B/s wr, 12 op/s 2024-05-08T05:31:47.495 INFO:tasks.workunit.client.0.smithi059.stderr:Updating files: 93% (11489/12271) Updating files: 94% (11535/12271) Updating files: 95% (11658/12271) Updating files: 96% (11781/12271) Updating files: 97% (11903/12271) Updating files: 98% (12026/12271) Updating files: 99% (12149/12271) Updating files: 100% (12271/12271) Updating files: 100% (12271/12271), done. 2024-05-08T05:31:48.223 INFO:tasks.workunit.client.0.smithi059.stderr:Note: switching to 'bee299f4ac344e4c45e550d7a80abcadd7bc3d5a'. 2024-05-08T05:31:48.223 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-05-08T05:31:48.223 INFO:tasks.workunit.client.0.smithi059.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr:changes and commit them, and you can discard any commits you make in this 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr:state without impacting any branches by switching back to a branch. 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr:If you want to create a new branch to retain commits you create, you may 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr:do so (now or later) by using -c with the switch command. Example: 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr: git switch -c 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr:Or undo this operation with: 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr: git switch - 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr: 2024-05-08T05:31:48.224 INFO:tasks.workunit.client.0.smithi059.stderr:HEAD is now at bee299f4ac3 mds: relax divergent backtrace scrub failures for replicated ancestor inodes 2024-05-08T05:31:48.230 DEBUG:teuthology.orchestra.run.smithi059:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-05-08T05:31:48.257 INFO:tasks.workunit.client.0.smithi059.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-05-08T05:31:48.259 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T05:31:48.259 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-05-08T05:31:49.000 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-05-08T05:31:49.060 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-05-08T05:31:49.103 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-05-08T05:31:49.105 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T05:31:49.105 INFO:tasks.workunit.client.0.smithi059.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-05-08T05:31:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.125+0000 7f1fa48d1640 1 -- 172.21.15.59:0/791079714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0104560 msgr2=0x7f1fa010ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.125+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/791079714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0104560 0x7f1fa010ca40 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f1f94009900 tx=0x7f1f9402a510 comp rx=0 tx=0).stop 2024-05-08T05:31:49.126 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.125+0000 7f1fa48d1640 1 -- 172.21.15.59:0/791079714 shutdown_connections 2024-05-08T05:31:49.130 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.125+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/791079714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa010cf80 0x7f1fa010f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.125+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/791079714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa0104560 0x7f1fa010ca40 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.125+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/791079714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fa0103c40 0x7f1fa0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.125+0000 7f1fa48d1640 1 -- 172.21.15.59:0/791079714 >> 172.21.15.59:0/791079714 conn(0x7f1fa0078a30 msgr2=0x7f1fa0078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.125+0000 7f1fa48d1640 1 -- 172.21.15.59:0/791079714 shutdown_connections 2024-05-08T05:31:49.131 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.125+0000 7f1fa48d1640 1 -- 172.21.15.59:0/791079714 wait complete. 2024-05-08T05:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.126+0000 7f1fa48d1640 1 Processor -- start 2024-05-08T05:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.126+0000 7f1fa48d1640 1 -- start start 2024-05-08T05:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.126+0000 7f1fa48d1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fa0103c40 0x7f1fa0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.140 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.126+0000 7f1fa48d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa0104560 0x7f1fa01996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.126+0000 7f1fa48d1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa010cf80 0x7f1fa019da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.126+0000 7f1fa48d1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa0112c30 con 0x7f1fa010cf80 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.126+0000 7f1fa48d1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa0112ab0 con 0x7f1fa0104560 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.126+0000 7f1fa48d1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa0112db0 con 0x7f1fa0103c40 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa010cf80 0x7f1fa019da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa010cf80 0x7f1fa019da50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38316/0 (socket says 172.21.15.59:38316) 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9ed9d640 1 -- 172.21.15.59:0/3533239402 learned_addr learned my addr 172.21.15.59:0/3533239402 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa0104560 0x7f1fa01996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fa0103c40 0x7f1fa0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9ed9d640 1 -- 172.21.15.59:0/3533239402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fa0103c40 msgr2=0x7f1fa0199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fa0103c40 0x7f1fa0199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9ed9d640 1 -- 172.21.15.59:0/3533239402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa0104560 msgr2=0x7f1fa01996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:49.141 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa0104560 0x7f1fa01996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9ed9d640 1 -- 172.21.15.59:0/3533239402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fa019e180 con 0x7f1fa010cf80 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa010cf80 0x7f1fa019da50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009fd0 tx=0x7f1f9800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f8f7fe640 1 -- 172.21.15.59:0/3533239402 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98017070 con 0x7f1fa010cf80 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f8f7fe640 1 -- 172.21.15.59:0/3533239402 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f98003e20 con 0x7f1fa010cf80 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1f8f7fe640 1 -- 172.21.15.59:0/3533239402 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98010460 con 0x7f1fa010cf80 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.127+0000 7f1fa48d1640 1 -- 172.21.15.59:0/3533239402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fa019e470 con 0x7f1fa010cf80 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.128+0000 7f1fa48d1640 1 -- 172.21.15.59:0/3533239402 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fa01a5d00 con 0x7f1fa010cf80 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.130+0000 7f1f8f7fe640 1 -- 172.21.15.59:0/3533239402 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1f98003970 con 0x7f1fa010cf80 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.131+0000 7f1f8f7fe640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1f880b2a10 0x7f1f880b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.131+0000 7f1f8f7fe640 1 -- 172.21.15.59:0/3533239402 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f1f980e25e0 con 0x7f1fa010cf80 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.131+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1f880b2a10 0x7f1f880b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.131+0000 7f1fa75ea740 1 Processor -- start 2024-05-08T05:31:49.142 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.131+0000 7f1fa75ea740 1 -- start start 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.131+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1f880b2a10 0x7f1f880b4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009bd0 tx=0x7f1f90005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4f3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d4fdfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 0x55a54d5008f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a54d4545d0 con 0x55a54d4f3320 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a54d40d890 con 0x55a54d4fe500 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a54d450590 con 0x55a54d4f3c40 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4f3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4f3700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38328/0 (socket says 172.21.15.59:38328) 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9e59c640 1 -- 172.21.15.59:0/2183906544 learned_addr learned my addr 172.21.15.59:0/2183906544 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 0x55a54d5008f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d4fdfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9e59c640 1 -- 172.21.15.59:0/2183906544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 msgr2=0x55a54d4fdfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:49.143 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d4fdfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9e59c640 1 -- 172.21.15.59:0/2183906544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 msgr2=0x55a54d5008f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 0x55a54d5008f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9e59c640 1 -- 172.21.15.59:0/2183906544 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54d166210 con 0x55a54d4f3320 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d4fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4f3700 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1f900058c0 tx=0x7f1f90014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4a3da57101b6f39 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f8cff9640 1 -- 172.21.15.59:0/2183906544 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90005020 con 0x55a54d4f3320 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f8cff9640 1 -- 172.21.15.59:0/2183906544 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f900047a0 con 0x55a54d4f3320 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.132+0000 7f1f8cff9640 1 -- 172.21.15.59:0/2183906544 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f9001e660 con 0x55a54d4f3320 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.133+0000 7f1fa75ea740 1 -- 172.21.15.59:0/2183906544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 msgr2=0x55a54d4f3700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.133+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4f3700 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f1f900058c0 tx=0x7f1f90014000 comp rx=0 tx=0).stop 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.133+0000 7f1fa75ea740 1 -- 172.21.15.59:0/2183906544 shutdown_connections 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.133+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 0x55a54d5008f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.144 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.133+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d4fdfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.133+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/2183906544 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4f3700 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.133+0000 7f1fa75ea740 1 -- 172.21.15.59:0/2183906544 >> 172.21.15.59:0/2183906544 conn(0x55a54d4ef830 msgr2=0x55a54d4efc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.133+0000 7f1fa75ea740 1 -- 172.21.15.59:0/2183906544 shutdown_connections 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.133+0000 7f1fa75ea740 1 -- 172.21.15.59:0/2183906544 wait complete. 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1fa75ea740 1 -- start start 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1fa75ea740 1 Processor -- start 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4fb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3c40 0x55a54d4fb970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4fe500 0x55a54d4fbeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a54d501130 con 0x55a54d4f3320 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a54d411370 con 0x55a54d4f3c40 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a54d5012b0 con 0x55a54d4fe500 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1fa75ea740 10 client.0 _ll_register_callbacks cb 0x55a54d165a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:49.145 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1f9e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4fb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1f9e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4fb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38332/0 (socket says 172.21.15.59:38332) 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1f9dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3c40 0x55a54d4fb970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1f9e59c640 1 -- 172.21.15.59:0/1049865412 learned_addr learned my addr 172.21.15.59:0/1049865412 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.134+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4fe500 0x55a54d4fbeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f9e59c640 1 -- 172.21.15.59:0/1049865412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4fe500 msgr2=0x55a54d4fbeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4fe500 0x55a54d4fbeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f9e59c640 1 -- 172.21.15.59:0/1049865412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3c40 msgr2=0x55a54d4fb970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3c40 0x55a54d4fb970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f9e59c640 1 -- 172.21.15.59:0/1049865412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54d232620 con 0x55a54d4f3320 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3c40 0x55a54d4fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:31:49.146 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4fb430 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1f90003e50 tx=0x7f1f90026c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b123f3b89a206b5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1049865412 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90036060 con 0x55a54d4f3320 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1049865412 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f1f9001e620 con 0x55a54d4f3320 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1049865412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f900047a0 con 0x55a54d4f3320 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1049865412 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f9003d050 con 0x55a54d4f3320 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1fa75ea740 10 client.15783 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1049865412 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a54d50cf10 con 0x55a54d4f3320 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.135+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1049865412 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f900047a0 con 0x55a54d4f3320 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1049865412 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1f900316b0 con 0x55a54d4f3320 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1fa75ea740 10 client.15783 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1fa75ea740 10 client.15783 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1fa75ea740 20 client.15783 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a54d50e260 0x55a54d510650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.147 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1fa75ea740 4 client.15783 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1049865412 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a54d4621e0 con 0x55a54d50e260 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a54d50e260 0x55a54d510650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1f85ffb640 10 client.15783 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.136+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a54d50e260 0x55a54d510650 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1eaa2e6672f03e3c server_cookie=42ebcc1e019da5ea in_seq=0 out_seq=0 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1049865412 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a54d4621e0 con 0x55a54d50e260 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1f85ffb640 10 client.15783 handle_command_reply: tid=0 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1fa75ea740 1 client.15783 shutdown 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1049865412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 msgr2=0x55a54d4fb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4fb430 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1f90003e50 tx=0x7f1f90026c30 comp rx=0 tx=0).stop 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1049865412 shutdown_connections 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a54d50e260 0x55a54d510650 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1f85ffb640 0 client.15783 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4fe500 0x55a54d4fbeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.148 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3c40 0x55a54d4fb970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1049865412 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d4fb430 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.137+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1049865412 >> 172.21.15.59:0/1049865412 conn(0x55a54d4ef830 msgr2=0x55a54d4ea580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.138+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1049865412 shutdown_connections 2024-05-08T05:31:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.138+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1049865412 wait complete. 2024-05-08T05:31:49.149 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.138+0000 7f1fa75ea740 20 client.15783 trim_cache size 0 max 16384 2024-05-08T05:31:49.150 INFO:tasks.workunit.client.0.smithi059.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-05-08T05:31:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.153+0000 7f1fa75ea740 1 Processor -- start 2024-05-08T05:31:49.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.153+0000 7f1fa75ea740 1 -- start start 2024-05-08T05:31:49.154 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:31:49.154 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.153+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3320 0x55a54d515d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.153+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d516240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.153+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4fe500 0x55a54d4f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.153+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01adb50 con 0x55a54d4fe500 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.153+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01af690 con 0x55a54d4f3320 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.153+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01ae220 con 0x55a54d4f3c40 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.153+0000 7f1f9ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4fe500 0x55a54d4f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4fe500 0x55a54d4f4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38340/0 (socket says 172.21.15.59:38340) 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9ed9d640 1 -- 172.21.15.59:0/1023186178 learned_addr learned my addr 172.21.15.59:0/1023186178 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:49.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d516240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3320 0x55a54d515d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9ed9d640 1 -- 172.21.15.59:0/1023186178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 msgr2=0x55a54d516240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d516240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9ed9d640 1 -- 172.21.15.59:0/1023186178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3320 msgr2=0x55a54d515d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3320 0x55a54d515d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9ed9d640 1 -- 172.21.15.59:0/1023186178 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a54d4f62a0 con 0x55a54d4fe500 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d516240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3320 0x55a54d515d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4fe500 0x55a54d4f4cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800b760 tx=0x7f1f980099f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf53b6f6c4d6ad8e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f87fff640 1 -- 172.21.15.59:0/1023186178 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f98020040 con 0x55a54d4fe500 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f87fff640 1 -- 172.21.15.59:0/1023186178 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f980e2040 con 0x55a54d4fe500 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.154+0000 7f1f87fff640 1 -- 172.21.15.59:0/1023186178 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f980dfdb0 con 0x55a54d4fe500 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1023186178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4fe500 msgr2=0x55a54d4f4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4fe500 0x55a54d4f4cd0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1f9800b760 tx=0x7f1f980099f0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1023186178 shutdown_connections 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4fe500 0x55a54d4f4cd0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d516240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1023186178 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4f3320 0x55a54d515d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1023186178 >> 172.21.15.59:0/1023186178 conn(0x55a54d4ef830 msgr2=0x55a54d4fa970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1023186178 shutdown_connections 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1023186178 wait complete. 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 -- start start 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.155+0000 7f1fa75ea740 1 Processor -- start 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d5065c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d506b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1fa75ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 0x55a54d501aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01aefc0 con 0x55a54d4f3320 2024-05-08T05:31:49.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01ae8f0 con 0x55a54d4fe500 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1fa75ea740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1fa01abb10 con 0x55a54d4f3c40 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1fa75ea740 10 client.0 _ll_register_callbacks cb 0x7f1f90004230 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d5065c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d5065c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38352/0 (socket says 172.21.15.59:38352) 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9e59c640 1 -- 172.21.15.59:0/1965308337 learned_addr learned my addr 172.21.15.59:0/1965308337 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 0x55a54d501aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d506b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9e59c640 1 -- 172.21.15.59:0/1965308337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 msgr2=0x55a54d506b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d506b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9e59c640 1 -- 172.21.15.59:0/1965308337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 msgr2=0x55a54d501aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 0x55a54d501aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9e59c640 1 -- 172.21.15.59:0/1965308337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a54d5022d0 con 0x55a54d4f3320 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9ed9d640 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 0x55a54d501aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:31:49.163 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f9e59c640 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d5065c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1f900058c0 tx=0x7f1f90005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a083eb81e7a809c5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f85ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1965308337 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f90036070 con 0x55a54d4f3320 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1965308337 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f1f90003e80 con 0x55a54d4f3320 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.156+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1965308337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f980e2040 con 0x55a54d4f3320 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1965308337 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=30) ==== 24+0+0 (secure 0 0 0) 0x7f1f9004a050 con 0x55a54d4f3320 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1fa75ea740 10 client.15789 fetch_fsmap learned FSMap version 30 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1965308337 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=30,monmap=4+}) -- 0x55a54d5783b0 con 0x55a54d4f3320 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1965308337 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f980e2040 con 0x55a54d4f3320 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1965308337 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 30) ==== 8362+0+0 (secure 0 0 0) 0x7f1f90004780 con 0x55a54d4f3320 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1fa75ea740 10 client.15789 fetch_fsmap finished waiting for FSMap version 30 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1fa75ea740 10 client.15789 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1fa75ea740 20 client.15789 populate_metadata read hostname 'smithi059' 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a54d578a70 0x55a54d57aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1fa75ea740 4 client.15789 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:31:49.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1965308337 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a54d45c050 con 0x55a54d578a70 2024-05-08T05:31:49.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.157+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a54d578a70 0x55a54d57aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:31:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.158+0000 7f1f85ffb640 10 client.15789 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.158+0000 7f1f9dd9b640 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a54d578a70 0x55a54d57aeb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f83b2ccc090b27ba server_cookie=447667b2614b5c46 in_seq=0 out_seq=0 2024-05-08T05:31:49.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.161+0000 7f1f85ffb640 1 -- 172.21.15.59:0/1965308337 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a54d45c050 con 0x55a54d578a70 2024-05-08T05:31:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.161+0000 7f1f85ffb640 10 client.15789 handle_command_reply: tid=0 2024-05-08T05:31:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.161+0000 7f1fa75ea740 1 client.15789 shutdown 2024-05-08T05:31:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.161+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1965308337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 msgr2=0x55a54d5065c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.161+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d5065c0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1f900058c0 tx=0x7f1f90005c50 comp rx=0 tx=0).stop 2024-05-08T05:31:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1965308337 shutdown_connections 2024-05-08T05:31:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a54d578a70 0x55a54d57aeb0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a54d4fe500 0x55a54d501aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a54d4f3c40 0x55a54d506b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1f85ffb640 0 client.15789 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:31:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa75ea740 1 --2- 172.21.15.59:0/1965308337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a54d4f3320 0x55a54d5065c0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1965308337 >> 172.21.15.59:0/1965308337 conn(0x55a54d4ef830 msgr2=0x55a54d4f5560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1965308337 shutdown_connections 2024-05-08T05:31:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa75ea740 1 -- 172.21.15.59:0/1965308337 wait complete. 2024-05-08T05:31:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa75ea740 20 client.15789 trim_cache size 0 max 16384 2024-05-08T05:31:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:31:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:31:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:31:49.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:31:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 -- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1f880b2a10 msgr2=0x7f1f880b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1f880b2a10 0x7f1f880b4ed0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f1f90009bd0 tx=0x7f1f90005f90 comp rx=0 tx=0).stop 2024-05-08T05:31:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 -- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa010cf80 msgr2=0x7f1fa019da50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:31:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa010cf80 0x7f1fa019da50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f1f98009fd0 tx=0x7f1f9800b260 comp rx=0 tx=0).stop 2024-05-08T05:31:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 -- 172.21.15.59:0/3533239402 shutdown_connections 2024-05-08T05:31:49.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1f880b2a10 0x7f1f880b4ed0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1fa010cf80 0x7f1fa019da50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1fa0104560 0x7f1fa01996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 --2- 172.21.15.59:0/3533239402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1fa0103c40 0x7f1fa0199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:31:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 -- 172.21.15.59:0/3533239402 >> 172.21.15.59:0/3533239402 conn(0x7f1fa0078a30 msgr2=0x7f1fa00fea40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:31:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.162+0000 7f1fa48d1640 1 -- 172.21.15.59:0/3533239402 shutdown_connections 2024-05-08T05:31:49.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:31:49.163+0000 7f1fa48d1640 1 -- 172.21.15.59:0/3533239402 wait complete. 2024-05-08T05:31:49.173 INFO:tasks.workunit:Running workunits matching suites/fsx.sh on client.0... 2024-05-08T05:31:49.175 INFO:tasks.workunit:Running workunit suites/fsx.sh... 2024-05-08T05:31:49.175 DEBUG:teuthology.orchestra.run.smithi059:workunit test suites/fsx.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bee299f4ac344e4c45e550d7a80abcadd7bc3d5a TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/fsx.sh 2024-05-08T05:31:49.237 INFO:tasks.workunit.client.0.smithi059.stderr:+ set -e 2024-05-08T05:31:49.237 INFO:tasks.workunit.client.0.smithi059.stderr:+ git clone https://git.ceph.com/xfstests-dev.git 2024-05-08T05:31:49.240 INFO:tasks.workunit.client.0.smithi059.stderr:Cloning into 'xfstests-dev'... 2024-05-08T05:31:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:48 smithi143 ceph-mon[34337]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:48 smithi096 ceph-mon[34253]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:49.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:48 smithi059 ceph-mon[30304]: pgmap v70: 129 pgs: 129 active+clean; 1.4 MiB data, 357 MiB used, 1.0 TiB / 1.0 TiB avail; 3.9 KiB/s rd, 255 B/s wr, 8 op/s 2024-05-08T05:31:51.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:50 smithi143 ceph-mon[34337]: pgmap v71: 129 pgs: 129 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-05-08T05:31:51.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:50 smithi096 ceph-mon[34253]: pgmap v71: 129 pgs: 129 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-05-08T05:31:51.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:50 smithi059 ceph-mon[30304]: pgmap v71: 129 pgs: 129 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 13 KiB/s wr, 13 op/s 2024-05-08T05:31:53.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:52 smithi143 ceph-mon[34337]: pgmap v72: 129 pgs: 129 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-08T05:31:53.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:52 smithi096 ceph-mon[34253]: pgmap v72: 129 pgs: 129 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-08T05:31:53.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:52 smithi059 ceph-mon[30304]: pgmap v72: 129 pgs: 129 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-08T05:31:55.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:54 smithi143 ceph-mon[34337]: pgmap v73: 129 pgs: 129 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-08T05:31:55.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:54 smithi096 ceph-mon[34253]: pgmap v73: 129 pgs: 129 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-08T05:31:55.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:54 smithi059 ceph-mon[30304]: pgmap v73: 129 pgs: 129 active+clean; 1.5 MiB data, 358 MiB used, 1.0 TiB / 1.0 TiB avail; 4.0 KiB/s rd, 13 KiB/s wr, 8 op/s 2024-05-08T05:31:57.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:56 smithi143 ceph-mon[34337]: pgmap v74: 129 pgs: 129 active+clean; 15 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-08T05:31:57.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:56 smithi096 ceph-mon[34253]: pgmap v74: 129 pgs: 129 active+clean; 15 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-08T05:31:57.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:56 smithi059 ceph-mon[30304]: pgmap v74: 129 pgs: 129 active+clean; 15 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 202 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-08T05:31:58.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:57 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:31:58.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:57 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:31:58.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:57 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:31:59.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:31:58 smithi096 ceph-mon[34253]: pgmap v75: 129 pgs: 129 active+clean; 15 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-05-08T05:31:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:31:58 smithi059 ceph-mon[30304]: pgmap v75: 129 pgs: 129 active+clean; 15 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-05-08T05:31:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:31:58 smithi143 ceph-mon[34337]: pgmap v75: 129 pgs: 129 active+clean; 15 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 200 KiB/s rd, 1.2 MiB/s wr, 13 op/s 2024-05-08T05:32:01.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:00 smithi096 ceph-mon[34253]: pgmap v76: 129 pgs: 129 active+clean; 16 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-05-08T05:32:01.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:00 smithi059 ceph-mon[30304]: pgmap v76: 129 pgs: 129 active+clean; 16 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-05-08T05:32:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:00 smithi143 ceph-mon[34337]: pgmap v76: 129 pgs: 129 active+clean; 16 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 274 KiB/s rd, 1.3 MiB/s wr, 22 op/s 2024-05-08T05:32:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.255+0000 7f76f3869640 1 -- 172.21.15.59:0/960688199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec1078a0 msgr2=0x7f76ec107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.255+0000 7f76f3869640 1 --2- 172.21.15.59:0/960688199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec1078a0 0x7f76ec107c80 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009860 tx=0x7f76e802a340 comp rx=0 tx=0).stop 2024-05-08T05:32:02.258 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.255+0000 7f76f3869640 1 -- 172.21.15.59:0/960688199 shutdown_connections 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.255+0000 7f76f3869640 1 --2- 172.21.15.59:0/960688199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1022a0 0x7f76ec10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.255+0000 7f76f3869640 1 --2- 172.21.15.59:0/960688199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76ec1018e0 0x7f76ec101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.255+0000 7f76f3869640 1 --2- 172.21.15.59:0/960688199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec1078a0 0x7f76ec107c80 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.255+0000 7f76f3869640 1 -- 172.21.15.59:0/960688199 >> 172.21.15.59:0/960688199 conn(0x7f76ec078a30 msgr2=0x7f76ec078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.255+0000 7f76f3869640 1 -- 172.21.15.59:0/960688199 shutdown_connections 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.255+0000 7f76f3869640 1 -- 172.21.15.59:0/960688199 wait complete. 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f3869640 1 Processor -- start 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f3869640 1 -- start start 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f3869640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1018e0 0x7f76ec0fe870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f3869640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec1022a0 0x7f76ec0fedb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f3869640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76ec1078a0 0x7f76ec0ff2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f3869640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec111b60 con 0x7f76ec1018e0 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f3869640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec1119e0 con 0x7f76ec1022a0 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f3869640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec111ce0 con 0x7f76ec1078a0 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f1605640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1018e0 0x7f76ec0fe870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f1605640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1018e0 0x7f76ec0fe870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47878/0 (socket says 172.21.15.59:47878) 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f1605640 1 -- 172.21.15.59:0/3962373508 learned_addr learned my addr 172.21.15.59:0/3962373508 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.256+0000 7f76f0e04640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec1022a0 0x7f76ec0fedb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f1e06640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76ec1078a0 0x7f76ec0ff2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f1605640 1 -- 172.21.15.59:0/3962373508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76ec1078a0 msgr2=0x7f76ec0ff2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f1605640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76ec1078a0 0x7f76ec0ff2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f1605640 1 -- 172.21.15.59:0/3962373508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec1022a0 msgr2=0x7f76ec0fedb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f1605640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec1022a0 0x7f76ec0fedb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f1605640 1 -- 172.21.15.59:0/3962373508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76ec1a7620 con 0x7f76ec1018e0 2024-05-08T05:32:02.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f1e06640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76ec1078a0 0x7f76ec0ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:32:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f1605640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1018e0 0x7f76ec0fe870 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009980 tx=0x7f76e8032e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76e27fc640 1 -- 172.21.15.59:0/3962373508 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8040020 con 0x7f76ec1018e0 2024-05-08T05:32:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f3869640 1 -- 172.21.15.59:0/3962373508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76ec1a78b0 con 0x7f76ec1018e0 2024-05-08T05:32:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76e27fc640 1 -- 172.21.15.59:0/3962373508 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e8030e40 con 0x7f76ec1018e0 2024-05-08T05:32:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76e27fc640 1 -- 172.21.15.59:0/3962373508 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e8037040 con 0x7f76ec1018e0 2024-05-08T05:32:02.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.257+0000 7f76f3869640 1 -- 172.21.15.59:0/3962373508 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76ec1a7da0 con 0x7f76ec1018e0 2024-05-08T05:32:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76e27fc640 1 -- 172.21.15.59:0/3962373508 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f76e8043020 con 0x7f76ec1018e0 2024-05-08T05:32:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f6578740 1 Processor -- start 2024-05-08T05:32:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f6578740 1 -- start start 2024-05-08T05:32:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f6578740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f6b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f6578740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 0x5569c4f75bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f6578740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f784e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f6578740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5569c4ed05d0 con 0x5569c4f6af10 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f6578740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569c4e89890 con 0x5569c4f6b830 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f6578740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5569c4ecc590 con 0x5569c4f760f0 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f1e06640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f6b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f1e06640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f6b2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47882/0 (socket says 172.21.15.59:47882) 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f1e06640 1 -- 172.21.15.59:0/2421779152 learned_addr learned my addr 172.21.15.59:0/2421779152 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f1605640 1 --2- 172.21.15.59:0/2421779152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 0x5569c4f75bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f1e06640 1 -- 172.21.15.59:0/2421779152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 msgr2=0x5569c4f784e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f1e06640 1 --2- 172.21.15.59:0/2421779152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f784e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f1e06640 1 -- 172.21.15.59:0/2421779152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 msgr2=0x5569c4f75bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f1e06640 1 --2- 172.21.15.59:0/2421779152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 0x5569c4f75bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.260+0000 7f76f1e06640 1 -- 172.21.15.59:0/2421779152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c4be2210 con 0x5569c4f6af10 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.261+0000 7f76e27fc640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f76d80b2a10 0x7f76d80b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.261+0000 7f76f1e06640 1 --2- 172.21.15.59:0/2421779152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f6b2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f76e400f400 tx=0x7f76e400f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9dd3f30514a27d87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.261+0000 7f76f1605640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f76d80b2a10 0x7f76d80b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.261+0000 7f76e27fc640 1 -- 172.21.15.59:0/3962373508 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f76e81029f0 con 0x7f76ec1018e0 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.261+0000 7f76d77fe640 1 -- 172.21.15.59:0/2421779152 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e401d070 con 0x5569c4f6af10 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.261+0000 7f76d77fe640 1 -- 172.21.15.59:0/2421779152 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e400cb50 con 0x5569c4f6af10 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.261+0000 7f76d77fe640 1 -- 172.21.15.59:0/2421779152 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4018b40 con 0x5569c4f6af10 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 -- 172.21.15.59:0/2421779152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 msgr2=0x5569c4f6b2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f1605640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f76d80b2a10 0x7f76d80b4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5569c4f6c4b0 tx=0x7f76e81064e0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 --2- 172.21.15.59:0/2421779152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f6b2f0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f76e400f400 tx=0x7f76e400f8c0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 -- 172.21.15.59:0/2421779152 shutdown_connections 2024-05-08T05:32:02.270 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 --2- 172.21.15.59:0/2421779152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f784e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 --2- 172.21.15.59:0/2421779152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 0x5569c4f75bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 --2- 172.21.15.59:0/2421779152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f6b2f0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 -- 172.21.15.59:0/2421779152 >> 172.21.15.59:0/2421779152 conn(0x5569c4f67420 msgr2=0x5569c4f67800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 -- 172.21.15.59:0/2421779152 shutdown_connections 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 -- 172.21.15.59:0/2421779152 wait complete. 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 -- start start 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.262+0000 7f76f6578740 1 Processor -- start 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f6578740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f730e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f6578740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f6b830 0x5569c4f73620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f6578740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f760f0 0x5569c4f73b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f6578740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5569c4f78d20 con 0x5569c4f760f0 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f6578740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5569c4e8d370 con 0x5569c4f6af10 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f6578740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5569c4f78ea0 con 0x5569c4f6b830 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f6578740 10 client.0 _ll_register_callbacks cb 0x5569c4be1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:02.271 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f760f0 0x5569c4f73b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f730e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f0e04640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f6b830 0x5569c4f73620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f730e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:59166/0 (socket says 172.21.15.59:59166) 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f0e04640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f6b830 0x5569c4f73620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:48656/0 (socket says 172.21.15.59:48656) 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 -- 172.21.15.59:0/3581990765 learned_addr learned my addr 172.21.15.59:0/3581990765 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 -- 172.21.15.59:0/3581990765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f6b830 msgr2=0x5569c4f73620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f6b830 0x5569c4f73620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 -- 172.21.15.59:0/3581990765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f760f0 msgr2=0x5569c4f73b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f760f0 0x5569c4f73b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.272 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 -- 172.21.15.59:0/3581990765 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c4cae620 con 0x5569c4f6af10 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f0e04640 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f6b830 0x5569c4f73620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76f1e06640 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f730e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009fd0 tx=0x7f76e4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ad884470e48fbe6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.263+0000 7f76d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3581990765 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e401d070 con 0x5569c4f6af10 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3581990765 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f76e4018af0 con 0x5569c4f6af10 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f6578740 1 -- 172.21.15.59:0/3581990765 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e400cb50 con 0x5569c4f6af10 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3581990765 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f76e4004a40 con 0x5569c4f6af10 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f6578740 10 client.25430 fetch_fsmap learned FSMap version 32 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f6578740 1 -- 172.21.15.59:0/3581990765 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5569c4f84c50 con 0x5569c4f6af10 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3581990765 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e400cb50 con 0x5569c4f6af10 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3581990765 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f76e400b040 con 0x5569c4f6af10 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f6578740 10 client.25430 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f6578740 10 client.25430 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f6578740 20 client.25430 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:02.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f6578740 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5569c4f860b0 0x5569c4f884a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f6578740 4 client.25430 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f6578740 1 -- 172.21.15.59:0/3581990765 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5569c4ede1e0 con 0x5569c4f860b0 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.264+0000 7f76f0e04640 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5569c4f860b0 0x5569c4f884a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.265+0000 7f76d4ff9640 10 client.25430 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.265+0000 7f76f0e04640 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5569c4f860b0 0x5569c4f884a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=920665227b9c44cd server_cookie=2b71e31556205494 in_seq=0 out_seq=0 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3581990765 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5569c4ede1e0 con 0x5569c4f860b0 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76d4ff9640 10 client.25430 handle_command_reply: tid=0 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 client.25430 shutdown 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 -- 172.21.15.59:0/3581990765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 msgr2=0x5569c4f730e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f730e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f76e4009fd0 tx=0x7f76e4005fd0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 -- 172.21.15.59:0/3581990765 shutdown_connections 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5569c4f860b0 0x5569c4f884a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f760f0 0x5569c4f73b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76d4ff9640 0 client.25430 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f6b830 0x5569c4f73620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 --2- 172.21.15.59:0/3581990765 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f730e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 -- 172.21.15.59:0/3581990765 >> 172.21.15.59:0/3581990765 conn(0x5569c4f67420 msgr2=0x5569c4f6d1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 -- 172.21.15.59:0/3581990765 shutdown_connections 2024-05-08T05:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 1 -- 172.21.15.59:0/3581990765 wait complete. 2024-05-08T05:32:02.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.267+0000 7f76f6578740 20 client.25430 trim_cache size 0 max 16384 2024-05-08T05:32:02.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.282+0000 7f76f6578740 1 Processor -- start 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f6578740 1 -- start start 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f6578740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f751a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f6578740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 0x5569c4f66400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f6578740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f73630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f6578740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec1b1f80 con 0x5569c4f6af10 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f6578740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec1b3b20 con 0x5569c4f6b830 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f6578740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec1b2650 con 0x5569c4f760f0 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f751a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f751a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47910/0 (socket says 172.21.15.59:47910) 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 -- 172.21.15.59:0/91517107 learned_addr learned my addr 172.21.15.59:0/91517107 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:02.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f0e04640 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 0x5569c4f66400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f73630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 -- 172.21.15.59:0/91517107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 msgr2=0x5569c4f73630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f73630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 -- 172.21.15.59:0/91517107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 msgr2=0x5569c4f66400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 0x5569c4f66400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 -- 172.21.15.59:0/91517107 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5569c4f784e0 con 0x5569c4f6af10 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f0e04640 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 0x5569c4f66400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76f1e06640 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f751a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005e10 tx=0x7f76e4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b7572f1c9c6d6d6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.283+0000 7f76d6ffd640 1 -- 172.21.15.59:0/91517107 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e401d070 con 0x5569c4f6af10 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76d6ffd640 1 -- 172.21.15.59:0/91517107 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e4018af0 con 0x5569c4f6af10 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76d6ffd640 1 -- 172.21.15.59:0/91517107 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76e4004730 con 0x5569c4f6af10 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 -- 172.21.15.59:0/91517107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 msgr2=0x5569c4f751a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f751a0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f76e4005e10 tx=0x7f76e4005d80 comp rx=0 tx=0).stop 2024-05-08T05:32:02.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 -- 172.21.15.59:0/91517107 shutdown_connections 2024-05-08T05:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f73630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6b830 0x5569c4f66400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 --2- 172.21.15.59:0/91517107 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6af10 0x5569c4f751a0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 -- 172.21.15.59:0/91517107 >> 172.21.15.59:0/91517107 conn(0x5569c4f67420 msgr2=0x5569c4f65cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 -- 172.21.15.59:0/91517107 shutdown_connections 2024-05-08T05:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 -- 172.21.15.59:0/91517107 wait complete. 2024-05-08T05:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.284+0000 7f76f6578740 1 -- start start 2024-05-08T05:32:02.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f6578740 1 Processor -- start 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f6578740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f79bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f6578740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6b830 0x5569c4f7a130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f6578740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f7a670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f6578740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec1b33f0 con 0x5569c4f6b830 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f6578740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec1b2d20 con 0x5569c4f6af10 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f6578740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f76ec1aff40 con 0x5569c4f760f0 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f6578740 10 client.0 _ll_register_callbacks cb 0x7f76e401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f0e04640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6b830 0x5569c4f7a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f1e06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f79bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f0e04640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6b830 0x5569c4f7a130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47924/0 (socket says 172.21.15.59:47924) 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f1e06640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f79bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:59184/0 (socket says 172.21.15.59:59184) 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f0e04640 1 -- 172.21.15.59:0/3409045246 learned_addr learned my addr 172.21.15.59:0/3409045246 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:02.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f1e06640 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f7a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f0e04640 1 -- 172.21.15.59:0/3409045246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 msgr2=0x5569c4f7a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f0e04640 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f7a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f0e04640 1 -- 172.21.15.59:0/3409045246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 msgr2=0x5569c4f79bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f0e04640 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f79bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f0e04640 1 -- 172.21.15.59:0/3409045246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5569c4fef680 con 0x5569c4f6b830 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.285+0000 7f76f1e06640 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f79bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f0e04640 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6b830 0x5569c4f7a130 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f76ec100300 tx=0x7f76dc00be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f1ed875e7afa7be server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76d4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3409045246 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76dc005220 con 0x5569c4f6b830 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f6578740 1 -- 172.21.15.59:0/3409045246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f76e4018af0 con 0x5569c4f6b830 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3409045246 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f76dc0053c0 con 0x5569c4f6b830 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3409045246 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=32) ==== 24+0+0 (secure 0 0 0) 0x7f76dc0055c0 con 0x5569c4f6b830 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f6578740 10 client.15807 fetch_fsmap learned FSMap version 32 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f6578740 1 -- 172.21.15.59:0/3409045246 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=32,monmap=4+}) -- 0x5569c4ff0930 con 0x5569c4f6b830 2024-05-08T05:32:02.292 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3409045246 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76e4018af0 con 0x5569c4f6b830 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3409045246 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 32) ==== 8370+0+0 (secure 0 0 0) 0x7f76dc024070 con 0x5569c4f6b830 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f6578740 10 client.15807 fetch_fsmap finished waiting for FSMap version 32 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f6578740 10 client.15807 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f6578740 20 client.15807 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f6578740 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5569c4ff10e0 0x5569c4ff3500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f6578740 4 client.15807 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.286+0000 7f76f6578740 1 -- 172.21.15.59:0/3409045246 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5569c4ed8050 con 0x5569c4ff10e0 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.287+0000 7f76f1e06640 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5569c4ff10e0 0x5569c4ff3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.287+0000 7f76f1e06640 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5569c4ff10e0 0x5569c4ff3500 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4caf1411c21cd05 server_cookie=c19e476cf65081f4 in_seq=0 out_seq=0 2024-05-08T05:32:02.293 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:02.287+0000 7f76d4ff9640 10 client.15807 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:03.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:03 smithi096 ceph-mon[34253]: pgmap v77: 129 pgs: 129 active+clean; 16 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-08T05:32:03.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:03 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76d4ff9640 1 -- 172.21.15.59:0/3409045246 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5569c4ed8050 con 0x5569c4ff10e0 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76d4ff9640 10 client.15807 handle_command_reply: tid=0 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 client.15807 shutdown 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 -- 172.21.15.59:0/3409045246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6b830 msgr2=0x5569c4f7a130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6b830 0x5569c4f7a130 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f76ec100300 tx=0x7f76dc00be90 comp rx=0 tx=0).stop 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 -- 172.21.15.59:0/3409045246 shutdown_connections 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5569c4ff10e0 0x5569c4ff3500 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5569c4f760f0 0x5569c4f7a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5569c4f6b830 0x5569c4f7a130 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:03.286 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 --2- 172.21.15.59:0/3409045246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5569c4f6af10 0x5569c4f79bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76d4ff9640 0 client.15807 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 -- 172.21.15.59:0/3409045246 >> 172.21.15.59:0/3409045246 conn(0x5569c4f67420 msgr2=0x5569c4f66580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 -- 172.21.15.59:0/3409045246 shutdown_connections 2024-05-08T05:32:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 1 -- 172.21.15.59:0/3409045246 wait complete. 2024-05-08T05:32:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.285+0000 7f76f6578740 20 client.15807 trim_cache size 0 max 16384 2024-05-08T05:32:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:32:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:32:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:32:03.287 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:32:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 -- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f76d80b2a10 msgr2=0x7f76d80b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f76d80b2a10 0x7f76d80b4ed0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x5569c4f6c4b0 tx=0x7f76e81064e0 comp rx=0 tx=0).stop 2024-05-08T05:32:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 -- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1018e0 msgr2=0x7f76ec0fe870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1018e0 0x7f76ec0fe870 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f76e8009980 tx=0x7f76e8032e90 comp rx=0 tx=0).stop 2024-05-08T05:32:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 -- 172.21.15.59:0/3962373508 shutdown_connections 2024-05-08T05:32:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f76d80b2a10 0x7f76d80b4ed0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:03.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f76ec1078a0 0x7f76ec0ff2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f76ec1022a0 0x7f76ec0fedb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 --2- 172.21.15.59:0/3962373508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f76ec1018e0 0x7f76ec0fe870 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 -- 172.21.15.59:0/3962373508 >> 172.21.15.59:0/3962373508 conn(0x7f76ec078a30 msgr2=0x7f76ec06a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 -- 172.21.15.59:0/3962373508 shutdown_connections 2024-05-08T05:32:03.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:03.286+0000 7f76f3869640 1 -- 172.21.15.59:0/3962373508 wait complete. 2024-05-08T05:32:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:03 smithi059 ceph-mon[30304]: pgmap v77: 129 pgs: 129 active+clean; 16 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-08T05:32:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:03 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:32:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:03 smithi143 ceph-mon[34337]: pgmap v77: 129 pgs: 129 active+clean; 16 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-08T05:32:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:03 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:32:05.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:05 smithi096 ceph-mon[34253]: pgmap v78: 129 pgs: 129 active+clean; 16 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-08T05:32:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:05 smithi059 ceph-mon[30304]: pgmap v78: 129 pgs: 129 active+clean; 16 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-08T05:32:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:05 smithi143 ceph-mon[34337]: pgmap v78: 129 pgs: 129 active+clean; 16 MiB data, 385 MiB used, 1.0 TiB / 1.0 TiB avail; 272 KiB/s rd, 1.2 MiB/s wr, 17 op/s 2024-05-08T05:32:07.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:07 smithi096 ceph-mon[34253]: pgmap v79: 129 pgs: 129 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.8 MiB/s wr, 574 op/s 2024-05-08T05:32:07.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:07 smithi059 ceph-mon[30304]: pgmap v79: 129 pgs: 129 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.8 MiB/s wr, 574 op/s 2024-05-08T05:32:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:07 smithi143 ceph-mon[34337]: pgmap v79: 129 pgs: 129 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 675 KiB/s rd, 1.8 MiB/s wr, 574 op/s 2024-05-08T05:32:09.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:09 smithi096 ceph-mon[34253]: pgmap v80: 129 pgs: 129 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 670 KiB/s wr, 565 op/s 2024-05-08T05:32:09.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:09 smithi059 ceph-mon[30304]: pgmap v80: 129 pgs: 129 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 670 KiB/s wr, 565 op/s 2024-05-08T05:32:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:09 smithi143 ceph-mon[34337]: pgmap v80: 129 pgs: 129 active+clean; 17 MiB data, 411 MiB used, 1.0 TiB / 1.0 TiB avail; 477 KiB/s rd, 670 KiB/s wr, 565 op/s 2024-05-08T05:32:11.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:11 smithi096 ceph-mon[34253]: pgmap v81: 129 pgs: 129 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 732 KiB/s wr, 572 op/s 2024-05-08T05:32:11.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:11 smithi059 ceph-mon[30304]: pgmap v81: 129 pgs: 129 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 732 KiB/s wr, 572 op/s 2024-05-08T05:32:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:11 smithi143 ceph-mon[34337]: pgmap v81: 129 pgs: 129 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 531 KiB/s rd, 732 KiB/s wr, 572 op/s 2024-05-08T05:32:13.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:13 smithi096 ceph-mon[34253]: pgmap v82: 129 pgs: 129 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 646 KiB/s wr, 563 op/s 2024-05-08T05:32:13.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:13 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:32:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:13 smithi059 ceph-mon[30304]: pgmap v82: 129 pgs: 129 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 646 KiB/s wr, 563 op/s 2024-05-08T05:32:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:13 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:32:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:13 smithi143 ceph-mon[34337]: pgmap v82: 129 pgs: 129 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 646 KiB/s wr, 563 op/s 2024-05-08T05:32:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:13 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:32:15.066 INFO:tasks.workunit.client.0.smithi059.stderr:Updating files: 11% (491/4287) Updating files: 12% (515/4287) Updating files: 13% (558/4287) Updating files: 14% (601/4287) Updating files: 15% (644/4287) Updating files: 16% (686/4287) Updating files: 17% (729/4287) Updating files: 18% (772/4287) Updating files: 19% (815/4287) Updating files: 20% (858/4287) Updating files: 21% (901/4287) Updating files: 21% (918/4287) Updating files: 22% (944/4287) Updating files: 23% (987/4287) Updating files: 24% (1029/4287) Updating files: 25% (1072/4287) Updating files: 26% (1115/4287) Updating files: 27% (1158/4287) Updating files: 28% (1201/4287) Updating files: 29% (1244/4287) Updating files: 30% (1287/4287) Updating files: 30% (1288/4287) Updating files: 31% (1329/4287) Updating files: 32% (1372/4287) Updating files: 32% (1381/4287) Updating files: 33% (1415/4287) Updating files: 34% (1458/4287) Updating files: 35% (1501/4287) Updating files: 36% (1544/4287) Updating files: 37% (1587/4287) Updating files: 38% (1630/4287) Updating files: 39% (1672/4287) Updating files: 40% (1715/4287) Updating files: 41% (1758/4287) Updating files: 42% (1801/4287) Updating files: 43% (1844/4287) Updating files: 44% (1887/4287) Updating files: 45% (1930/4287) Updating files: 46% (1973/4287) Updating files: 47% (2015/4287) Updating files: 48% (2058/4287) Updating files: 49% (2101/4287) Updating files: 50% (2144/4287) Updating files: 51% (2187/4287) Updating files: 52% (2230/4287) Updating files: 53% (2273/4287) Updating files: 54% (2315/4287) Updating files: 54% (2325/4287) Updating files: 55% (2358/4287) Updating files: 56% (2401/4287) Updating files: 57% (2444/4287) Updating files: 58% (2487/4287) Updating files: 59% (2530/4287) Updating files: 60% (2573/4287) Updating files: 61% (2616/4287) Updating files: 62% (2658/4287) Updating files: 63% (2701/4287) Updating files: 64% (2744/4287) Updating files: 64% (2784/4287) Updating files: 65% (2787/4287) Updating files: 66% (2830/4287) Updating files: 67% (2873/4287) Updating files: 68% (2916/4287) Updating files: 69% (2959/4287) Updating files: 70% (3001/4287) Updating files: 70% (3022/4287) Updating files: 70% (3023/4287) Updating files: 71% (3044/4287) Updating files: 72% (3087/4287) Updating files: 73% (3130/4287) Updating files: 74% (3173/4287) Updating files: 75% (3216/4287) Updating files: 76% (3259/4287) Updating files: 77% (3301/4287) Updating files: 77% (3328/4287) Updating files: 78% (3344/4287) Updating files: 79% (3387/4287) Updating files: 80% (3430/4287) Updating files: 81% (3473/4287) Updating files: 82% (3516/4287) Updating files: 83% (3559/4287) Updating files: 84% (3602/4287) Updating files: 85% (3644/4287) Updating files: 86% (3687/4287) Updating files: 87% (3730/4287) Updating files: 88% (3773/4287) Updating files: 89% (3816/4287) Updating files: 90% (3859/4287) Updating files: 91% (3902/4287) Updating files: 91% (3914/4287) Updating files: 92% (3945/4287) Updating files: 93% (3987/4287) Updating files: 94% (4030/4287) Updating files: 95% (4073/4287) Updating files: 96% (4116/4287) Updating files: 97% (4159/4287) Updating files: 98% (4202/4287) Updating files: 99% (4245/4287) Updating files: 100% (4287/4287) Updating files: 100% (4287/4287), done. 2024-05-08T05:32:15.072 INFO:tasks.workunit.client.0.smithi059.stderr:+ cd xfstests-dev 2024-05-08T05:32:15.072 INFO:tasks.workunit.client.0.smithi059.stderr:+ make -j4 2024-05-08T05:32:15.108 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize -cfi 2024-05-08T05:32:15.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:15 smithi143 ceph-mon[34337]: pgmap v83: 129 pgs: 129 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 646 KiB/s wr, 563 op/s 2024-05-08T05:32:15.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:15 smithi059 ceph-mon[30304]: pgmap v83: 129 pgs: 129 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 646 KiB/s wr, 563 op/s 2024-05-08T05:32:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:15 smithi096 ceph-mon[34253]: pgmap v83: 129 pgs: 129 active+clean; 17 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 456 KiB/s rd, 646 KiB/s wr, 563 op/s 2024-05-08T05:32:15.611 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: putting auxiliary files in '.'. 2024-05-08T05:32:15.611 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: copying file './config.guess' 2024-05-08T05:32:15.616 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: copying file './config.sub' 2024-05-08T05:32:15.621 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: copying file './install-sh' 2024-05-08T05:32:15.626 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: copying file './ltmain.sh' 2024-05-08T05:32:15.651 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: putting macros in AC_CONFIG_MACRO_DIRS, 'm4'. 2024-05-08T05:32:15.651 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: copying file 'm4/libtool.m4' 2024-05-08T05:32:15.657 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: copying file 'm4/ltoptions.m4' 2024-05-08T05:32:15.663 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: copying file 'm4/ltsugar.m4' 2024-05-08T05:32:15.669 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: copying file 'm4/ltversion.m4' 2024-05-08T05:32:15.675 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: copying file 'm4/lt~obsolete.m4' 2024-05-08T05:32:15.681 INFO:tasks.workunit.client.0.smithi059.stdout:libtoolize: Consider adding '-I m4' to ACLOCAL_AMFLAGS in Makefile.am. 2024-05-08T05:32:15.683 INFO:tasks.workunit.client.0.smithi059.stdout:cp include/install-sh . 2024-05-08T05:32:15.693 INFO:tasks.workunit.client.0.smithi059.stdout:aclocal -I m4 2024-05-08T05:32:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.379+0000 7f650ce1d640 1 -- 172.21.15.59:0/2812102508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65081022c0 msgr2=0x7f650810e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.379+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2812102508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65081022c0 0x7f650810e780 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f6500009cd0 tx=0x7f650002a8e0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.379+0000 7f650ce1d640 1 -- 172.21.15.59:0/2812102508 shutdown_connections 2024-05-08T05:32:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.379+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2812102508 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65081022c0 0x7f650810e780 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.379+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2812102508 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6508101900 0x7f6508101d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.379+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2812102508 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65081078c0 0x7f6508107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.379+0000 7f650ce1d640 1 -- 172.21.15.59:0/2812102508 >> 172.21.15.59:0/2812102508 conn(0x7f6508078a00 msgr2=0x7f6508078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.379+0000 7f650ce1d640 1 -- 172.21.15.59:0/2812102508 shutdown_connections 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.379+0000 7f650ce1d640 1 -- 172.21.15.59:0/2812102508 wait complete. 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.380+0000 7f650ce1d640 1 Processor -- start 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.380+0000 7f650ce1d640 1 -- start start 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.380+0000 7f650ce1d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508101900 0x7f65080fe860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.380+0000 7f650ce1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081022c0 0x7f65080feda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.380+0000 7f650ce1d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65081078c0 0x7f65080ff2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.380+0000 7f650ce1d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6508111b80 con 0x7f6508101900 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.380+0000 7f650ce1d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f6508111a00 con 0x7f65081022c0 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.380+0000 7f650ce1d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f6508111d00 con 0x7f65081078c0 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508101900 0x7f65080fe860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508101900 0x7f65080fe860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52750/0 (socket says 172.21.15.59:52750) 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650659c640 1 -- 172.21.15.59:0/2724557973 learned_addr learned my addr 172.21.15.59:0/2724557973 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f6505d9b640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081022c0 0x7f65080feda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f6506d9d640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65081078c0 0x7f65080ff2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650659c640 1 -- 172.21.15.59:0/2724557973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65081078c0 msgr2=0x7f65080ff2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650659c640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65081078c0 0x7f65080ff2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650659c640 1 -- 172.21.15.59:0/2724557973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081022c0 msgr2=0x7f65080feda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650659c640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081022c0 0x7f65080feda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650659c640 1 -- 172.21.15.59:0/2724557973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65081a7620 con 0x7f6508101900 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f6505d9b640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081022c0 0x7f65080feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650659c640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508101900 0x7f65080fe860 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d770 tx=0x7f64f800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:16.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f64f77fe640 1 -- 172.21.15.59:0/2724557973 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8017070 con 0x7f6508101900 2024-05-08T05:32:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650ce1d640 1 -- 172.21.15.59:0/2724557973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65081a7910 con 0x7f6508101900 2024-05-08T05:32:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f64f77fe640 1 -- 172.21.15.59:0/2724557973 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f80054d0 con 0x7f6508101900 2024-05-08T05:32:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f64f77fe640 1 -- 172.21.15.59:0/2724557973 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8004c30 con 0x7f6508101900 2024-05-08T05:32:16.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.381+0000 7f650ce1d640 1 -- 172.21.15.59:0/2724557973 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65081a7e60 con 0x7f6508101900 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.384+0000 7f650fc10740 1 Processor -- start 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.384+0000 7f650fc10740 1 -- start start 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.384+0000 7f650fc10740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c861f10 0x55a93c8622f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.384+0000 7f650fc10740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c862830 0x55a93c86cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.384+0000 7f650fc10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c86d0f0 0x55a93c86f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f650fc10740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a93c7c75d0 con 0x55a93c862830 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f650fc10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a93c780890 con 0x55a93c86d0f0 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f650fc10740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a93c7c3590 con 0x55a93c861f10 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f6505d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c86d0f0 0x55a93c86f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f6505d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c86d0f0 0x55a93c86f4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:32874/0 (socket says 172.21.15.59:32874) 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f6505d9b640 1 -- 172.21.15.59:0/2981928515 learned_addr learned my addr 172.21.15.59:0/2981928515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f6506d9d640 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c861f10 0x55a93c8622f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f64f77fe640 1 -- 172.21.15.59:0/2724557973 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f64f8004e10 con 0x7f6508101900 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f6505d9b640 1 -- 172.21.15.59:0/2981928515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c861f10 msgr2=0x55a93c8622f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f6505d9b640 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c861f10 0x55a93c8622f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f650659c640 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c862830 0x55a93c86cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f6505d9b640 1 -- 172.21.15.59:0/2981928515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c862830 msgr2=0x55a93c86cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f6505d9b640 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c862830 0x55a93c86cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f6505d9b640 1 -- 172.21.15.59:0/2981928515 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93c4d9210 con 0x55a93c86d0f0 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.385+0000 7f650659c640 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c862830 0x55a93c86cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.386+0000 7f6505d9b640 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c86d0f0 0x55a93c86f4e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00bcb0 tx=0x7f64fc00f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e83b25ede5858125 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.386+0000 7f64f4ff9640 1 -- 172.21.15.59:0/2981928515 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc011440 con 0x55a93c86d0f0 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.386+0000 7f64f4ff9640 1 -- 172.21.15.59:0/2981928515 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64fc011a90 con 0x55a93c86d0f0 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.386+0000 7f64f4ff9640 1 -- 172.21.15.59:0/2981928515 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64fc01ba40 con 0x55a93c86d0f0 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.386+0000 7f64f77fe640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f64f00b2a10 0x7f64f00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.386+0000 7f6506d9d640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f64f00b2a10 0x7f64f00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.386+0000 7f64f77fe640 1 -- 172.21.15.59:0/2724557973 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f64f80e24a0 con 0x7f6508101900 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 -- 172.21.15.59:0/2981928515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c86d0f0 msgr2=0x55a93c86f4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c86d0f0 0x55a93c86f4e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f64fc00bcb0 tx=0x7f64fc00f730 comp rx=0 tx=0).stop 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 -- 172.21.15.59:0/2981928515 shutdown_connections 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c86d0f0 0x55a93c86f4e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c862830 0x55a93c86cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 --2- 172.21.15.59:0/2981928515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c861f10 0x55a93c8622f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 -- 172.21.15.59:0/2981928515 >> 172.21.15.59:0/2981928515 conn(0x55a93c85e420 msgr2=0x55a93c85e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f6506d9d640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f64f00b2a10 0x7f64f00b4ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6500005e30 tx=0x7f6500005c30 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 -- 172.21.15.59:0/2981928515 shutdown_connections 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 -- 172.21.15.59:0/2981928515 wait complete. 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 -- start start 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 Processor -- start 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c861f10 0x55a93c86a040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.387+0000 7f650fc10740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c862830 0x55a93c86a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f650fc10740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c86d0f0 0x55a93c86aac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f650fc10740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a93c86fd20 con 0x55a93c86d0f0 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f650fc10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a93c784370 con 0x55a93c861f10 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f650fc10740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a93c86fea0 con 0x55a93c862830 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f650fc10740 10 client.0 _ll_register_callbacks cb 0x55a93c4d8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6506d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c86d0f0 0x55a93c86aac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6506d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c86d0f0 0x55a93c86aac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52776/0 (socket says 172.21.15.59:52776) 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6506d9d640 1 -- 172.21.15.59:0/58998746 learned_addr learned my addr 172.21.15.59:0/58998746 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f650659c640 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c861f10 0x55a93c86a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6505d9b640 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c862830 0x55a93c86a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6506d9d640 1 -- 172.21.15.59:0/58998746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c862830 msgr2=0x55a93c86a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:16.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6506d9d640 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c862830 0x55a93c86a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6506d9d640 1 -- 172.21.15.59:0/58998746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c861f10 msgr2=0x55a93c86a040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6506d9d640 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c861f10 0x55a93c86a040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6506d9d640 1 -- 172.21.15.59:0/58998746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93c5a5620 con 0x55a93c86d0f0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f650659c640 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c861f10 0x55a93c86a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f6506d9d640 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c86d0f0 0x55a93c86aac0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6500009cd0 tx=0x7f650002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5e08eabbc04bb6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f64edffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f64edffb640 1 -- 172.21.15.59:0/58998746 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6500056060 con 0x55a93c86d0f0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f64edffb640 1 -- 172.21.15.59:0/58998746 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f6500041030 con 0x55a93c86d0f0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.388+0000 7f650fc10740 1 -- 172.21.15.59:0/58998746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64fc011a90 con 0x55a93c86d0f0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f64edffb640 1 -- 172.21.15.59:0/58998746 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f650005d050 con 0x55a93c86d0f0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f650fc10740 10 client.15816 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f650fc10740 1 -- 172.21.15.59:0/58998746 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a93c87bb50 con 0x55a93c86d0f0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f64edffb640 1 -- 172.21.15.59:0/58998746 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64fc011a90 con 0x55a93c86d0f0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f64edffb640 1 -- 172.21.15.59:0/58998746 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f6500030070 con 0x55a93c86d0f0 2024-05-08T05:32:16.396 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f650fc10740 10 client.15816 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f650fc10740 10 client.15816 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f650fc10740 20 client.15816 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f650fc10740 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a93c87d060 0x55a93c87f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f650fc10740 4 client.15816 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f650fc10740 1 -- 172.21.15.59:0/58998746 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a93c7d51e0 con 0x55a93c87d060 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.389+0000 7f650659c640 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a93c87d060 0x55a93c87f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.390+0000 7f650659c640 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a93c87d060 0x55a93c87f490 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df18eb6abdaf4343 server_cookie=2b99dc895654cc45 in_seq=0 out_seq=0 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.390+0000 7f64edffb640 10 client.15816 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.390+0000 7f64edffb640 1 -- 172.21.15.59:0/58998746 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a93c7d51e0 con 0x55a93c87d060 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.390+0000 7f64edffb640 10 client.15816 handle_command_reply: tid=0 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.390+0000 7f650fc10740 1 client.15816 shutdown 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 -- 172.21.15.59:0/58998746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c86d0f0 msgr2=0x55a93c86aac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c86d0f0 0x55a93c86aac0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f6500009cd0 tx=0x7f650002a8e0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 -- 172.21.15.59:0/58998746 shutdown_connections 2024-05-08T05:32:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a93c87d060 0x55a93c87f490 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c86d0f0 0x55a93c86aac0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c862830 0x55a93c86a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f64edffb640 0 client.15816 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 --2- 172.21.15.59:0/58998746 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c861f10 0x55a93c86a040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 -- 172.21.15.59:0/58998746 >> 172.21.15.59:0/58998746 conn(0x55a93c85e420 msgr2=0x55a93c8641a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 -- 172.21.15.59:0/58998746 shutdown_connections 2024-05-08T05:32:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 1 -- 172.21.15.59:0/58998746 wait complete. 2024-05-08T05:32:16.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.391+0000 7f650fc10740 20 client.15816 trim_cache size 0 max 16384 2024-05-08T05:32:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650fc10740 1 Processor -- start 2024-05-08T05:32:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650fc10740 1 -- start start 2024-05-08T05:32:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650fc10740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c8648d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650fc10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 0x55a93c86a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650fc10740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c884da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650fc10740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65081b1db0 con 0x55a93c861f10 2024-05-08T05:32:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650fc10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65081b3980 con 0x55a93c862830 2024-05-08T05:32:16.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650fc10740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65081b2480 con 0x55a93c86d0f0 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c8648d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c8648d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52792/0 (socket says 172.21.15.59:52792) 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650659c640 1 -- 172.21.15.59:0/1985090309 learned_addr learned my addr 172.21.15.59:0/1985090309 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f6506d9d640 1 --2- 172.21.15.59:0/1985090309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c884da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650659c640 1 -- 172.21.15.59:0/1985090309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 msgr2=0x55a93c884da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650659c640 1 --2- 172.21.15.59:0/1985090309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c884da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650659c640 1 -- 172.21.15.59:0/1985090309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 msgr2=0x55a93c86a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650659c640 1 --2- 172.21.15.59:0/1985090309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 0x55a93c86a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f650659c640 1 -- 172.21.15.59:0/1985090309 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a93c864e10 con 0x55a93c861f10 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.406+0000 7f6506d9d640 1 --2- 172.21.15.59:0/1985090309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c884da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650659c640 1 --2- 172.21.15.59:0/1985090309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c8648d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009610 tx=0x7f64f8091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96eb57940e06d05 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f64effff640 1 -- 172.21.15.59:0/1985090309 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8020040 con 0x55a93c861f10 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f64effff640 1 -- 172.21.15.59:0/1985090309 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f80a9410 con 0x55a93c861f10 2024-05-08T05:32:16.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f64effff640 1 -- 172.21.15.59:0/1985090309 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f80a56a0 con 0x55a93c861f10 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650fc10740 1 -- 172.21.15.59:0/1985090309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 msgr2=0x55a93c8648d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650fc10740 1 --2- 172.21.15.59:0/1985090309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c8648d0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f64f8009610 tx=0x7f64f8091040 comp rx=0 tx=0).stop 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650fc10740 1 -- 172.21.15.59:0/1985090309 shutdown_connections 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650fc10740 1 --2- 172.21.15.59:0/1985090309 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c884da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650fc10740 1 --2- 172.21.15.59:0/1985090309 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 0x55a93c86a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650fc10740 1 --2- 172.21.15.59:0/1985090309 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c8648d0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650fc10740 1 -- 172.21.15.59:0/1985090309 >> 172.21.15.59:0/1985090309 conn(0x55a93c85e420 msgr2=0x55a93c86a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650fc10740 1 -- 172.21.15.59:0/1985090309 shutdown_connections 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.407+0000 7f650fc10740 1 -- 172.21.15.59:0/1985090309 wait complete. 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650fc10740 1 -- start start 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650fc10740 1 Processor -- start 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650fc10740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c870e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650fc10740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 0x55a93c871350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650fc10740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c871890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.413 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650fc10740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65081b3220 con 0x55a93c861f10 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650fc10740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65081b2b50 con 0x55a93c862830 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650fc10740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65081afd70 con 0x55a93c86d0f0 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650fc10740 10 client.0 _ll_register_callbacks cb 0x7f6500004a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c870e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c870e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52806/0 (socket says 172.21.15.59:52806) 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650659c640 1 -- 172.21.15.59:0/2639141037 learned_addr learned my addr 172.21.15.59:0/2639141037 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f6506d9d640 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c871890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f6505d9b640 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 0x55a93c871350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650659c640 1 -- 172.21.15.59:0/2639141037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 msgr2=0x55a93c871890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650659c640 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c871890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650659c640 1 -- 172.21.15.59:0/2639141037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 msgr2=0x55a93c871350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650659c640 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 0x55a93c871350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.408+0000 7f650659c640 1 -- 172.21.15.59:0/2639141037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a93c8e6a80 con 0x55a93c861f10 2024-05-08T05:32:16.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f6506d9d640 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c871890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f6505d9b640 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 0x55a93c871350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650659c640 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c870e10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f64f8005820 tx=0x7f64f80ad890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=895cee2242bbdc9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f64edffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f64edffb640 1 -- 172.21.15.59:0/2639141037 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64f8020030 con 0x55a93c861f10 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f64edffb640 1 -- 172.21.15.59:0/2639141037 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f64f80a56a0 con 0x55a93c861f10 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650fc10740 1 -- 172.21.15.59:0/2639141037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64f80a9410 con 0x55a93c861f10 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f64edffb640 1 -- 172.21.15.59:0/2639141037 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f64f80dfb00 con 0x55a93c861f10 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650fc10740 10 client.15822 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650fc10740 1 -- 172.21.15.59:0/2639141037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55a93c8e7ce0 con 0x55a93c861f10 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f64edffb640 1 -- 172.21.15.59:0/2639141037 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64f80a9410 con 0x55a93c861f10 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f64edffb640 1 -- 172.21.15.59:0/2639141037 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f64f80a3b30 con 0x55a93c861f10 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650fc10740 10 client.15822 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650fc10740 10 client.15822 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:16.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650fc10740 20 client.15822 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650fc10740 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a93c8e8460 0x55a93c8ea880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650fc10740 4 client.15822 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:32:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.409+0000 7f650fc10740 1 -- 172.21.15.59:0/2639141037 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55a93c7cf050 con 0x55a93c8e8460 2024-05-08T05:32:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.410+0000 7f6505d9b640 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a93c8e8460 0x55a93c8ea880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.410+0000 7f64edffb640 10 client.15822 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:16.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:16.410+0000 7f6505d9b640 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a93c8e8460 0x55a93c8ea880 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8515613ae8f15e76 server_cookie=cd8c75a1cacf4a29 in_seq=0 out_seq=0 2024-05-08T05:32:17.185 INFO:tasks.workunit.client.0.smithi059.stdout:autoheader 2024-05-08T05:32:17.221 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.220+0000 7f64edffb640 1 -- 172.21.15.59:0/2639141037 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55a93c7cf050 con 0x55a93c8e8460 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.220+0000 7f64edffb640 10 client.15822 handle_command_reply: tid=0 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.220+0000 7f650fc10740 1 client.15822 shutdown 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 -- 172.21.15.59:0/2639141037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 msgr2=0x55a93c870e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c870e10 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f64f8005820 tx=0x7f64f80ad890 comp rx=0 tx=0).stop 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 -- 172.21.15.59:0/2639141037 shutdown_connections 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a93c8e8460 0x55a93c8ea880 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a93c86d0f0 0x55a93c871890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a93c862830 0x55a93c871350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 --2- 172.21.15.59:0/2639141037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a93c861f10 0x55a93c870e10 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 -- 172.21.15.59:0/2639141037 >> 172.21.15.59:0/2639141037 conn(0x55a93c85e420 msgr2=0x55a93c869fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f64edffb640 0 client.15822 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 -- 172.21.15.59:0/2639141037 shutdown_connections 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 1 -- 172.21.15.59:0/2639141037 wait complete. 2024-05-08T05:32:17.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.221+0000 7f650fc10740 20 client.15822 trim_cache size 0 max 16384 2024-05-08T05:32:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:32:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:32:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:32:17.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 -- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f64f00b2a10 msgr2=0x7f64f00b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f64f00b2a10 0x7f64f00b4ed0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6500005e30 tx=0x7f6500005c30 comp rx=0 tx=0).stop 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 -- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508101900 msgr2=0x7f65080fe860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508101900 0x7f65080fe860 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f64f800d770 tx=0x7f64f800dc30 comp rx=0 tx=0).stop 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 -- 172.21.15.59:0/2724557973 shutdown_connections 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f64f00b2a10 0x7f64f00b4ed0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65081078c0 0x7f65080ff2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65081022c0 0x7f65080feda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 --2- 172.21.15.59:0/2724557973 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6508101900 0x7f65080fe860 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:17.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 -- 172.21.15.59:0/2724557973 >> 172.21.15.59:0/2724557973 conn(0x7f6508078a00 msgr2=0x7f650806af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 -- 172.21.15.59:0/2724557973 shutdown_connections 2024-05-08T05:32:17.225 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:17.222+0000 7f650ce1d640 1 -- 172.21.15.59:0/2724557973 wait complete. 2024-05-08T05:32:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:17 smithi059 ceph-mon[30304]: pgmap v84: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 840 op/s 2024-05-08T05:32:17.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:17 smithi143 ceph-mon[34337]: pgmap v84: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 840 op/s 2024-05-08T05:32:17.518 INFO:tasks.workunit.client.0.smithi059.stdout:autoconf 2024-05-08T05:32:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:17 smithi096 ceph-mon[34253]: pgmap v84: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 840 op/s 2024-05-08T05:32:17.704 INFO:tasks.workunit.client.0.smithi059.stdout:./configure \ 2024-05-08T05:32:17.704 INFO:tasks.workunit.client.0.smithi059.stdout: --libexecdir=/usr/lib \ 2024-05-08T05:32:17.704 INFO:tasks.workunit.client.0.smithi059.stdout: --exec_prefix=/var/lib 2024-05-08T05:32:17.831 INFO:tasks.workunit.client.0.smithi059.stdout:checking build system type... x86_64-pc-linux-gnu 2024-05-08T05:32:17.832 INFO:tasks.workunit.client.0.smithi059.stdout:checking host system type... x86_64-pc-linux-gnu 2024-05-08T05:32:17.832 INFO:tasks.workunit.client.0.smithi059.stdout:checking how to print strings... printf 2024-05-08T05:32:17.833 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gcc... gcc 2024-05-08T05:32:17.917 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether the C compiler works... yes 2024-05-08T05:32:17.917 INFO:tasks.workunit.client.0.smithi059.stdout:checking for C compiler default output file name... a.out 2024-05-08T05:32:17.956 INFO:tasks.workunit.client.0.smithi059.stdout:checking for suffix of executables... 2024-05-08T05:32:17.995 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether we are cross compiling... no 2024-05-08T05:32:18.032 INFO:tasks.workunit.client.0.smithi059.stdout:checking for suffix of object files... o 2024-05-08T05:32:18.057 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether we are using the GNU C compiler... yes 2024-05-08T05:32:18.082 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether gcc accepts -g... yes 2024-05-08T05:32:18.155 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gcc option to accept ISO C89... none needed 2024-05-08T05:32:18.158 INFO:tasks.workunit.client.0.smithi059.stdout:checking for a sed that does not truncate output... /usr/bin/sed 2024-05-08T05:32:18.162 INFO:tasks.workunit.client.0.smithi059.stdout:checking for grep that handles long lines and -e... /usr/bin/grep 2024-05-08T05:32:18.164 INFO:tasks.workunit.client.0.smithi059.stdout:checking for egrep... /usr/bin/grep -E 2024-05-08T05:32:18.165 INFO:tasks.workunit.client.0.smithi059.stdout:checking for fgrep... /usr/bin/grep -F 2024-05-08T05:32:18.168 INFO:tasks.workunit.client.0.smithi059.stdout:checking for ld used by gcc... /usr/bin/ld 2024-05-08T05:32:18.170 INFO:tasks.workunit.client.0.smithi059.stdout:checking if the linker (/usr/bin/ld) is GNU ld... yes 2024-05-08T05:32:18.190 INFO:tasks.workunit.client.0.smithi059.stdout:checking for BSD- or MS-compatible name lister (nm)... /usr/bin/nm -B 2024-05-08T05:32:19.094 INFO:tasks.workunit.client.0.smithi059.stdout:checking the name lister (/usr/bin/nm -B) interface... BSD nm 2024-05-08T05:32:19.095 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether ln -s works... yes 2024-05-08T05:32:19.101 INFO:tasks.workunit.client.0.smithi059.stdout:checking the maximum length of command line arguments... 1572864 2024-05-08T05:32:19.106 INFO:tasks.workunit.client.0.smithi059.stdout:checking how to convert x86_64-pc-linux-gnu file names to x86_64-pc-linux-gnu format... func_convert_file_noop 2024-05-08T05:32:19.106 INFO:tasks.workunit.client.0.smithi059.stdout:checking how to convert x86_64-pc-linux-gnu file names to toolchain format... func_convert_file_noop 2024-05-08T05:32:19.106 INFO:tasks.workunit.client.0.smithi059.stdout:checking for /usr/bin/ld option to reload object files... -r 2024-05-08T05:32:19.106 INFO:tasks.workunit.client.0.smithi059.stdout:checking for objdump... objdump 2024-05-08T05:32:19.106 INFO:tasks.workunit.client.0.smithi059.stdout:checking how to recognize dependent libraries... pass_all 2024-05-08T05:32:19.106 INFO:tasks.workunit.client.0.smithi059.stdout:checking for dlltool... no 2024-05-08T05:32:19.106 INFO:tasks.workunit.client.0.smithi059.stdout:checking how to associate runtime and link libraries... printf %s\n 2024-05-08T05:32:19.106 INFO:tasks.workunit.client.0.smithi059.stdout:checking for ar... ar 2024-05-08T05:32:19.211 INFO:tasks.workunit.client.0.smithi059.stdout:checking for archiver @FILE support... @ 2024-05-08T05:32:19.211 INFO:tasks.workunit.client.0.smithi059.stdout:checking for strip... strip 2024-05-08T05:32:19.211 INFO:tasks.workunit.client.0.smithi059.stdout:checking for ranlib... ranlib 2024-05-08T05:32:19.211 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gawk... gawk 2024-05-08T05:32:19.318 INFO:tasks.workunit.client.0.smithi059.stdout:checking command to parse /usr/bin/nm -B output from gcc object... ok 2024-05-08T05:32:19.319 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sysroot... no 2024-05-08T05:32:19.332 INFO:tasks.workunit.client.0.smithi059.stdout:checking for a working dd... /usr/bin/dd 2024-05-08T05:32:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:19 smithi059 ceph-mon[30304]: pgmap v85: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2024-05-08T05:32:19.344 INFO:tasks.workunit.client.0.smithi059.stdout:checking how to truncate binary pipes... /usr/bin/dd bs=4096 count=1 2024-05-08T05:32:19.378 INFO:tasks.workunit.client.0.smithi059.stdout:checking for mt... no 2024-05-08T05:32:19.393 INFO:tasks.workunit.client.0.smithi059.stdout:checking if : is a manifest tool... no 2024-05-08T05:32:19.485 INFO:tasks.workunit.client.0.smithi059.stdout:checking how to run the C preprocessor... gcc -E 2024-05-08T05:32:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:19 smithi143 ceph-mon[34337]: pgmap v85: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2024-05-08T05:32:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:19 smithi096 ceph-mon[34253]: pgmap v85: 129 pgs: 129 active+clean; 38 MiB data, 496 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.0 MiB/s wr, 283 op/s 2024-05-08T05:32:19.679 INFO:tasks.workunit.client.0.smithi059.stdout:checking for ANSI C header files... yes 2024-05-08T05:32:19.714 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/types.h... yes 2024-05-08T05:32:19.796 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/stat.h... yes 2024-05-08T05:32:19.837 INFO:tasks.workunit.client.0.smithi059.stdout:checking for stdlib.h... yes 2024-05-08T05:32:19.879 INFO:tasks.workunit.client.0.smithi059.stdout:checking for string.h... yes 2024-05-08T05:32:19.924 INFO:tasks.workunit.client.0.smithi059.stdout:checking for memory.h... yes 2024-05-08T05:32:19.967 INFO:tasks.workunit.client.0.smithi059.stdout:checking for strings.h... yes 2024-05-08T05:32:20.012 INFO:tasks.workunit.client.0.smithi059.stdout:checking for inttypes.h... yes 2024-05-08T05:32:20.058 INFO:tasks.workunit.client.0.smithi059.stdout:checking for stdint.h... yes 2024-05-08T05:32:20.101 INFO:tasks.workunit.client.0.smithi059.stdout:checking for unistd.h... yes 2024-05-08T05:32:20.146 INFO:tasks.workunit.client.0.smithi059.stdout:checking for dlfcn.h... yes 2024-05-08T05:32:20.152 INFO:tasks.workunit.client.0.smithi059.stdout:checking for objdir... .libs 2024-05-08T05:32:20.276 INFO:tasks.workunit.client.0.smithi059.stdout:checking if gcc supports -fno-rtti -fno-exceptions... no 2024-05-08T05:32:20.276 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gcc option to produce PIC... -fPIC -DPIC 2024-05-08T05:32:20.316 INFO:tasks.workunit.client.0.smithi059.stdout:checking if gcc PIC flag -fPIC -DPIC works... yes 2024-05-08T05:32:20.365 INFO:tasks.workunit.client.0.smithi059.stdout:checking if gcc static flag -static works... no 2024-05-08T05:32:20.428 INFO:tasks.workunit.client.0.smithi059.stdout:checking if gcc supports -c -o file.o... yes 2024-05-08T05:32:20.429 INFO:tasks.workunit.client.0.smithi059.stdout:checking if gcc supports -c -o file.o... (cached) yes 2024-05-08T05:32:20.435 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether the gcc linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes 2024-05-08T05:32:20.492 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether -lc should be explicitly linked in... no 2024-05-08T05:32:20.563 INFO:tasks.workunit.client.0.smithi059.stdout:checking dynamic linker characteristics... GNU/Linux ld.so 2024-05-08T05:32:20.563 INFO:tasks.workunit.client.0.smithi059.stdout:checking how to hardcode library paths into programs... immediate 2024-05-08T05:32:20.565 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether stripping libraries is possible... yes 2024-05-08T05:32:20.565 INFO:tasks.workunit.client.0.smithi059.stdout:checking if libtool supports shared libraries... yes 2024-05-08T05:32:20.565 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether to build shared libraries... yes 2024-05-08T05:32:20.565 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether to build static libraries... yes 2024-05-08T05:32:20.571 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gcc... (cached) gcc 2024-05-08T05:32:20.616 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether we are using the GNU C compiler... (cached) yes 2024-05-08T05:32:20.616 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether gcc accepts -g... (cached) yes 2024-05-08T05:32:20.616 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gcc option to accept ISO C89... (cached) none needed 2024-05-08T05:32:20.616 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gmake... /usr/bin/gmake 2024-05-08T05:32:20.616 INFO:tasks.workunit.client.0.smithi059.stderr:./configure: line 12082: test: $(SHELL): binary operator expected 2024-05-08T05:32:20.617 INFO:tasks.workunit.client.0.smithi059.stdout:checking for tar... /bin/tar 2024-05-08T05:32:20.617 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gzip... /bin/gzip 2024-05-08T05:32:20.639 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether gcc -MM is supported... yes 2024-05-08T05:32:20.639 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sort... /bin/sort 2024-05-08T05:32:20.639 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether ln -s works... yes 2024-05-08T05:32:20.639 INFO:tasks.workunit.client.0.smithi059.stdout:checking for rpm... /bin/rpm 2024-05-08T05:32:20.642 INFO:tasks.workunit.client.0.smithi059.stdout:checking for rpmbuild... no 2024-05-08T05:32:20.703 INFO:tasks.workunit.client.0.smithi059.stdout:checking assert.h usability... yes 2024-05-08T05:32:20.720 INFO:tasks.workunit.client.0.smithi059.stdout:checking assert.h presence... yes 2024-05-08T05:32:20.721 INFO:tasks.workunit.client.0.smithi059.stdout:checking for assert.h... yes 2024-05-08T05:32:20.774 INFO:tasks.workunit.client.0.smithi059.stdout:checking bstring.h usability... no 2024-05-08T05:32:20.803 INFO:tasks.workunit.client.0.smithi059.stdout:checking bstring.h presence... no 2024-05-08T05:32:20.804 INFO:tasks.workunit.client.0.smithi059.stdout:checking for bstring.h... no 2024-05-08T05:32:20.847 INFO:tasks.workunit.client.0.smithi059.stdout:checking libgen.h usability... yes 2024-05-08T05:32:20.865 INFO:tasks.workunit.client.0.smithi059.stdout:checking libgen.h presence... yes 2024-05-08T05:32:20.865 INFO:tasks.workunit.client.0.smithi059.stdout:checking for libgen.h... yes 2024-05-08T05:32:20.914 INFO:tasks.workunit.client.0.smithi059.stdout:checking dirent.h usability... yes 2024-05-08T05:32:20.932 INFO:tasks.workunit.client.0.smithi059.stdout:checking dirent.h presence... yes 2024-05-08T05:32:20.932 INFO:tasks.workunit.client.0.smithi059.stdout:checking for dirent.h... yes 2024-05-08T05:32:20.983 INFO:tasks.workunit.client.0.smithi059.stdout:checking errno.h usability... yes 2024-05-08T05:32:21.005 INFO:tasks.workunit.client.0.smithi059.stdout:checking errno.h presence... yes 2024-05-08T05:32:21.005 INFO:tasks.workunit.client.0.smithi059.stdout:checking for errno.h... yes 2024-05-08T05:32:21.052 INFO:tasks.workunit.client.0.smithi059.stdout:checking malloc.h usability... yes 2024-05-08T05:32:21.071 INFO:tasks.workunit.client.0.smithi059.stdout:checking malloc.h presence... yes 2024-05-08T05:32:21.071 INFO:tasks.workunit.client.0.smithi059.stdout:checking for malloc.h... yes 2024-05-08T05:32:21.128 INFO:tasks.workunit.client.0.smithi059.stdout:checking uuid.h usability... no 2024-05-08T05:32:21.163 INFO:tasks.workunit.client.0.smithi059.stdout:checking uuid.h presence... no 2024-05-08T05:32:21.163 INFO:tasks.workunit.client.0.smithi059.stdout:checking for uuid.h... no 2024-05-08T05:32:21.226 INFO:tasks.workunit.client.0.smithi059.stdout:checking uuid/uuid.h usability... yes 2024-05-08T05:32:21.245 INFO:tasks.workunit.client.0.smithi059.stdout:checking uuid/uuid.h presence... yes 2024-05-08T05:32:21.245 INFO:tasks.workunit.client.0.smithi059.stdout:checking for uuid/uuid.h... yes 2024-05-08T05:32:21.306 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/uuid.h usability... no 2024-05-08T05:32:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:21 smithi059 ceph-mon[30304]: pgmap v86: 129 pgs: 129 active+clean; 24 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 907 op/s 2024-05-08T05:32:21.337 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/uuid.h presence... no 2024-05-08T05:32:21.337 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/uuid.h... no 2024-05-08T05:32:21.382 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/file.h usability... yes 2024-05-08T05:32:21.400 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/file.h presence... yes 2024-05-08T05:32:21.400 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/file.h... yes 2024-05-08T05:32:21.448 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/fcntl.h usability... yes 2024-05-08T05:32:21.465 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/fcntl.h presence... yes 2024-05-08T05:32:21.465 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/fcntl.h... yes 2024-05-08T05:32:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:21 smithi143 ceph-mon[34337]: pgmap v86: 129 pgs: 129 active+clean; 24 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 907 op/s 2024-05-08T05:32:21.517 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/syssgi.h usability... no 2024-05-08T05:32:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:21 smithi096 ceph-mon[34253]: pgmap v86: 129 pgs: 129 active+clean; 24 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 2.4 MiB/s wr, 907 op/s 2024-05-08T05:32:21.548 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/syssgi.h presence... no 2024-05-08T05:32:21.548 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/syssgi.h... no 2024-05-08T05:32:21.618 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/param.h usability... yes 2024-05-08T05:32:21.639 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/param.h presence... yes 2024-05-08T05:32:21.639 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/param.h... yes 2024-05-08T05:32:21.653 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/stat.h... (cached) yes 2024-05-08T05:32:21.693 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/statvfs.h usability... yes 2024-05-08T05:32:21.711 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/statvfs.h presence... yes 2024-05-08T05:32:21.711 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/statvfs.h... yes 2024-05-08T05:32:21.766 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/time.h usability... yes 2024-05-08T05:32:21.788 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/time.h presence... yes 2024-05-08T05:32:21.788 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/time.h... yes 2024-05-08T05:32:21.836 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/ioctl.h usability... yes 2024-05-08T05:32:21.854 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/ioctl.h presence... yes 2024-05-08T05:32:21.854 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/ioctl.h... yes 2024-05-08T05:32:21.927 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/wait.h usability... yes 2024-05-08T05:32:21.945 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/wait.h presence... yes 2024-05-08T05:32:21.946 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/wait.h... yes 2024-05-08T05:32:21.965 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/types.h... (cached) yes 2024-05-08T05:32:21.970 INFO:tasks.workunit.client.0.smithi059.stdout:checking for strings.h... (cached) yes 2024-05-08T05:32:22.007 INFO:tasks.workunit.client.0.smithi059.stdout:checking err.h usability... yes 2024-05-08T05:32:22.025 INFO:tasks.workunit.client.0.smithi059.stdout:checking err.h presence... yes 2024-05-08T05:32:22.025 INFO:tasks.workunit.client.0.smithi059.stdout:checking for err.h... yes 2024-05-08T05:32:22.072 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/falloc.h usability... yes 2024-05-08T05:32:22.089 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/falloc.h presence... yes 2024-05-08T05:32:22.089 INFO:tasks.workunit.client.0.smithi059.stdout:checking for linux/falloc.h... yes 2024-05-08T05:32:22.141 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/fs/xfs_fsops.h usability... no 2024-05-08T05:32:22.170 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/fs/xfs_fsops.h presence... no 2024-05-08T05:32:22.170 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/fs/xfs_fsops.h... no 2024-05-08T05:32:22.215 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/fs/xfs_itable.h usability... no 2024-05-08T05:32:22.246 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/fs/xfs_itable.h presence... no 2024-05-08T05:32:22.246 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/fs/xfs_itable.h... no 2024-05-08T05:32:22.290 INFO:tasks.workunit.client.0.smithi059.stdout:checking xfs/platform_defs.h usability... no 2024-05-08T05:32:22.324 INFO:tasks.workunit.client.0.smithi059.stdout:checking xfs/platform_defs.h presence... no 2024-05-08T05:32:22.324 INFO:tasks.workunit.client.0.smithi059.stdout:checking for xfs/platform_defs.h... no 2024-05-08T05:32:22.375 INFO:tasks.workunit.client.0.smithi059.stdout:checking btrfs/ioctl.h usability... no 2024-05-08T05:32:22.411 INFO:tasks.workunit.client.0.smithi059.stdout:checking btrfs/ioctl.h presence... no 2024-05-08T05:32:22.412 INFO:tasks.workunit.client.0.smithi059.stdout:checking for btrfs/ioctl.h... no 2024-05-08T05:32:22.456 INFO:tasks.workunit.client.0.smithi059.stdout:checking cifs/ioctl.h usability... no 2024-05-08T05:32:22.486 INFO:tasks.workunit.client.0.smithi059.stdout:checking cifs/ioctl.h presence... no 2024-05-08T05:32:22.486 INFO:tasks.workunit.client.0.smithi059.stdout:checking for cifs/ioctl.h... no 2024-05-08T05:32:22.532 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/mman.h usability... yes 2024-05-08T05:32:22.549 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/mman.h presence... yes 2024-05-08T05:32:22.549 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/mman.h... yes 2024-05-08T05:32:22.601 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/ext4.h usability... no 2024-05-08T05:32:22.632 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/ext4.h presence... no 2024-05-08T05:32:22.632 INFO:tasks.workunit.client.0.smithi059.stdout:checking for linux/ext4.h... no 2024-05-08T05:32:22.660 INFO:tasks.workunit.client.0.smithi059.stdout:checking for xfs/xfs_log_format.h... no 2024-05-08T05:32:22.716 INFO:tasks.workunit.client.0.smithi059.stdout:checking for uuid_compare... no 2024-05-08T05:32:22.757 INFO:tasks.workunit.client.0.smithi059.stdout:checking for uuid_compare in -luuid... yes 2024-05-08T05:32:22.813 INFO:tasks.workunit.client.0.smithi059.stdout:checking for xfs/xfs.h... yes 2024-05-08T05:32:22.849 INFO:tasks.workunit.client.0.smithi059.stdout:checking for xfs/libxfs.h... no 2024-05-08T05:32:22.880 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether xlog_assign_lsn is declared... no 2024-05-08T05:32:22.920 INFO:tasks.workunit.client.0.smithi059.stdout:checking for xfs/xqm.h... yes 2024-05-08T05:32:22.975 INFO:tasks.workunit.client.0.smithi059.stdout:checking xfsctl from xfs/xfs.h... ok 2024-05-08T05:32:23.013 INFO:tasks.workunit.client.0.smithi059.stdout:checking xfs/handle.h usability... yes 2024-05-08T05:32:23.030 INFO:tasks.workunit.client.0.smithi059.stdout:checking xfs/handle.h presence... yes 2024-05-08T05:32:23.030 INFO:tasks.workunit.client.0.smithi059.stdout:checking for xfs/handle.h... yes 2024-05-08T05:32:23.091 INFO:tasks.workunit.client.0.smithi059.stdout:checking for attr_list_by_handle in -lhandle... yes 2024-05-08T05:32:23.133 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/acl.h usability... yes 2024-05-08T05:32:23.151 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/acl.h presence... yes 2024-05-08T05:32:23.151 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/acl.h... yes 2024-05-08T05:32:23.199 INFO:tasks.workunit.client.0.smithi059.stdout:checking acl/libacl.h usability... yes 2024-05-08T05:32:23.219 INFO:tasks.workunit.client.0.smithi059.stdout:checking acl/libacl.h presence... yes 2024-05-08T05:32:23.220 INFO:tasks.workunit.client.0.smithi059.stdout:checking for acl/libacl.h... yes 2024-05-08T05:32:23.262 INFO:tasks.workunit.client.0.smithi059.stdout:checking for acl_init in -lacl... yes 2024-05-08T05:32:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:23 smithi059 ceph-mon[30304]: pgmap v87: 129 pgs: 129 active+clean; 24 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 901 op/s 2024-05-08T05:32:23.337 INFO:tasks.workunit.client.0.smithi059.stdout:checking gdbm-ndbm.h usability... no 2024-05-08T05:32:23.367 INFO:tasks.workunit.client.0.smithi059.stdout:checking gdbm-ndbm.h presence... no 2024-05-08T05:32:23.367 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gdbm-ndbm.h... no 2024-05-08T05:32:23.413 INFO:tasks.workunit.client.0.smithi059.stdout:checking gdbm/ndbm.h usability... no 2024-05-08T05:32:23.444 INFO:tasks.workunit.client.0.smithi059.stdout:checking gdbm/ndbm.h presence... no 2024-05-08T05:32:23.444 INFO:tasks.workunit.client.0.smithi059.stdout:checking for gdbm/ndbm.h... no 2024-05-08T05:32:23.483 INFO:tasks.workunit.client.0.smithi059.stdout:checking ndbm.h usability... no 2024-05-08T05:32:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:23 smithi143 ceph-mon[34337]: pgmap v87: 129 pgs: 129 active+clean; 24 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 901 op/s 2024-05-08T05:32:23.512 INFO:tasks.workunit.client.0.smithi059.stdout:checking ndbm.h presence... no 2024-05-08T05:32:23.512 INFO:tasks.workunit.client.0.smithi059.stdout:checking for ndbm.h... no 2024-05-08T05:32:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:23 smithi096 ceph-mon[34253]: pgmap v87: 129 pgs: 129 active+clean; 24 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 901 op/s 2024-05-08T05:32:23.545 INFO:tasks.workunit.client.0.smithi059.stdout:checking libaio.h usability... yes 2024-05-08T05:32:23.564 INFO:tasks.workunit.client.0.smithi059.stdout:checking libaio.h presence... yes 2024-05-08T05:32:23.564 INFO:tasks.workunit.client.0.smithi059.stdout:checking for libaio.h... yes 2024-05-08T05:32:23.611 INFO:tasks.workunit.client.0.smithi059.stdout:checking liburing.h usability... no 2024-05-08T05:32:23.641 INFO:tasks.workunit.client.0.smithi059.stdout:checking liburing.h presence... no 2024-05-08T05:32:23.641 INFO:tasks.workunit.client.0.smithi059.stdout:checking for liburing.h... no 2024-05-08T05:32:23.681 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/capability.h usability... no 2024-05-08T05:32:23.710 INFO:tasks.workunit.client.0.smithi059.stdout:checking sys/capability.h presence... no 2024-05-08T05:32:23.711 INFO:tasks.workunit.client.0.smithi059.stdout:checking for sys/capability.h... no 2024-05-08T05:32:23.745 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/fiemap.h usability... yes 2024-05-08T05:32:23.761 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/fiemap.h presence... yes 2024-05-08T05:32:23.761 INFO:tasks.workunit.client.0.smithi059.stdout:checking for linux/fiemap.h... yes 2024-05-08T05:32:23.800 INFO:tasks.workunit.client.0.smithi059.stdout:checking for fallocate... yes 2024-05-08T05:32:23.846 INFO:tasks.workunit.client.0.smithi059.stdout:checking for open_by_handle_at... yes 2024-05-08T05:32:23.885 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/fs.h usability... yes 2024-05-08T05:32:23.902 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/fs.h presence... yes 2024-05-08T05:32:23.902 INFO:tasks.workunit.client.0.smithi059.stdout:checking for linux/fs.h... yes 2024-05-08T05:32:23.961 INFO:tasks.workunit.client.0.smithi059.stdout:checking btrfsutil.h usability... no 2024-05-08T05:32:23.991 INFO:tasks.workunit.client.0.smithi059.stdout:checking btrfsutil.h presence... no 2024-05-08T05:32:23.991 INFO:tasks.workunit.client.0.smithi059.stdout:checking for btrfsutil.h... no 2024-05-08T05:32:24.033 INFO:tasks.workunit.client.0.smithi059.stdout:checking for copy_file_range... yes 2024-05-08T05:32:24.079 INFO:tasks.workunit.client.0.smithi059.stdout:checking for SEEK_DATA... yes 2024-05-08T05:32:24.139 INFO:tasks.workunit.client.0.smithi059.stdout:checking for BMV_OF_SHARED... yes 2024-05-08T05:32:24.185 INFO:tasks.workunit.client.0.smithi059.stdout:checking for nftw... yes 2024-05-08T05:32:24.239 INFO:tasks.workunit.client.0.smithi059.stdout:checking for RLIMIT_NOFILE... yes 2024-05-08T05:32:24.284 INFO:tasks.workunit.client.0.smithi059.stdout:checking for XFS_IOC_EXCHANGE_RANGE... no 2024-05-08T05:32:24.350 INFO:tasks.workunit.client.0.smithi059.stdout:checking for FICLONE... yes 2024-05-08T05:32:24.397 INFO:tasks.workunit.client.0.smithi059.stdout:checking for renameat2... yes 2024-05-08T05:32:24.441 INFO:tasks.workunit.client.0.smithi059.stdout:checking for reallocarray... yes 2024-05-08T05:32:24.508 INFO:tasks.workunit.client.0.smithi059.stdout:checking for struct mount_attr... yes 2024-05-08T05:32:24.570 INFO:tasks.workunit.client.0.smithi059.stdout:checking for struct btrfs_qgroup_limit... yes 2024-05-08T05:32:24.629 INFO:tasks.workunit.client.0.smithi059.stdout:checking for struct btrfs_qgroup_inherit... yes 2024-05-08T05:32:24.690 INFO:tasks.workunit.client.0.smithi059.stdout:checking for struct btrfs_ioctl_vol_args... yes 2024-05-08T05:32:24.749 INFO:tasks.workunit.client.0.smithi059.stdout:checking for struct btrfs_ioctl_vol_args_v2... yes 2024-05-08T05:32:24.808 INFO:tasks.workunit.client.0.smithi059.stdout:checking for struct btrfs_ioctl_ino_lookup_args... yes 2024-05-08T05:32:24.866 INFO:tasks.workunit.client.0.smithi059.stdout:checking for struct btrfs_ioctl_ino_lookup_user_args... yes 2024-05-08T05:32:25.051 INFO:tasks.workunit.client.0.smithi059.stdout:checking for struct btrfs_ioctl_get_subvol_rootref_args... yes 2024-05-08T05:32:25.151 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/btrfs.h usability... yes 2024-05-08T05:32:25.168 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/btrfs.h presence... yes 2024-05-08T05:32:25.168 INFO:tasks.workunit.client.0.smithi059.stdout:checking for linux/btrfs.h... yes 2024-05-08T05:32:25.223 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/btrfs_tree.h usability... yes 2024-05-08T05:32:25.241 INFO:tasks.workunit.client.0.smithi059.stdout:checking linux/btrfs_tree.h presence... yes 2024-05-08T05:32:25.241 INFO:tasks.workunit.client.0.smithi059.stdout:checking for linux/btrfs_tree.h... yes 2024-05-08T05:32:25.284 INFO:tasks.workunit.client.0.smithi059.stdout:checking for struct btrfs_ioctl_vol_args_v2.subvolid... yes 2024-05-08T05:32:25.315 INFO:tasks.workunit.client.0.smithi059.stdout:checking whether BTRFS_IOC_SNAP_DESTROY_V2 is declared... yes 2024-05-08T05:32:25.336 INFO:tasks.workunit.client.0.smithi059.stdout:configure: creating ./config.status 2024-05-08T05:32:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:25 smithi059 ceph-mon[30304]: pgmap v88: 129 pgs: 129 active+clean; 24 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 901 op/s 2024-05-08T05:32:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:25 smithi143 ceph-mon[34337]: pgmap v88: 129 pgs: 129 active+clean; 24 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 901 op/s 2024-05-08T05:32:25.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:25 smithi096 ceph-mon[34253]: pgmap v88: 129 pgs: 129 active+clean; 24 MiB data, 508 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 2.3 MiB/s wr, 901 op/s 2024-05-08T05:32:25.697 INFO:tasks.workunit.client.0.smithi059.stdout:config.status: creating include/builddefs 2024-05-08T05:32:25.711 INFO:tasks.workunit.client.0.smithi059.stdout:config.status: creating include/config.h 2024-05-08T05:32:25.722 INFO:tasks.workunit.client.0.smithi059.stdout:config.status: executing libtool commands 2024-05-08T05:32:25.808 INFO:tasks.workunit.client.0.smithi059.stdout:Building include 2024-05-08T05:32:25.808 INFO:tasks.workunit.client.0.smithi059.stdout:Building lib 2024-05-08T05:32:25.808 INFO:tasks.workunit.client.0.smithi059.stdout:Building tests 2024-05-08T05:32:25.816 INFO:tasks.workunit.client.0.smithi059.stdout: [LTDEP] 2024-05-08T05:32:25.823 INFO:tasks.workunit.client.0.smithi059.stdout:Building ceph 2024-05-08T05:32:25.824 INFO:tasks.workunit.client.0.smithi059.stdout:Building btrfs 2024-05-08T05:32:25.824 INFO:tasks.workunit.client.0.smithi059.stdout:Building cifs 2024-05-08T05:32:25.832 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/cifs/group.list 2024-05-08T05:32:25.832 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ceph/group.list 2024-05-08T05:32:25.850 INFO:tasks.workunit.client.0.smithi059.stdout:Building ext4 2024-05-08T05:32:25.851 INFO:tasks.workunit.client.0.smithi059.stdout:Building f2fs 2024-05-08T05:32:25.858 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/f2fs/group.list 2024-05-08T05:32:25.874 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ext4/group.list 2024-05-08T05:32:25.881 INFO:tasks.workunit.client.0.smithi059.stdout:Building generic 2024-05-08T05:32:25.937 INFO:tasks.workunit.client.0.smithi059.stdout:Building nfs 2024-05-08T05:32:25.944 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/nfs/group.list 2024-05-08T05:32:25.955 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/btrfs/group.list 2024-05-08T05:32:25.963 INFO:tasks.workunit.client.0.smithi059.stdout:Building ocfs2 2024-05-08T05:32:25.971 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/ocfs2/group.list 2024-05-08T05:32:25.989 INFO:tasks.workunit.client.0.smithi059.stdout:Building overlay 2024-05-08T05:32:26.006 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/generic/group.list 2024-05-08T05:32:26.010 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/overlay/group.list 2024-05-08T05:32:26.013 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] dataascii.lo 2024-05-08T05:32:26.035 INFO:tasks.workunit.client.0.smithi059.stdout:Building perf 2024-05-08T05:32:26.042 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/perf/group.list 2024-05-08T05:32:26.058 INFO:tasks.workunit.client.0.smithi059.stdout:Building selftest 2024-05-08T05:32:26.063 INFO:tasks.workunit.client.0.smithi059.stdout:Building shared 2024-05-08T05:32:26.065 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/selftest/group.list 2024-05-08T05:32:26.070 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/shared/group.list 2024-05-08T05:32:26.083 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] databin.lo 2024-05-08T05:32:26.093 INFO:tasks.workunit.client.0.smithi059.stdout:Building tmpfs 2024-05-08T05:32:26.100 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/tmpfs/group.list 2024-05-08T05:32:26.118 INFO:tasks.workunit.client.0.smithi059.stdout:Building udf 2024-05-08T05:32:26.125 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/udf/group.list 2024-05-08T05:32:26.142 INFO:tasks.workunit.client.0.smithi059.stdout:Building xfs 2024-05-08T05:32:26.149 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] datapid.lo 2024-05-08T05:32:26.185 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] file_lock.lo 2024-05-08T05:32:26.229 INFO:tasks.workunit.client.0.smithi059.stdout: [GROUP] /home/ubuntu/cephtest/mnt.0/client.0/tmp/xfstests-dev/tests/xfs/group.list 2024-05-08T05:32:26.273 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] forker.lo 2024-05-08T05:32:26.291 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] pattern.lo 2024-05-08T05:32:26.363 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] open_flags.lo 2024-05-08T05:32:26.381 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] random_range.lo 2024-05-08T05:32:26.445 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] string_to_tokens.lo 2024-05-08T05:32:26.462 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] str_to_bytes.lo 2024-05-08T05:32:26.546 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] tlibio.lo 2024-05-08T05:32:26.583 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] write_log.lo 2024-05-08T05:32:26.663 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] random.lo 2024-05-08T05:32:26.879 INFO:tasks.workunit.client.0.smithi059.stdout: [LD] 2024-05-08T05:32:26.992 INFO:tasks.workunit.client.0.smithi059.stdout:Building ltp 2024-05-08T05:32:26.992 INFO:tasks.workunit.client.0.smithi059.stdout:Building src 2024-05-08T05:32:26.992 INFO:tasks.workunit.client.0.smithi059.stdout:Building m4 2024-05-08T05:32:26.993 INFO:tasks.workunit.client.0.smithi059.stdout:Building common 2024-05-08T05:32:26.998 INFO:tasks.workunit.client.0.smithi059.stdout: [DEP] 2024-05-08T05:32:27.010 INFO:tasks.workunit.client.0.smithi059.stdout: [DEP] 2024-05-08T05:32:27.190 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] doio 2024-05-08T05:32:27.191 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fsstress 2024-05-08T05:32:27.191 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fsx 2024-05-08T05:32:27.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:27 smithi059 ceph-mon[30304]: pgmap v89: 129 pgs: 129 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 917 op/s 2024-05-08T05:32:27.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:27 smithi143 ceph-mon[34337]: pgmap v89: 129 pgs: 129 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 917 op/s 2024-05-08T05:32:27.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:27 smithi096 ceph-mon[34253]: pgmap v89: 129 pgs: 129 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 2.6 MiB/s wr, 917 op/s 2024-05-08T05:32:27.883 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] growfiles 2024-05-08T05:32:27.959 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] iogen 2024-05-08T05:32:28.343 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-stress 2024-05-08T05:32:29.010 INFO:tasks.workunit.client.0.smithi059.stdout: [DEP] 2024-05-08T05:32:29.011 INFO:tasks.workunit.client.0.smithi059.stdout: [DEP] 2024-05-08T05:32:29.011 INFO:tasks.workunit.client.0.smithi059.stdout: [DEP] 2024-05-08T05:32:29.011 INFO:tasks.workunit.client.0.smithi059.stdout: [DEP] 2024-05-08T05:32:29.249 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] dirstress 2024-05-08T05:32:29.249 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fill 2024-05-08T05:32:29.249 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fill2 2024-05-08T05:32:29.249 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] getpagesize 2024-05-08T05:32:29.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:29 smithi059 ceph-mon[30304]: pgmap v90: 129 pgs: 129 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 717 KiB/s wr, 640 op/s 2024-05-08T05:32:29.396 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] holes 2024-05-08T05:32:29.414 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] lstat64 2024-05-08T05:32:29.483 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] nametest 2024-05-08T05:32:29.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:29 smithi143 ceph-mon[34337]: pgmap v90: 129 pgs: 129 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 717 KiB/s wr, 640 op/s 2024-05-08T05:32:29.501 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] permname 2024-05-08T05:32:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:29 smithi096 ceph-mon[34253]: pgmap v90: 129 pgs: 129 active+clean; 27 MiB data, 512 MiB used, 1.0 TiB / 1.0 TiB avail; 454 KiB/s rd, 717 KiB/s wr, 640 op/s 2024-05-08T05:32:29.605 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] randholes 2024-05-08T05:32:29.613 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] runas 2024-05-08T05:32:29.738 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] truncfile 2024-05-08T05:32:29.743 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] usemem 2024-05-08T05:32:29.799 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] mmapcat 2024-05-08T05:32:29.905 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] append_reader 2024-05-08T05:32:29.905 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] append_writer 2024-05-08T05:32:29.937 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] dirperf 2024-05-08T05:32:29.944 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] metaperf 2024-05-08T05:32:30.058 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] devzero 2024-05-08T05:32:30.065 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] feature 2024-05-08T05:32:30.189 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] alloc 2024-05-08T05:32:30.233 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fault 2024-05-08T05:32:30.236 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fstest 2024-05-08T05:32:30.324 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_access_root 2024-05-08T05:32:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.348+0000 7f20b131c640 1 -- 172.21.15.59:0/2347003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac1018b0 msgr2=0x7f20ac101d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.348+0000 7f20b131c640 1 --2- 172.21.15.59:0/2347003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac1018b0 0x7f20ac101d30 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f20a0009930 tx=0x7f20a002a420 comp rx=0 tx=0).stop 2024-05-08T05:32:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.348+0000 7f20b131c640 1 -- 172.21.15.59:0/2347003 shutdown_connections 2024-05-08T05:32:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.348+0000 7f20b131c640 1 --2- 172.21.15.59:0/2347003 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20ac102270 0x7f20ac10e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.348+0000 7f20b131c640 1 --2- 172.21.15.59:0/2347003 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac1018b0 0x7f20ac101d30 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.348+0000 7f20b131c640 1 --2- 172.21.15.59:0/2347003 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac1078c0 0x7f20ac107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.348+0000 7f20b131c640 1 -- 172.21.15.59:0/2347003 >> 172.21.15.59:0/2347003 conn(0x7f20ac078a00 msgr2=0x7f20ac078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.348+0000 7f20b131c640 1 -- 172.21.15.59:0/2347003 shutdown_connections 2024-05-08T05:32:30.349 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.348+0000 7f20b131c640 1 -- 172.21.15.59:0/2347003 wait complete. 2024-05-08T05:32:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20b131c640 1 Processor -- start 2024-05-08T05:32:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20b131c640 1 -- start start 2024-05-08T05:32:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20b131c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20ac1018b0 0x7f20ac196ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20b131c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac102270 0x7f20ac197400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20b131c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac1078c0 0x7f20ac19b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20b131c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac110ac0 con 0x7f20ac1078c0 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20b131c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac110940 con 0x7f20ac1018b0 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20b131c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac110c40 con 0x7f20ac102270 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20aaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20ac1018b0 0x7f20ac196ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac1078c0 0x7f20ac19b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20aaffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20ac1018b0 0x7f20ac196ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:51776/0 (socket says 172.21.15.59:51776) 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20aaffd640 1 -- 172.21.15.59:0/911340053 learned_addr learned my addr 172.21.15.59:0/911340053 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.349+0000 7f20aa7fc640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac102270 0x7f20ac197400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f20aaffd640 1 -- 172.21.15.59:0/911340053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac102270 msgr2=0x7f20ac197400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f20aaffd640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac102270 0x7f20ac197400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f20aaffd640 1 -- 172.21.15.59:0/911340053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac1078c0 msgr2=0x7f20ac19b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f20aaffd640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac1078c0 0x7f20ac19b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f20aaffd640 1 -- 172.21.15.59:0/911340053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20ac19beb0 con 0x7f20ac1018b0 2024-05-08T05:32:30.352 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f20aaffd640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20ac1018b0 0x7f20ac196ec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f209c00e9f0 tx=0x7f209c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f2093fff640 1 -- 172.21.15.59:0/911340053 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c00cd80 con 0x7f20ac1018b0 2024-05-08T05:32:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f2093fff640 1 -- 172.21.15.59:0/911340053 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f209c0108e0 con 0x7f20ac1018b0 2024-05-08T05:32:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f2093fff640 1 -- 172.21.15.59:0/911340053 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f209c018980 con 0x7f20ac1018b0 2024-05-08T05:32:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f20b131c640 1 -- 172.21.15.59:0/911340053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20ac19c1a0 con 0x7f20ac1018b0 2024-05-08T05:32:30.353 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.350+0000 7f20b131c640 1 -- 172.21.15.59:0/911340053 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20ac1a3a30 con 0x7f20ac1018b0 2024-05-08T05:32:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.354+0000 7f20b4030740 1 Processor -- start 2024-05-08T05:32:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.354+0000 7f20b4030740 1 -- start start 2024-05-08T05:32:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.354+0000 7f2093fff640 1 -- 172.21.15.59:0/911340053 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f209c010430 con 0x7f20ac1018b0 2024-05-08T05:32:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.354+0000 7f20b4030740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc618210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20b4030740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc626fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20b4030740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 0x55fbbc6298f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20b4030740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc57d5d0 con 0x55fbbc617e30 2024-05-08T05:32:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20b4030740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc536890 con 0x55fbbc618750 2024-05-08T05:32:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20b4030740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc579590 con 0x55fbbc627500 2024-05-08T05:32:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc618210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc618210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47832/0 (socket says 172.21.15.59:47832) 2024-05-08T05:32:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20ab7fe640 1 -- 172.21.15.59:0/1311119269 learned_addr learned my addr 172.21.15.59:0/1311119269 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20aa7fc640 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 0x55fbbc6298f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20aaffd640 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc626fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20ab7fe640 1 -- 172.21.15.59:0/1311119269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 msgr2=0x55fbbc6298f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 0x55fbbc6298f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20ab7fe640 1 -- 172.21.15.59:0/1311119269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 msgr2=0x55fbbc626fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc626fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.355+0000 7f20ab7fe640 1 -- 172.21.15.59:0/1311119269 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbbc28f210 con 0x55fbbc617e30 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.356+0000 7f20aaffd640 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc626fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.356+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc618210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009fd0 tx=0x7f20a40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4340bf0b92b934b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.356+0000 7f2093fff640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f20940b2b10 0x7f20940b4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.356+0000 7f20917fa640 1 -- 172.21.15.59:0/1311119269 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4004d20 con 0x55fbbc617e30 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.356+0000 7f20917fa640 1 -- 172.21.15.59:0/1311119269 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a4004050 con 0x55fbbc617e30 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.356+0000 7f20917fa640 1 -- 172.21.15.59:0/1311119269 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4005280 con 0x55fbbc617e30 2024-05-08T05:32:30.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.356+0000 7f20aaffd640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f20940b2b10 0x7f20940b4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.356+0000 7f2093fff640 1 -- 172.21.15.59:0/911340053 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f209c0e2c50 con 0x7f20ac1018b0 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20b4030740 1 -- 172.21.15.59:0/1311119269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 msgr2=0x55fbbc618210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20b4030740 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc618210 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f20a4009fd0 tx=0x7f20a40118d0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20aaffd640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f20940b2b10 0x7f20940b4fd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55fbbc61aa30 tx=0x7f209c00cf20 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20b4030740 1 -- 172.21.15.59:0/1311119269 shutdown_connections 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20b4030740 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 0x55fbbc6298f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20b4030740 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc626fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20b4030740 1 --2- 172.21.15.59:0/1311119269 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc618210 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20b4030740 1 -- 172.21.15.59:0/1311119269 >> 172.21.15.59:0/1311119269 conn(0x55fbbc614420 msgr2=0x55fbbc614800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20b4030740 1 -- 172.21.15.59:0/1311119269 shutdown_connections 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.357+0000 7f20b4030740 1 -- 172.21.15.59:0/1311119269 wait complete. 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20b4030740 1 -- start start 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20b4030740 1 Processor -- start 2024-05-08T05:32:30.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20b4030740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc624460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20b4030740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc6249a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20b4030740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 0x55fbbc624ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20b4030740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc62a130 con 0x55fbbc617e30 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20b4030740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc53a370 con 0x55fbbc618750 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20b4030740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fbbc62a2b0 con 0x55fbbc627500 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20b4030740 10 client.0 _ll_register_callbacks cb 0x55fbbc28ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc624460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc624460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47848/0 (socket says 172.21.15.59:47848) 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20ab7fe640 1 -- 172.21.15.59:0/285203483 learned_addr learned my addr 172.21.15.59:0/285203483 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20aa7fc640 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc6249a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.358+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 0x55fbbc624ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20ab7fe640 1 -- 172.21.15.59:0/285203483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 msgr2=0x55fbbc624ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 0x55fbbc624ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20ab7fe640 1 -- 172.21.15.59:0/285203483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 msgr2=0x55fbbc6249a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc6249a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20ab7fe640 1 -- 172.21.15.59:0/285203483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbbc35b620 con 0x55fbbc617e30 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20aa7fc640 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc6249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc624460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f20a401b040 tx=0x7f20a40047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2782c12854a3f963 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f208eff5640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f208eff5640 1 -- 172.21.15.59:0/285203483 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4004c40 con 0x55fbbc617e30 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f208eff5640 1 -- 172.21.15.59:0/285203483 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f20a4004de0 con 0x55fbbc617e30 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20b4030740 1 -- 172.21.15.59:0/285203483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a4004050 con 0x55fbbc617e30 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f208eff5640 1 -- 172.21.15.59:0/285203483 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20a4035050 con 0x55fbbc617e30 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20b4030740 10 client.15834 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.359+0000 7f20b4030740 1 -- 172.21.15.59:0/285203483 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fbbc635f30 con 0x55fbbc617e30 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.360+0000 7f208eff5640 1 -- 172.21.15.59:0/285203483 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a4004050 con 0x55fbbc617e30 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.360+0000 7f208eff5640 1 -- 172.21.15.59:0/285203483 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f20a400f600 con 0x55fbbc617e30 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.360+0000 7f20b4030740 10 client.15834 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:30.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.360+0000 7f20b4030740 10 client.15834 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.360+0000 7f20b4030740 20 client.15834 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.360+0000 7f20b4030740 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fbbc637330 0x55fbbc639720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.360+0000 7f20b4030740 4 client.15834 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.360+0000 7f20b4030740 1 -- 172.21.15.59:0/285203483 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fbbc58b1e0 con 0x55fbbc637330 2024-05-08T05:32:30.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.360+0000 7f20aa7fc640 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fbbc637330 0x55fbbc639720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.361+0000 7f208eff5640 10 client.15834 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.361+0000 7f20aa7fc640 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fbbc637330 0x55fbbc639720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9ba5896fb3307593 server_cookie=d7e794bd6a11653b in_seq=0 out_seq=0 2024-05-08T05:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f208eff5640 1 -- 172.21.15.59:0/285203483 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fbbc58b1e0 con 0x55fbbc637330 2024-05-08T05:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f208eff5640 10 client.15834 handle_command_reply: tid=0 2024-05-08T05:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 client.15834 shutdown 2024-05-08T05:32:30.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 -- 172.21.15.59:0/285203483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 msgr2=0x55fbbc624460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc624460 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f20a401b040 tx=0x7f20a40047d0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 -- 172.21.15.59:0/285203483 shutdown_connections 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fbbc637330 0x55fbbc639720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc627500 0x55fbbc624ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc6249a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 --2- 172.21.15.59:0/285203483 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc617e30 0x55fbbc624460 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f208eff5640 0 client.15834 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 -- 172.21.15.59:0/285203483 >> 172.21.15.59:0/285203483 conn(0x55fbbc614420 msgr2=0x55fbbc61a290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 -- 172.21.15.59:0/285203483 shutdown_connections 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 1 -- 172.21.15.59:0/285203483 wait complete. 2024-05-08T05:32:30.367 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.362+0000 7f20b4030740 20 client.15834 trim_cache size 0 max 16384 2024-05-08T05:32:30.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.379+0000 7f20b4030740 1 Processor -- start 2024-05-08T05:32:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.379+0000 7f20b4030740 1 -- start start 2024-05-08T05:32:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.379+0000 7f20b4030740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc617e30 0x55fbbc613400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.379+0000 7f20b4030740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc63ee90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.379+0000 7f20b4030740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc63f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.379+0000 7f20b4030740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1adbc0 con 0x55fbbc627500 2024-05-08T05:32:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.379+0000 7f20b4030740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1af790 con 0x55fbbc618750 2024-05-08T05:32:30.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.379+0000 7f20b4030740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1ae290 con 0x55fbbc617e30 2024-05-08T05:32:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc63f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc63f3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47858/0 (socket says 172.21.15.59:47858) 2024-05-08T05:32:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20aa7fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc63ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 -- 172.21.15.59:0/2426074016 learned_addr learned my addr 172.21.15.59:0/2426074016 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/2426074016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc617e30 0x55fbbc613400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 -- 172.21.15.59:0/2426074016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc617e30 msgr2=0x55fbbc613400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/2426074016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc617e30 0x55fbbc613400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 -- 172.21.15.59:0/2426074016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 msgr2=0x55fbbc63ee90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/2426074016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc63ee90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 -- 172.21.15.59:0/2426074016 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fbbc618580 con 0x55fbbc627500 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.380+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/2426074016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc63f3d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f20a400fa50 tx=0x7f20a40118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b705a4734acb6cd0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.381+0000 7f2090ff9640 1 -- 172.21.15.59:0/2426074016 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4009c20 con 0x55fbbc627500 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.381+0000 7f2090ff9640 1 -- 172.21.15.59:0/2426074016 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a402c070 con 0x55fbbc627500 2024-05-08T05:32:30.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.381+0000 7f2090ff9640 1 -- 172.21.15.59:0/2426074016 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4005040 con 0x55fbbc627500 2024-05-08T05:32:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 -- 172.21.15.59:0/2426074016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 msgr2=0x55fbbc63f3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 --2- 172.21.15.59:0/2426074016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc63f3d0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f20a400fa50 tx=0x7f20a40118d0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 -- 172.21.15.59:0/2426074016 shutdown_connections 2024-05-08T05:32:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 --2- 172.21.15.59:0/2426074016 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc63f3d0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 --2- 172.21.15.59:0/2426074016 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc618750 0x55fbbc63ee90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 --2- 172.21.15.59:0/2426074016 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc617e30 0x55fbbc613400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 -- 172.21.15.59:0/2426074016 >> 172.21.15.59:0/2426074016 conn(0x55fbbc614420 msgr2=0x55fbbc612cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 -- 172.21.15.59:0/2426074016 shutdown_connections 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 -- 172.21.15.59:0/2426074016 wait complete. 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 -- start start 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.382+0000 7f20b4030740 1 Processor -- start 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20b4030740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc617e30 0x55fbbc62f500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20b4030740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc618750 0x55fbbc62fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20b4030740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc62c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20b4030740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1af030 con 0x55fbbc627500 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20b4030740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1ae960 con 0x55fbbc617e30 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20b4030740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f20ac1abb80 con 0x55fbbc618750 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20b4030740 10 client.0 _ll_register_callbacks cb 0x7f20a40093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc62c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc62c4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47872/0 (socket says 172.21.15.59:47872) 2024-05-08T05:32:30.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20ab7fe640 1 -- 172.21.15.59:0/2709828515 learned_addr learned my addr 172.21.15.59:0/2709828515 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc617e30 0x55fbbc62f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20aa7fc640 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc618750 0x55fbbc62fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20ab7fe640 1 -- 172.21.15.59:0/2709828515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc618750 msgr2=0x55fbbc62fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc618750 0x55fbbc62fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20ab7fe640 1 -- 172.21.15.59:0/2709828515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc617e30 msgr2=0x55fbbc62f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc617e30 0x55fbbc62f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.383+0000 7f20ab7fe640 1 -- 172.21.15.59:0/2709828515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fbbc62ca10 con 0x55fbbc627500 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.384+0000 7f20ab7fe640 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc62c4d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20a4003de0 tx=0x7f20a4009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6dfe90b5103a31 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.384+0000 7f208eff5640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.384+0000 7f208eff5640 1 -- 172.21.15.59:0/2709828515 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20a4004c80 con 0x55fbbc627500 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.384+0000 7f208eff5640 1 -- 172.21.15.59:0/2709828515 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f20a400f4e0 con 0x55fbbc627500 2024-05-08T05:32:30.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.384+0000 7f20b4030740 1 -- 172.21.15.59:0/2709828515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f20a402c070 con 0x55fbbc627500 2024-05-08T05:32:30.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.384+0000 7f208eff5640 1 -- 172.21.15.59:0/2709828515 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f20a4004e20 con 0x55fbbc627500 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.384+0000 7f20b4030740 10 client.15840 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.385+0000 7f20b4030740 1 -- 172.21.15.59:0/2709828515 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55fbbc6a2260 con 0x55fbbc627500 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.385+0000 7f208eff5640 1 -- 172.21.15.59:0/2709828515 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20a400f6e0 con 0x55fbbc627500 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.386+0000 7f208eff5640 1 -- 172.21.15.59:0/2709828515 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f20a4005040 con 0x55fbbc627500 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.386+0000 7f20b4030740 10 client.15840 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.386+0000 7f20b4030740 10 client.15840 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.386+0000 7f20b4030740 20 client.15840 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.386+0000 7f20b4030740 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fbbc6a2950 0x55fbbc6a4d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.386+0000 7f20b4030740 4 client.15840 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.386+0000 7f20b4030740 1 -- 172.21.15.59:0/2709828515 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x7f20a402c070 con 0x55fbbc6a2950 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.386+0000 7f20aa7fc640 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fbbc6a2950 0x55fbbc6a4d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.387+0000 7f208eff5640 10 client.15840 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:30.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:30.387+0000 7f20aa7fc640 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fbbc6a2950 0x55fbbc6a4d40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=857d0ccd93b5305f server_cookie=456cc7133b51924d in_seq=0 out_seq=0 2024-05-08T05:32:30.410 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] godown 2024-05-08T05:32:30.430 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] resvtest 2024-05-08T05:32:30.501 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] writemod 2024-05-08T05:32:30.501 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] writev_on_pagefault 2024-05-08T05:32:30.597 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] makeextents 2024-05-08T05:32:30.620 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] itrash 2024-05-08T05:32:30.652 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] rename 2024-05-08T05:32:30.658 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] multi_open_unlink 2024-05-08T05:32:30.786 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] unwritten_sync 2024-05-08T05:32:30.789 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] genhashnames 2024-05-08T05:32:30.793 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_holes 2024-05-08T05:32:30.829 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mmap_writev 2024-05-08T05:32:30.948 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_truncate_cmtime 2024-05-08T05:32:30.969 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] dirhash_collide 2024-05-08T05:32:30.979 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_rename_overwrite 2024-05-08T05:32:30.984 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] holetest 2024-05-08T05:32:31.112 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_truncate_self 2024-05-08T05:32:31.134 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] af_unix 2024-05-08T05:32:31.173 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mmap_stale_pmd 2024-05-08T05:32:31.218 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mmap_cow_race 2024-05-08T05:32:31.254 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mmap_fallocate 2024-05-08T05:32:31.286 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fsync-err 2024-05-08T05:32:31.331 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mmap_write_ro 2024-05-08T05:32:31.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:31 smithi059 ceph-mon[30304]: pgmap v91: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 816 KiB/s wr, 656 op/s 2024-05-08T05:32:31.398 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_ext4_dax_journal_corruption 2024-05-08T05:32:31.415 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_ext4_dax_inline_corruption 2024-05-08T05:32:31.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:31 smithi143 ceph-mon[34337]: pgmap v91: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 816 KiB/s wr, 656 op/s 2024-05-08T05:32:31.496 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_ofd_locks 2024-05-08T05:32:31.515 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mmap_collision 2024-05-08T05:32:31.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:31 smithi096 ceph-mon[34253]: pgmap v91: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 555 KiB/s rd, 816 KiB/s wr, 656 op/s 2024-05-08T05:32:31.571 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] mmap-write-concurrent 2024-05-08T05:32:31.573 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_get_file_time 2024-05-08T05:32:31.760 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_create_short_dirs 2024-05-08T05:32:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.815+0000 7f208eff5640 1 -- 172.21.15.59:0/2709828515 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x7f20a402c070 con 0x55fbbc6a2950 2024-05-08T05:32:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.815+0000 7f208eff5640 10 client.15840 handle_command_reply: tid=0 2024-05-08T05:32:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 client.15840 shutdown 2024-05-08T05:32:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 -- 172.21.15.59:0/2709828515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 msgr2=0x55fbbc62c4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc62c4d0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f20a4003de0 tx=0x7f20a4009610 comp rx=0 tx=0).stop 2024-05-08T05:32:31.817 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 -- 172.21.15.59:0/2709828515 shutdown_connections 2024-05-08T05:32:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fbbc6a2950 0x55fbbc6a4d40 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fbbc627500 0x55fbbc62c4d0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fbbc618750 0x55fbbc62fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 --2- 172.21.15.59:0/2709828515 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fbbc617e30 0x55fbbc62f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 -- 172.21.15.59:0/2709828515 >> 172.21.15.59:0/2709828515 conn(0x55fbbc614420 msgr2=0x55fbbc613580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f208eff5640 0 client.15840 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 -- 172.21.15.59:0/2709828515 shutdown_connections 2024-05-08T05:32:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 1 -- 172.21.15.59:0/2709828515 wait complete. 2024-05-08T05:32:31.818 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.816+0000 7f20b4030740 20 client.15840 trim_cache size 0 max 16384 2024-05-08T05:32:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:32:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:32:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:32:31.819 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:32:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 -- 172.21.15.59:0/911340053 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f20940b2b10 msgr2=0x7f20940b4fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f20940b2b10 0x7f20940b4fd0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x55fbbc61aa30 tx=0x7f209c00cf20 comp rx=0 tx=0).stop 2024-05-08T05:32:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 -- 172.21.15.59:0/911340053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20ac1018b0 msgr2=0x7f20ac196ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20ac1018b0 0x7f20ac196ec0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f209c00e9f0 tx=0x7f209c00eeb0 comp rx=0 tx=0).stop 2024-05-08T05:32:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 -- 172.21.15.59:0/911340053 shutdown_connections 2024-05-08T05:32:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f20940b2b10 0x7f20940b4fd0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:31.820 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f20ac1078c0 0x7f20ac19b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f20ac102270 0x7f20ac197400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 --2- 172.21.15.59:0/911340053 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f20ac1018b0 0x7f20ac196ec0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.818+0000 7f20b131c640 1 -- 172.21.15.59:0/911340053 >> 172.21.15.59:0/911340053 conn(0x7f20ac078a00 msgr2=0x7f20ac1043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.819+0000 7f20b131c640 1 -- 172.21.15.59:0/911340053 shutdown_connections 2024-05-08T05:32:31.821 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:31.819+0000 7f20b131c640 1 -- 172.21.15.59:0/911340053 wait complete. 2024-05-08T05:32:31.861 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_create_long_dirs 2024-05-08T05:32:31.887 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_enospc 2024-05-08T05:32:31.896 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mmap_writev_overlap 2024-05-08T05:32:31.983 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] checkpoint_journal 2024-05-08T05:32:32.035 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] mmap-rw-fault 2024-05-08T05:32:32.073 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] allocstale 2024-05-08T05:32:32.103 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mmap_cow_memory_failure 2024-05-08T05:32:32.146 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fake-dump-rootino 2024-05-08T05:32:32.234 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] dio-buf-fault 2024-05-08T05:32:32.249 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] rewinddir-test 2024-05-08T05:32:32.283 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] readdir-while-renames 2024-05-08T05:32:32.309 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] xfsctl 2024-05-08T05:32:32.393 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] bstat 2024-05-08T05:32:32.443 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mtab 2024-05-08T05:32:32.451 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] getdevicesize 2024-05-08T05:32:32.551 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] preallo_rw_pattern_reader 2024-05-08T05:32:32.641 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] preallo_rw_pattern_writer 2024-05-08T05:32:32.696 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] ftrunc 2024-05-08T05:32:32.725 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] trunc 2024-05-08T05:32:33.039 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fs_perms 2024-05-08T05:32:33.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:33 smithi059 ceph-mon[30304]: pgmap v92: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 429 KiB/s wr, 32 op/s 2024-05-08T05:32:33.344 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] testx 2024-05-08T05:32:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:33 smithi143 ceph-mon[34337]: pgmap v92: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 429 KiB/s wr, 32 op/s 2024-05-08T05:32:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:33 smithi096 ceph-mon[34253]: pgmap v92: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 429 KiB/s wr, 32 op/s 2024-05-08T05:32:33.974 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] looptest 2024-05-08T05:32:33.990 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] locktest 2024-05-08T05:32:34.048 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] unwritten_mmap 2024-05-08T05:32:34.118 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] bulkstat_unlink_test 2024-05-08T05:32:34.188 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] deduperace 2024-05-08T05:32:34.218 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] bulkstat_unlink_test_modified 2024-05-08T05:32:34.320 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_dir_offset 2024-05-08T05:32:34.328 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_futimens 2024-05-08T05:32:34.400 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_immutable 2024-05-08T05:32:34.411 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] stale_handle 2024-05-08T05:32:34.472 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] pwrite_mmap_blocked 2024-05-08T05:32:34.472 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_dir_offset2 2024-05-08T05:32:34.597 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] seek_sanity_test 2024-05-08T05:32:34.742 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] seek_copy_test 2024-05-08T05:32:34.743 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_readdir_1 2024-05-08T05:32:34.912 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_readdir_2 2024-05-08T05:32:34.949 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fsync-tester 2024-05-08T05:32:35.144 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] nsexec 2024-05-08T05:32:35.177 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] cloner 2024-05-08T05:32:35.201 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] renameat2 2024-05-08T05:32:35.330 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_getcwd 2024-05-08T05:32:35.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:35 smithi059 ceph-mon[30304]: pgmap v93: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 429 KiB/s wr, 32 op/s 2024-05-08T05:32:35.362 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] e4compact 2024-05-08T05:32:35.365 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] test-nextquota 2024-05-08T05:32:35.366 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] punch-alternating 2024-05-08T05:32:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:35 smithi143 ceph-mon[34337]: pgmap v93: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 429 KiB/s wr, 32 op/s 2024-05-08T05:32:35.497 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] attr-list-by-handle-cursor-test 2024-05-08T05:32:35.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:35 smithi096 ceph-mon[34253]: pgmap v93: 129 pgs: 129 active+clean; 28 MiB data, 514 MiB used, 1.0 TiB / 1.0 TiB avail; 430 KiB/s rd, 429 KiB/s wr, 32 op/s 2024-05-08T05:32:35.547 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] listxattr 2024-05-08T05:32:35.547 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] dio-interleaved 2024-05-08T05:32:35.565 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_dir_type 2024-05-08T05:32:35.668 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] dio-invalidate-cache 2024-05-08T05:32:35.704 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] stat_test 2024-05-08T05:32:35.712 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_encrypted_d_revalidate 2024-05-08T05:32:35.729 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] attr_replace_test 2024-05-08T05:32:35.879 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] swapon 2024-05-08T05:32:35.889 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] mkswap 2024-05-08T05:32:35.893 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_attr_corruption 2024-05-08T05:32:36.055 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_open_tmpfiles 2024-05-08T05:32:36.087 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fscrypt-crypt-util 2024-05-08T05:32:36.092 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] bulkstat_null_ocount 2024-05-08T05:32:36.093 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] splice-test 2024-05-08T05:32:36.239 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] chprojid_fail 2024-05-08T05:32:36.260 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] detached_mounts_propagation 2024-05-08T05:32:36.279 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] ext4_resize 2024-05-08T05:32:36.415 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_readdir_3 2024-05-08T05:32:36.422 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] splice2pipe 2024-05-08T05:32:36.446 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] uuid_ioctl 2024-05-08T05:32:36.582 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_snapshot_deleted_subvolume 2024-05-08T05:32:36.602 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fiemap-fault 2024-05-08T05:32:36.619 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fiemap-tester 2024-05-08T05:32:36.755 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_stripealign 2024-05-08T05:32:36.767 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] open_by_handle 2024-05-08T05:32:36.854 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_mmap_dio 2024-05-08T05:32:36.897 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] xfsfind 2024-05-08T05:32:36.924 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] t_reflink_read_race 2024-05-08T05:32:37.019 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] fssum 2024-05-08T05:32:37.058 INFO:tasks.workunit.client.0.smithi059.stdout:Building log-writes 2024-05-08T05:32:37.065 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] replay-log 2024-05-08T05:32:37.100 INFO:tasks.workunit.client.0.smithi059.stdout:Building perf 2024-05-08T05:32:37.104 INFO:tasks.workunit.client.0.smithi059.stdout:Building aio-dio-regress 2024-05-08T05:32:37.112 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aiocp 2024-05-08T05:32:37.128 INFO:tasks.workunit.client.0.smithi059.stdout:Building vfs 2024-05-08T05:32:37.134 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] vfstest 2024-05-08T05:32:37.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:37 smithi059 ceph-mon[30304]: pgmap v94: 129 pgs: 129 active+clean; 37 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.6 MiB/s wr, 334 op/s 2024-05-08T05:32:37.342 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-append-write-fallocate-race 2024-05-08T05:32:37.401 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] mount-idmapped 2024-05-08T05:32:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:37 smithi143 ceph-mon[34337]: pgmap v94: 129 pgs: 129 active+clean; 37 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.6 MiB/s wr, 334 op/s 2024-05-08T05:32:37.507 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-append-write-read-race 2024-05-08T05:32:37.526 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-cow-race 2024-05-08T05:32:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:37 smithi096 ceph-mon[34253]: pgmap v94: 129 pgs: 129 active+clean; 37 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 749 KiB/s rd, 1.6 MiB/s wr, 334 op/s 2024-05-08T05:32:37.686 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-cycle-write 2024-05-08T05:32:37.705 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-eof-race 2024-05-08T05:32:37.840 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-extend-stat 2024-05-08T05:32:37.879 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-fcntl-race 2024-05-08T05:32:37.903 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-hole-filling-race 2024-05-08T05:32:38.012 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-invalidate-failure 2024-05-08T05:32:38.036 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-invalidate-readahead 2024-05-08T05:32:38.077 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aiodio_sparse2 2024-05-08T05:32:38.184 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-subblock-eof-read 2024-05-08T05:32:38.215 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-dio-write-verify 2024-05-08T05:32:38.306 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-free-ring-with-bogus-nr-pages 2024-05-08T05:32:38.329 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-io-setup-with-nonwritable-context-pointer 2024-05-08T05:32:38.433 INFO:tasks.workunit.client.0.smithi059.stdout: [CC] aio-last-ref-held-by-io 2024-05-08T05:32:39.216 INFO:tasks.workunit.client.0.smithi059.stderr:+ cd .. 2024-05-08T05:32:39.216 INFO:tasks.workunit.client.0.smithi059.stderr:+ cp xfstests-dev/ltp/fsx . 2024-05-08T05:32:39.218 INFO:tasks.workunit.client.0.smithi059.stderr:+ OPTIONS=-z 2024-05-08T05:32:39.219 INFO:tasks.workunit.client.0.smithi059.stderr:+ ./fsx -z 1MB -N 50000 -p 10000 -l 1048576 2024-05-08T05:32:39.219 INFO:tasks.workunit.client.0.smithi059.stdout:Seed set to 1 2024-05-08T05:32:39.257 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-05-08T05:32:39.257 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-05-08T05:32:39.260 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-08T05:32:39.260 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-08T05:32:39.260 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support clone range, disabling! 2024-05-08T05:32:39.262 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-08T05:32:39.264 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero size read 2024-05-08T05:32:39.264 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero size read 2024-05-08T05:32:39.265 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support exchange range, disabling! 2024-05-08T05:32:39.265 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0x81d54 2024-05-08T05:32:39.271 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xb86d7 2024-05-08T05:32:39.280 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xe627e 2024-05-08T05:32:39.326 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0xf3194 2024-05-08T05:32:39.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:39 smithi059 ceph-mon[30304]: pgmap v95: 129 pgs: 129 active+clean; 37 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 317 op/s 2024-05-08T05:32:39.356 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xf3ceb 2024-05-08T05:32:39.362 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0xf47e4 2024-05-08T05:32:39.470 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xfaccc 2024-05-08T05:32:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:39 smithi143 ceph-mon[34337]: pgmap v95: 129 pgs: 129 active+clean; 37 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 317 op/s 2024-05-08T05:32:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:39 smithi096 ceph-mon[34253]: pgmap v95: 129 pgs: 129 active+clean; 37 MiB data, 563 MiB used, 1.0 TiB / 1.0 TiB avail; 420 KiB/s rd, 1.3 MiB/s wr, 317 op/s 2024-05-08T05:32:40.000 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xfe423 2024-05-08T05:32:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:41 smithi059 ceph-mon[30304]: pgmap v96: 129 pgs: 129 active+clean; 37 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 414 op/s 2024-05-08T05:32:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:41 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:32:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:41 smithi143 ceph-mon[34337]: pgmap v96: 129 pgs: 129 active+clean; 37 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 414 op/s 2024-05-08T05:32:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:41 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:32:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:41 smithi096 ceph-mon[34253]: pgmap v96: 129 pgs: 129 active+clean; 37 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 586 KiB/s rd, 1.6 MiB/s wr, 414 op/s 2024-05-08T05:32:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:41 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:32:42.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:42 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:32:42.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:42 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:32:42.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:42 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:32:42.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:42 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:32:42.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:42 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:32:42.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:42 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:32:42.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:42 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:32:42.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:42 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:32:42.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:42 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:32:43.077 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xfe492 2024-05-08T05:32:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:43 smithi059 ceph-mon[30304]: pgmap v97: 129 pgs: 129 active+clean; 37 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.5 MiB/s wr, 398 op/s 2024-05-08T05:32:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:43 smithi143 ceph-mon[34337]: pgmap v97: 129 pgs: 129 active+clean; 37 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.5 MiB/s wr, 398 op/s 2024-05-08T05:32:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:43 smithi096 ceph-mon[34253]: pgmap v97: 129 pgs: 129 active+clean; 37 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.5 MiB/s wr, 398 op/s 2024-05-08T05:32:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.922+0000 7f519f4d1640 1 -- 172.21.15.59:0/2460154225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5198103360 msgr2=0x7f519810f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.922+0000 7f519f4d1640 1 --2- 172.21.15.59:0/2460154225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5198103360 0x7f519810f870 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f5190009cd0 tx=0x7f519002a8e0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.922+0000 7f519f4d1640 1 -- 172.21.15.59:0/2460154225 shutdown_connections 2024-05-08T05:32:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.922+0000 7f519f4d1640 1 --2- 172.21.15.59:0/2460154225 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5198103360 0x7f519810f870 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.922+0000 7f519f4d1640 1 --2- 172.21.15.59:0/2460154225 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f51981029a0 0x7f5198102e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.926 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.922+0000 7f519f4d1640 1 --2- 172.21.15.59:0/2460154225 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51981089b0 0x7f5198108d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.922+0000 7f519f4d1640 1 -- 172.21.15.59:0/2460154225 >> 172.21.15.59:0/2460154225 conn(0x7f51980fe470 msgr2=0x7f5198100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.922+0000 7f519f4d1640 1 -- 172.21.15.59:0/2460154225 shutdown_connections 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.922+0000 7f519f4d1640 1 -- 172.21.15.59:0/2460154225 wait complete. 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519f4d1640 1 Processor -- start 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519f4d1640 1 -- start start 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519f4d1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51981029a0 0x7f51981990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519f4d1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5198103360 0x7f5198199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519f4d1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51981089b0 0x7f519819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519f4d1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5198112c70 con 0x7f51981089b0 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519f4d1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5198112af0 con 0x7f51981029a0 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519f4d1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5198112df0 con 0x7f5198103360 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519da6e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51981089b0 0x7f519819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519da6e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51981089b0 0x7f519819d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38386/0 (socket says 172.21.15.59:38386) 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519da6e640 1 -- 172.21.15.59:0/1114412436 learned_addr learned my addr 172.21.15.59:0/1114412436 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:44.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519ca6c640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5198103360 0x7f5198199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.923+0000 7f519d26d640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51981029a0 0x7f51981990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f519da6e640 1 -- 172.21.15.59:0/1114412436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5198103360 msgr2=0x7f5198199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f519da6e640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5198103360 0x7f5198199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f519da6e640 1 -- 172.21.15.59:0/1114412436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51981029a0 msgr2=0x7f51981990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f519da6e640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51981029a0 0x7f51981990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f519da6e640 1 -- 172.21.15.59:0/1114412436 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f519819e0e0 con 0x7f51981089b0 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f519d26d640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51981029a0 0x7f51981990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f519da6e640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51981089b0 0x7f519819d9b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5190033040 tx=0x7f5190009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f518e7fc640 1 -- 172.21.15.59:0/1114412436 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190004040 con 0x7f51981089b0 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f518e7fc640 1 -- 172.21.15.59:0/1114412436 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5190004690 con 0x7f51981089b0 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f519f4d1640 1 -- 172.21.15.59:0/1114412436 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f519819e370 con 0x7f51981089b0 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f518e7fc640 1 -- 172.21.15.59:0/1114412436 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5190030d90 con 0x7f51981089b0 2024-05-08T05:32:44.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.924+0000 7f519f4d1640 1 -- 172.21.15.59:0/1114412436 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f51981a5c60 con 0x7f51981089b0 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.927+0000 7f518e7fc640 1 -- 172.21.15.59:0/1114412436 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f51900041e0 con 0x7f51981089b0 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f51a21ea740 1 Processor -- start 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f51a21ea740 1 -- start start 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c12f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f518e7fc640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f51840b2a90 0x7f51840b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f51a21ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5610a53265d0 con 0x5610a53cc0f0 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f51a21ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5610a52df890 con 0x5610a53c1830 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f51a21ea740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5610a5322590 con 0x5610a53c0f10 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f519ca6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c12f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f519ca6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c12f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:54788/0 (socket says 172.21.15.59:54788) 2024-05-08T05:32:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f519ca6c640 1 -- 172.21.15.59:0/2482663156 learned_addr learned my addr 172.21.15.59:0/2482663156 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.928+0000 7f519da6e640 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519ca6c640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f51840b2a90 0x7f51840b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519d26d640 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f518e7fc640 1 -- 172.21.15.59:0/1114412436 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f519002e030 con 0x7f51981089b0 2024-05-08T05:32:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519ca6c640 1 -- 172.21.15.59:0/2482663156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 msgr2=0x5610a53cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519ca6c640 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519ca6c640 1 -- 172.21.15.59:0/2482663156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 msgr2=0x5610a53ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:44.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519ca6c640 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519ca6c640 1 -- 172.21.15.59:0/2482663156 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a5038210 con 0x5610a53c0f10 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519d26d640 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519ca6c640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f51840b2a90 0x7f51840b4f50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f518800f8c0 tx=0x7f5188018000 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f519ca6c640 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c12f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5188011ae0 tx=0x7f518800f3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b312925a444c4f1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f51837fe640 1 -- 172.21.15.59:0/2482663156 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5188011050 con 0x5610a53c0f10 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f51837fe640 1 -- 172.21.15.59:0/2482663156 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51880111f0 con 0x5610a53c0f10 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.929+0000 7f51837fe640 1 -- 172.21.15.59:0/2482663156 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51880035f0 con 0x5610a53c0f10 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.930+0000 7f51a21ea740 1 -- 172.21.15.59:0/2482663156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 msgr2=0x5610a53c12f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.930+0000 7f51a21ea740 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c12f0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f5188011ae0 tx=0x7f518800f3c0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.930+0000 7f51a21ea740 1 -- 172.21.15.59:0/2482663156 shutdown_connections 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.930+0000 7f51a21ea740 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.930+0000 7f51a21ea740 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.930+0000 7f51a21ea740 1 --2- 172.21.15.59:0/2482663156 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c12f0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.930+0000 7f51a21ea740 1 -- 172.21.15.59:0/2482663156 >> 172.21.15.59:0/2482663156 conn(0x5610a53bd420 msgr2=0x5610a53bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.930+0000 7f51a21ea740 1 -- 172.21.15.59:0/2482663156 shutdown_connections 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.930+0000 7f51a21ea740 1 -- 172.21.15.59:0/2482663156 wait complete. 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.931+0000 7f51a21ea740 1 -- start start 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.931+0000 7f51a21ea740 1 Processor -- start 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.931+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c9070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.931+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53c95b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.931+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53c9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.931+0000 7f51a21ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5610a53cecc0 con 0x5610a53cc0f0 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.931+0000 7f51a21ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5610a52e3370 con 0x5610a53c1830 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.931+0000 7f51a21ea740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5610a53cee40 con 0x5610a53c0f10 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.931+0000 7f51a21ea740 10 client.0 _ll_register_callbacks cb 0x5610a5037a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519da6e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53c9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519d26d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519ca6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53c95b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519da6e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53c9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38404/0 (socket says 172.21.15.59:38404) 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519da6e640 1 -- 172.21.15.59:0/1826421725 learned_addr learned my addr 172.21.15.59:0/1826421725 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519ca6c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53c95b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:58500/0 (socket says 172.21.15.59:58500) 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519d26d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:54804/0 (socket says 172.21.15.59:54804) 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519ca6c640 1 -- 172.21.15.59:0/1826421725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 msgr2=0x5610a53c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519ca6c640 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c9070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519ca6c640 1 -- 172.21.15.59:0/1826421725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 msgr2=0x5610a53c9af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519ca6c640 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53c9af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519ca6c640 1 -- 172.21.15.59:0/1826421725 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a5104620 con 0x5610a53c1830 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519da6e640 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519d26d640 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f519ca6c640 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53c95b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5188006010 tx=0x7f51880113a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11cf003512f1823d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.932+0000 7f5180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f5180ff9640 1 -- 172.21.15.59:0/1826421725 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f518802a070 con 0x5610a53c1830 2024-05-08T05:32:44.938 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f51a21ea740 1 -- 172.21.15.59:0/1826421725 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f51880111f0 con 0x5610a53c1830 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f5180ff9640 1 -- 172.21.15.59:0/1826421725 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f5188005050 con 0x5610a53c1830 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f5180ff9640 1 -- 172.21.15.59:0/1826421725 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5188039050 con 0x5610a53c1830 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f51a21ea740 10 client.25475 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f51a21ea740 1 -- 172.21.15.59:0/1826421725 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610a53dabc0 con 0x5610a53c1830 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f5180ff9640 1 -- 172.21.15.59:0/1826421725 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f51880111f0 con 0x5610a53c1830 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f5180ff9640 1 -- 172.21.15.59:0/1826421725 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f5188003890 con 0x5610a53c1830 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f51a21ea740 10 client.25475 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f51a21ea740 10 client.25475 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f51a21ea740 20 client.25475 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.933+0000 7f51a21ea740 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5610a53dc080 0x5610a53de470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.934+0000 7f51a21ea740 4 client.25475 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.934+0000 7f51a21ea740 1 -- 172.21.15.59:0/1826421725 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610a53341e0 con 0x5610a53dc080 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.934+0000 7f519d26d640 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5610a53dc080 0x5610a53de470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.939 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.934+0000 7f5180ff9640 10 client.25475 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:44.940 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.934+0000 7f519d26d640 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5610a53dc080 0x5610a53de470 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=16524c0a10b37170 server_cookie=136f1160f3fffea0 in_seq=0 out_seq=0 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f5180ff9640 1 -- 172.21.15.59:0/1826421725 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5610a53341e0 con 0x5610a53dc080 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f5180ff9640 10 client.25475 handle_command_reply: tid=0 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 client.25475 shutdown 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 -- 172.21.15.59:0/1826421725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 msgr2=0x5610a53c95b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53c95b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5188006010 tx=0x7f51880113a0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 -- 172.21.15.59:0/1826421725 shutdown_connections 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5610a53dc080 0x5610a53de470 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53c9af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c1830 0x5610a53c95b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 --2- 172.21.15.59:0/1826421725 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c0f10 0x5610a53c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.941 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 -- 172.21.15.59:0/1826421725 >> 172.21.15.59:0/1826421725 conn(0x5610a53bd420 msgr2=0x5610a53c31a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f5180ff9640 0 client.25475 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 -- 172.21.15.59:0/1826421725 shutdown_connections 2024-05-08T05:32:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 1 -- 172.21.15.59:0/1826421725 wait complete. 2024-05-08T05:32:44.942 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.935+0000 7f51a21ea740 20 client.25475 trim_cache size 0 max 16384 2024-05-08T05:32:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f51a21ea740 1 Processor -- start 2024-05-08T05:32:44.960 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f51a21ea740 1 -- start start 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c0f10 0x5610a53c9910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 0x5610a53e3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53e4210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f51a21ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51981afbe0 con 0x5610a53cc0f0 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f51a21ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b17b0 con 0x5610a53c0f10 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f51a21ea740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b02b0 con 0x5610a53c1830 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519da6e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53e4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519ca6c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 0x5610a53e3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519da6e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53e4210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38414/0 (socket says 172.21.15.59:38414) 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519d26d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c0f10 0x5610a53c9910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519da6e640 1 -- 172.21.15.59:0/867924549 learned_addr learned my addr 172.21.15.59:0/867924549 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519da6e640 1 -- 172.21.15.59:0/867924549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 msgr2=0x5610a53e3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:44.961 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519da6e640 1 --2- 172.21.15.59:0/867924549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 0x5610a53e3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519da6e640 1 -- 172.21.15.59:0/867924549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c0f10 msgr2=0x5610a53c9910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519da6e640 1 --2- 172.21.15.59:0/867924549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c0f10 0x5610a53c9910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.953+0000 7f519da6e640 1 -- 172.21.15.59:0/867924549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610a53ce4e0 con 0x5610a53cc0f0 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.954+0000 7f519da6e640 1 --2- 172.21.15.59:0/867924549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53e4210 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f51900c66f0 tx=0x7f51900d1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce740f889182bb37 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.954+0000 7f5182ffd640 1 -- 172.21.15.59:0/867924549 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51900ca440 con 0x5610a53cc0f0 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.954+0000 7f5182ffd640 1 -- 172.21.15.59:0/867924549 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5190036060 con 0x5610a53cc0f0 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.954+0000 7f5182ffd640 1 -- 172.21.15.59:0/867924549 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51900c9de0 con 0x5610a53cc0f0 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 -- 172.21.15.59:0/867924549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 msgr2=0x5610a53e4210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 --2- 172.21.15.59:0/867924549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53e4210 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f51900c66f0 tx=0x7f51900d1040 comp rx=0 tx=0).stop 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 -- 172.21.15.59:0/867924549 shutdown_connections 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 --2- 172.21.15.59:0/867924549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53cc0f0 0x5610a53e4210 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 --2- 172.21.15.59:0/867924549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 0x5610a53e3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.962 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 --2- 172.21.15.59:0/867924549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53c0f10 0x5610a53c9910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 -- 172.21.15.59:0/867924549 >> 172.21.15.59:0/867924549 conn(0x5610a53bd420 msgr2=0x5610a53c43d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 -- 172.21.15.59:0/867924549 shutdown_connections 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 -- 172.21.15.59:0/867924549 wait complete. 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 -- start start 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 Processor -- start 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53c0f10 0x5610a53d4090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 0x5610a53d45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53cc0f0 0x5610a53c5690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b1050 con 0x5610a53c0f10 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f51981b0980 con 0x5610a53cc0f0 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.955+0000 7f51a21ea740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f51981adba0 con 0x5610a53c1830 2024-05-08T05:32:44.963 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f51a21ea740 10 client.0 _ll_register_callbacks cb 0x7f51900305a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519d26d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53c0f10 0x5610a53d4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519d26d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53c0f10 0x5610a53d4090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:38420/0 (socket says 172.21.15.59:38420) 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519d26d640 1 -- 172.21.15.59:0/3694027630 learned_addr learned my addr 172.21.15.59:0/3694027630 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519ca6c640 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 0x5610a53d45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519da6e640 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53cc0f0 0x5610a53c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519d26d640 1 -- 172.21.15.59:0/3694027630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 msgr2=0x5610a53d45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519d26d640 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 0x5610a53d45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519d26d640 1 -- 172.21.15.59:0/3694027630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53cc0f0 msgr2=0x5610a53c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519d26d640 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53cc0f0 0x5610a53c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519d26d640 1 -- 172.21.15.59:0/3694027630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610a53c5dd0 con 0x5610a53c0f10 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f519d26d640 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53c0f10 0x5610a53d4090 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5610a53dec70 tx=0x7f5194012e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7a107cb7b8c2cdf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f5180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f5180ff9640 1 -- 172.21.15.59:0/3694027630 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5194020b50 con 0x5610a53c0f10 2024-05-08T05:32:44.964 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f5180ff9640 1 -- 172.21.15.59:0/3694027630 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f5194020cf0 con 0x5610a53c0f10 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.956+0000 7f51a21ea740 1 -- 172.21.15.59:0/3694027630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5190036060 con 0x5610a53c0f10 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f5180ff9640 1 -- 172.21.15.59:0/3694027630 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f5194032050 con 0x5610a53c0f10 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f51a21ea740 10 client.15858 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f51a21ea740 1 -- 172.21.15.59:0/3694027630 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x5610a54462f0 con 0x5610a53c0f10 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f5180ff9640 1 -- 172.21.15.59:0/3694027630 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5190036060 con 0x5610a53c0f10 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f5180ff9640 1 -- 172.21.15.59:0/3694027630 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f5194004b40 con 0x5610a53c0f10 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f51a21ea740 10 client.15858 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f51a21ea740 10 client.15858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f51a21ea740 20 client.15858 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f51a21ea740 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5610a5446b10 0x5610a5448f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f51a21ea740 4 client.15858 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f51a21ea740 1 -- 172.21.15.59:0/3694027630 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5610a532e050 con 0x5610a5446b10 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.957+0000 7f519ca6c640 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5610a5446b10 0x5610a5448f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:44.965 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.958+0000 7f5180ff9640 10 client.15858 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:44.966 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:44.958+0000 7f519ca6c640 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5610a5446b10 0x5610a5448f30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2184a2b48fd2b7b server_cookie=6434edb48cf1c2b5 in_seq=0 out_seq=0 2024-05-08T05:32:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f5180ff9640 1 -- 172.21.15.59:0/3694027630 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5610a532e050 con 0x5610a5446b10 2024-05-08T05:32:45.079 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f5180ff9640 10 client.15858 handle_command_reply: tid=0 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 client.15858 shutdown 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 -- 172.21.15.59:0/3694027630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53c0f10 msgr2=0x5610a53d4090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53c0f10 0x5610a53d4090 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x5610a53dec70 tx=0x7f5194012e50 comp rx=0 tx=0).stop 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 -- 172.21.15.59:0/3694027630 shutdown_connections 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5610a5446b10 0x5610a5448f30 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5610a53cc0f0 0x5610a53c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5610a53c1830 0x5610a53d45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f5180ff9640 0 client.15858 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 --2- 172.21.15.59:0/3694027630 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5610a53c0f10 0x5610a53d4090 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 -- 172.21.15.59:0/3694027630 >> 172.21.15.59:0/3694027630 conn(0x5610a53bd420 msgr2=0x5610a53bbcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 -- 172.21.15.59:0/3694027630 shutdown_connections 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.077+0000 7f51a21ea740 1 -- 172.21.15.59:0/3694027630 wait complete. 2024-05-08T05:32:45.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f51a21ea740 20 client.15858 trim_cache size 0 max 16384 2024-05-08T05:32:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:32:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:32:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:32:45.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:32:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 -- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f51840b2a90 msgr2=0x7f51840b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f51840b2a90 0x7f51840b4f50 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f518800f8c0 tx=0x7f5188018000 comp rx=0 tx=0).stop 2024-05-08T05:32:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 -- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51981089b0 msgr2=0x7f519819d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51981089b0 0x7f519819d9b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f5190033040 tx=0x7f5190009360 comp rx=0 tx=0).stop 2024-05-08T05:32:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 -- 172.21.15.59:0/1114412436 shutdown_connections 2024-05-08T05:32:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f51840b2a90 0x7f51840b4f50 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f51981089b0 0x7f519819d9b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5198103360 0x7f5198199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:45.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 --2- 172.21.15.59:0/1114412436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f51981029a0 0x7f51981990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 -- 172.21.15.59:0/1114412436 >> 172.21.15.59:0/1114412436 conn(0x7f51980fe470 msgr2=0x7f51980feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.078+0000 7f519f4d1640 1 -- 172.21.15.59:0/1114412436 shutdown_connections 2024-05-08T05:32:45.083 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:45.079+0000 7f519f4d1640 1 -- 172.21.15.59:0/1114412436 wait complete. 2024-05-08T05:32:45.330 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:45 smithi096 ceph-mon[34253]: pgmap v98: 129 pgs: 129 active+clean; 37 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.5 MiB/s wr, 398 op/s 2024-05-08T05:32:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:45 smithi059 ceph-mon[30304]: pgmap v98: 129 pgs: 129 active+clean; 37 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.5 MiB/s wr, 398 op/s 2024-05-08T05:32:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:45 smithi143 ceph-mon[34337]: pgmap v98: 129 pgs: 129 active+clean; 37 MiB data, 588 MiB used, 1.0 TiB / 1.0 TiB avail; 485 KiB/s rd, 1.5 MiB/s wr, 398 op/s 2024-05-08T05:32:45.838 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xff038 2024-05-08T05:32:46.977 INFO:tasks.quiescer.fs.[cephfs]:Total duration (42.1) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-08T05:32:46.977 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-05-08T05:32:46.978 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:32:47.099 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0xffcac 2024-05-08T05:32:47.267 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:32:47.270 INFO:tasks.workunit.client.0.smithi059.stdout:10000 copy from 0x2d237 to 0x30552, (0x331b bytes) at 0x82aed 2024-05-08T05:32:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:47 smithi059 ceph-mon[30304]: pgmap v99: 129 pgs: 129 active+clean; 41 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 653 op/s 2024-05-08T05:32:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:47 smithi143 ceph-mon[34337]: pgmap v99: 129 pgs: 129 active+clean; 41 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 653 op/s 2024-05-08T05:32:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:47 smithi096 ceph-mon[34253]: pgmap v99: 129 pgs: 129 active+clean; 41 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 3.1 MiB/s wr, 653 op/s 2024-05-08T05:32:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.096+0000 7f49b27b2700 1 -- 172.21.15.59:0/474128850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ac102d60 msgr2=0x7f49ac10f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.096+0000 7f49b27b2700 1 --2- 172.21.15.59:0/474128850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ac102d60 0x7f49ac10f250 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f49a4009f90 tx=0x7f49a400b330 comp rx=0 tx=0).stop 2024-05-08T05:32:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.096+0000 7f49b27b2700 1 -- 172.21.15.59:0/474128850 shutdown_connections 2024-05-08T05:32:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.096+0000 7f49b27b2700 1 --2- 172.21.15.59:0/474128850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ac102d60 0x7f49ac10f250 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.096+0000 7f49b27b2700 1 --2- 172.21.15.59:0/474128850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49ac108410 0x7f49ac1087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.096+0000 7f49b27b2700 1 --2- 172.21.15.59:0/474128850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49ac1023a0 0x7f49ac102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.096+0000 7f49b27b2700 1 -- 172.21.15.59:0/474128850 >> 172.21.15.59:0/474128850 conn(0x7f49ac079350 msgr2=0x7f49ac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:49.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.096+0000 7f49b27b2700 1 -- 172.21.15.59:0/474128850 shutdown_connections 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.096+0000 7f49b27b2700 1 -- 172.21.15.59:0/474128850 wait complete. 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.097+0000 7f49b27b2700 1 Processor -- start 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.097+0000 7f49b27b2700 1 -- start start 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.097+0000 7f49b27b2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49ac1023a0 0x7f49ac19e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.097+0000 7f49b27b2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ac102d60 0x7f49ac19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.097+0000 7f49b27b2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49ac108410 0x7f49ac198880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49b27b2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac112560 con 0x7f49ac102d60 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49b27b2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac1123e0 con 0x7f49ac1023a0 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49b27b2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f49ac1126e0 con 0x7f49ac108410 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49abfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49ac1023a0 0x7f49ac19e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49abfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49ac1023a0 0x7f49ac19e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:58348/0 (socket says 172.21.15.59:58348) 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49b0d4f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49ac108410 0x7f49ac198880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49abfff700 1 -- 172.21.15.59:0/2050846750 learned_addr learned my addr 172.21.15.59:0/2050846750 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49ab7fe700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ac102d60 0x7f49ac19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:49.101 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49abfff700 1 -- 172.21.15.59:0/2050846750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49ac108410 msgr2=0x7f49ac198880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49abfff700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49ac108410 0x7f49ac198880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49abfff700 1 -- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ac102d60 msgr2=0x7f49ac19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49abfff700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ac102d60 0x7f49ac19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49abfff700 1 -- 172.21.15.59:0/2050846750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49ac199060 con 0x7f49ac1023a0 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49ab7fe700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ac102d60 0x7f49ac19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49b0d4f700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49ac108410 0x7f49ac198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49abfff700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49ac1023a0 0x7f49ac19e6b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f499c00ec10 tx=0x7f499c00ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49a97fa700 1 -- 172.21.15.59:0/2050846750 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c00cc90 con 0x7f49ac1023a0 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49a97fa700 1 -- 172.21.15.59:0/2050846750 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f499c00ce30 con 0x7f49ac1023a0 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49ac199350 con 0x7f49ac1023a0 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.098+0000 7f49a97fa700 1 -- 172.21.15.59:0/2050846750 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f499c010430 con 0x7f49ac1023a0 2024-05-08T05:32:49.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.099+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49ac06b180 con 0x7f49ac1023a0 2024-05-08T05:32:49.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.099+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49ac103ac0 con 0x7f49ac1023a0 2024-05-08T05:32:49.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.101+0000 7f49a97fa700 1 -- 172.21.15.59:0/2050846750 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f499c0479e0 con 0x7f49ac1023a0 2024-05-08T05:32:49.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.102+0000 7f49a97fa700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f49940b29f0 0x7f49940b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:49.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.103+0000 7f49ab7fe700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f49940b29f0 0x7f49940b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:49.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.103+0000 7f49a97fa700 1 -- 172.21.15.59:0/2050846750 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f499c0e2c00 con 0x7f49ac1023a0 2024-05-08T05:32:49.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.103+0000 7f49ab7fe700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f49940b29f0 0x7f49940b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49ac079950 tx=0x7f49a0009260 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:49.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.105+0000 7f49a97fa700 1 -- 172.21.15.59:0/2050846750 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f499c0aa230 con 0x7f49ac1023a0 2024-05-08T05:32:49.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:49 smithi059 ceph-mon[30304]: pgmap v100: 129 pgs: 129 active+clean; 41 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-05-08T05:32:49.422 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.421+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f49ac06a190 con 0x7f49ac1023a0 2024-05-08T05:32:49.423 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.422+0000 7f49a97fa700 1 -- 172.21.15.59:0/2050846750 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11934 (secure 0 0 0) 0x7f49ac06a190 con 0x7f49ac1023a0 2024-05-08T05:32:49.424 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:32:49.424 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:32:11.760389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f49940b29f0 msgr2=0x7f49940b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f49940b29f0 0x7f49940b4eb0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f49ac079950 tx=0x7f49a0009260 comp rx=0 tx=0).stop 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49ac1023a0 msgr2=0x7f49ac19e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49ac1023a0 0x7f49ac19e6b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f499c00ec10 tx=0x7f499c00ef20 comp rx=0 tx=0).stop 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 shutdown_connections 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f49940b29f0 0x7f49940b4eb0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f49ac108410 0x7f49ac198880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f49ac1023a0 0x7f49ac19e6b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 --2- 172.21.15.59:0/2050846750 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f49ac102d60 0x7f49ac19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 >> 172.21.15.59:0/2050846750 conn(0x7f49ac079350 msgr2=0x7f49ac100fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 shutdown_connections 2024-05-08T05:32:49.427 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:49.425+0000 7f49b27b2700 1 -- 172.21.15.59:0/2050846750 wait complete. 2024-05-08T05:32:49.428 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-05-08T05:32:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:49 smithi143 ceph-mon[34337]: pgmap v100: 129 pgs: 129 active+clean; 41 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-05-08T05:32:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:49 smithi096 ceph-mon[34253]: pgmap v100: 129 pgs: 129 active+clean; 41 MiB data, 634 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.9 MiB/s wr, 351 op/s 2024-05-08T05:32:50.218 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db / --timeout 60 --expiration 81.05 --await-for 42.0' 2024-05-08T05:32:50.218 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db / --timeout 60 --expiration 81.05 --await-for 42.0 2024-05-08T05:32:50.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:50 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2050846750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:32:50.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:50 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2050846750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:32:50.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:50 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2050846750' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:32:50.531 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:32:51.284 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:51 smithi059 ceph-mon[30304]: pgmap v101: 129 pgs: 129 active+clean; 35 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 629 op/s 2024-05-08T05:32:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:51 smithi143 ceph-mon[34337]: pgmap v101: 129 pgs: 129 active+clean; 35 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 629 op/s 2024-05-08T05:32:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:51 smithi096 ceph-mon[34253]: pgmap v101: 129 pgs: 129 active+clean; 35 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 3.3 MiB/s wr, 629 op/s 2024-05-08T05:32:53.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.192+0000 7fbe55608700 1 -- 172.21.15.59:0/47159298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe5006c430 msgr2=0x7fbe50110350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:53.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.192+0000 7fbe55608700 1 --2- 172.21.15.59:0/47159298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe5006c430 0x7fbe50110350 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800b0a0 tx=0x7fbe4800b3b0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.192+0000 7fbe55608700 1 -- 172.21.15.59:0/47159298 shutdown_connections 2024-05-08T05:32:53.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.192+0000 7fbe55608700 1 --2- 172.21.15.59:0/47159298 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe5006c430 0x7fbe50110350 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.192+0000 7fbe55608700 1 --2- 172.21.15.59:0/47159298 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe50109530 0x7fbe50109910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.192+0000 7fbe55608700 1 --2- 172.21.15.59:0/47159298 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe5006ba70 0x7fbe5006bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.192+0000 7fbe55608700 1 -- 172.21.15.59:0/47159298 >> 172.21.15.59:0/47159298 conn(0x7fbe500793e0 msgr2=0x7fbe500797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:53.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.192+0000 7fbe55608700 1 -- 172.21.15.59:0/47159298 shutdown_connections 2024-05-08T05:32:53.194 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.192+0000 7fbe55608700 1 -- 172.21.15.59:0/47159298 wait complete. 2024-05-08T05:32:53.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe55608700 1 Processor -- start 2024-05-08T05:32:53.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe55608700 1 -- start start 2024-05-08T05:32:53.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe55608700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe5006ba70 0x7fbe5019f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe55608700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe5006c430 0x7fbe5019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe55608700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe50109530 0x7fbe50199860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe55608700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe50113640 con 0x7fbe5006ba70 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe55608700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501134c0 con 0x7fbe50109530 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe55608700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe501137c0 con 0x7fbe5006c430 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe4e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe5006c430 0x7fbe5019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe4e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe5006c430 0x7fbe5019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:52106/0 (socket says 172.21.15.59:52106) 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.193+0000 7fbe4e7fc700 1 -- 172.21.15.59:0/3729990980 learned_addr learned my addr 172.21.15.59:0/3729990980 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe4f7fe700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe50109530 0x7fbe50199860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe4effd700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe5006ba70 0x7fbe5019f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe4e7fc700 1 -- 172.21.15.59:0/3729990980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe50109530 msgr2=0x7fbe50199860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe50109530 0x7fbe50199860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe4e7fc700 1 -- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe5006ba70 msgr2=0x7fbe5019f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:53.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe5006ba70 0x7fbe5019f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe4e7fc700 1 -- 172.21.15.59:0/3729990980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe50199fc0 con 0x7fbe5006c430 2024-05-08T05:32:53.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe4effd700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe5006ba70 0x7fbe5019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:32:53.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe5006c430 0x7fbe5019fbe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400e9a0 tx=0x7fbe4400ecb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:53.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe37fff700 1 -- 172.21.15.59:0/3729990980 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe4400cbd0 con 0x7fbe5006c430 2024-05-08T05:32:53.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe55608700 1 -- 172.21.15.59:0/3729990980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe5019a2b0 con 0x7fbe5006c430 2024-05-08T05:32:53.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe55608700 1 -- 172.21.15.59:0/3729990980 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe501a6570 con 0x7fbe5006c430 2024-05-08T05:32:53.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe37fff700 1 -- 172.21.15.59:0/3729990980 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe4400cd70 con 0x7fbe5006c430 2024-05-08T05:32:53.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.194+0000 7fbe37fff700 1 -- 172.21.15.59:0/3729990980 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe44018680 con 0x7fbe5006c430 2024-05-08T05:32:53.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.197+0000 7fbe37fff700 1 -- 172.21.15.59:0/3729990980 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbe44018820 con 0x7fbe5006c430 2024-05-08T05:32:53.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.198+0000 7fbe37fff700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe380b2b20 0x7fbe380b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.198+0000 7fbe4effd700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe380b2b20 0x7fbe380b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.198+0000 7fbe37fff700 1 -- 172.21.15.59:0/3729990980 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fbe440e2b50 con 0x7fbe5006c430 2024-05-08T05:32:53.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.198+0000 7fbe5fe9bb80 1 Processor -- start 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.198+0000 7fbe5fe9bb80 1 -- start start 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f31155f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 0x563f31162610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f310a7ad0 con 0x563f31155b00 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f3100cf30 con 0x563f311564c0 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563f310a3a90 con 0x563f3115b850 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4effd700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115bc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:52112/0 (socket says 172.21.15.59:52112) 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4effd700 1 -- 172.21.15.59:0/718034948 learned_addr learned my addr 172.21.15.59:0/718034948 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4f7fe700 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 0x563f31162610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f31155f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.202 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4effd700 1 -- 172.21.15.59:0/718034948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 msgr2=0x563f31162610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4effd700 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 0x563f31162610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4effd700 1 -- 172.21.15.59:0/718034948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 msgr2=0x563f31155f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4effd700 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f31155f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4effd700 1 -- 172.21.15.59:0/718034948 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f30ff4a60 con 0x563f3115b850 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.199+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f31155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.200+0000 7fbe4effd700 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115bc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbe40013b20 tx=0x7fbe40013e30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=74576aa2dcd7fbb3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.200+0000 7fbe4effd700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe380b2b20 0x7fbe380b4fe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000f640 tx=0x7fbe40011000 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.200+0000 7fbe357fa700 1 -- 172.21.15.59:0/718034948 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40023070 con 0x563f3115b850 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.200+0000 7fbe357fa700 1 -- 172.21.15.59:0/718034948 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe400044e0 con 0x563f3115b850 2024-05-08T05:32:53.203 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.200+0000 7fbe357fa700 1 -- 172.21.15.59:0/718034948 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40003c80 con 0x563f3115b850 2024-05-08T05:32:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.201+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/718034948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 msgr2=0x563f3115bc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.201+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115bc30 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbe40013b20 tx=0x7fbe40013e30 comp rx=0 tx=0).stop 2024-05-08T05:32:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.201+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/718034948 shutdown_connections 2024-05-08T05:32:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.201+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 0x563f31162610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.208 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.201+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115bc30 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.201+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/718034948 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f31155f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.201+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/718034948 >> 172.21.15.59:0/718034948 conn(0x563f31081d00 msgr2=0x563f31150880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.201+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/718034948 shutdown_connections 2024-05-08T05:32:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.201+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/718034948 wait complete. 2024-05-08T05:32:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe5fe9bb80 1 -- start start 2024-05-08T05:32:53.209 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe5fe9bb80 1 Processor -- start 2024-05-08T05:32:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f31162170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f311564c0 0x563f3115f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3115b850 0x563f3115fa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563f31166f60 con 0x563f31155b00 2024-05-08T05:32:53.210 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563f31061060 con 0x563f3115b850 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563f311670e0 con 0x563f311564c0 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe4e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f311564c0 0x563f3115f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe4e7fc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f311564c0 0x563f3115f510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:52116/0 (socket says 172.21.15.59:52116) 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe4e7fc700 1 -- 172.21.15.59:0/3644713547 learned_addr learned my addr 172.21.15.59:0/3644713547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.202+0000 7fbe4f7fe700 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3115b850 0x563f3115fa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe4effd700 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f31162170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe5fe9bb80 10 client.0 _ll_register_callbacks cb 0x563f3103b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe4f7fe700 1 -- 172.21.15.59:0/3644713547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f311564c0 msgr2=0x563f3115f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe4f7fe700 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f311564c0 0x563f3115f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe4f7fe700 1 -- 172.21.15.59:0/3644713547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 msgr2=0x563f31162170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:53.211 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe4f7fe700 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f31162170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe4f7fe700 1 -- 172.21.15.59:0/3644713547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f3105e8f0 con 0x563f3115b850 2024-05-08T05:32:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f311564c0 0x563f3115f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe4f7fe700 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3115b850 0x563f3115fa50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800d010 tx=0x7fbe4800bd50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=af0a6b62edf9745b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:32:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe32ff5700 1 -- 172.21.15.59:0/3644713547 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe480076e0 con 0x563f3115b850 2024-05-08T05:32:53.212 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/3644713547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe400044e0 con 0x563f3115b850 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe32ff5700 1 -- 172.21.15.59:0/3644713547 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fbe48007880 con 0x563f3115b850 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe32ff5700 1 -- 172.21.15.59:0/3644713547 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbe48030050 con 0x563f3115b850 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe5fe9bb80 10 client.25499 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.203+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/3644713547 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563f311715c0 con 0x563f3115b850 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.204+0000 7fbe32ff5700 1 -- 172.21.15.59:0/3644713547 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe400044e0 con 0x563f3115b850 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.204+0000 7fbe32ff5700 1 -- 172.21.15.59:0/3644713547 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fbe48011070 con 0x563f3115b850 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.204+0000 7fbe5fe9bb80 10 client.25499 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.204+0000 7fbe5fe9bb80 10 client.25499 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.204+0000 7fbe5fe9bb80 20 client.25499 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:53.213 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.204+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563f31173ba0 0x563f31176000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.204+0000 7fbe5fe9bb80 4 client.25499 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.204+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/3644713547 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563f310b54b0 con 0x563f31173ba0 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.204+0000 7fbe4effd700 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563f31173ba0 0x563f31176000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.205+0000 7fbe32ff5700 10 client.25499 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.205+0000 7fbe4effd700 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563f31173ba0 0x563f31176000 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8769a8da3627384 server_cookie=1a1cf164dbe4d2a2 in_seq=0 out_seq=0 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.205+0000 7fbe32ff5700 1 -- 172.21.15.59:0/3644713547 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563f310b54b0 con 0x563f31173ba0 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.205+0000 7fbe32ff5700 10 client.25499 handle_command_reply: tid=0 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.205+0000 7fbe5fe9bb80 1 client.25499 shutdown 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/3644713547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3115b850 msgr2=0x563f3115fa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3115b850 0x563f3115fa50 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fbe4800d010 tx=0x7fbe4800bd50 comp rx=0 tx=0).stop 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/3644713547 shutdown_connections 2024-05-08T05:32:53.214 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563f31173ba0 0x563f31176000 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f3115b850 0x563f3115fa50 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f31162170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe32ff5700 0 client.25499 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/3644713547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f311564c0 0x563f3115f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/3644713547 >> 172.21.15.59:0/3644713547 conn(0x563f31081d00 msgr2=0x563f3114ae20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/3644713547 shutdown_connections 2024-05-08T05:32:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/3644713547 wait complete. 2024-05-08T05:32:53.215 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.206+0000 7fbe5fe9bb80 20 client.25499 trim_cache size 0 max 16384 2024-05-08T05:32:53.237 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.231+0000 7fbe5fe9bb80 1 Processor -- start 2024-05-08T05:32:53.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.231+0000 7fbe5fe9bb80 1 -- start start 2024-05-08T05:32:53.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.231+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f31155b00 0x563f3114af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.231+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f311564c0 0x563f3115a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.231+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.231+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c00ecc0 con 0x563f311564c0 2024-05-08T05:32:53.238 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.231+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c010890 con 0x563f31155b00 2024-05-08T05:32:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.231+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c00f390 con 0x563f3115b850 2024-05-08T05:32:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f311564c0 0x563f3115a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4e7fc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f311564c0 0x563f3115a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34398/0 (socket says 172.21.15.59:34398) 2024-05-08T05:32:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4e7fc700 1 -- 172.21.15.59:0/1293622292 learned_addr learned my addr 172.21.15.59:0/1293622292 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4effd700 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f31155b00 0x563f3114af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.239 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4f7fe700 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4e7fc700 1 -- 172.21.15.59:0/1293622292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 msgr2=0x563f3115aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4e7fc700 1 -- 172.21.15.59:0/1293622292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f31155b00 msgr2=0x563f3114af30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f31155b00 0x563f3114af30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4e7fc700 1 -- 172.21.15.59:0/1293622292 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563f31162950 con 0x563f311564c0 2024-05-08T05:32:53.240 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4effd700 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f31155b00 0x563f3114af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:32:53.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f311564c0 0x563f3115a540 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbe440a6cf0 tx=0x7fbe440a5a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4044d91bcf2e80c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:53.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe34ff9700 1 -- 172.21.15.59:0/1293622292 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe440aa180 con 0x563f311564c0 2024-05-08T05:32:53.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.232+0000 7fbe34ff9700 1 -- 172.21.15.59:0/1293622292 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe440a9a80 con 0x563f311564c0 2024-05-08T05:32:53.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.233+0000 7fbe34ff9700 1 -- 172.21.15.59:0/1293622292 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe440a8fd0 con 0x563f311564c0 2024-05-08T05:32:53.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.233+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/1293622292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f311564c0 msgr2=0x563f3115a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:53.241 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.233+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f311564c0 0x563f3115a540 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7fbe440a6cf0 tx=0x7fbe440a5a70 comp rx=0 tx=0).stop 2024-05-08T05:32:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.233+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/1293622292 shutdown_connections 2024-05-08T05:32:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.233+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f3115aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.233+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f31155b00 0x563f3114af30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.233+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/1293622292 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f311564c0 0x563f3115a540 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.233+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/1293622292 >> 172.21.15.59:0/1293622292 conn(0x563f31081d00 msgr2=0x563f3115c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:53.242 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.233+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/1293622292 shutdown_connections 2024-05-08T05:32:53.243 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/1293622292 wait complete. 2024-05-08T05:32:53.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 1 -- start start 2024-05-08T05:32:53.247 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 1 Processor -- start 2024-05-08T05:32:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f3116c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 0x563f3116ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f31157250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c010130 con 0x563f31155b00 2024-05-08T05:32:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c00fa60 con 0x563f311564c0 2024-05-08T05:32:53.248 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe3c00cc40 con 0x563f3115b850 2024-05-08T05:32:53.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe5fe9bb80 10 client.0 _ll_register_callbacks cb 0x563f3103b740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:53.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.234+0000 7fbe4effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f3116c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4effd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f3116c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34400/0 (socket says 172.21.15.59:34400) 2024-05-08T05:32:53.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4effd700 1 -- 172.21.15.59:0/39558421 learned_addr learned my addr 172.21.15.59:0/39558421 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:53.249 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 0x563f3116ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4effd700 1 -- 172.21.15.59:0/39558421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 msgr2=0x563f31157250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:53.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4effd700 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f31157250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4effd700 1 -- 172.21.15.59:0/39558421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 msgr2=0x563f3116ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:53.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4effd700 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 0x563f3116ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:53.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4effd700 1 -- 172.21.15.59:0/39558421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563f31157a60 con 0x563f31155b00 2024-05-08T05:32:53.250 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 0x563f3116ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:32:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe4effd700 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f3116c510 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009c40 tx=0x7fbe4000f6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d53f678696512b9b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe32ff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:32:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe32ff5700 1 -- 172.21.15.59:0/39558421 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe40023040 con 0x563f31155b00 2024-05-08T05:32:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe32ff5700 1 -- 172.21.15.59:0/39558421 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fbe400050c0 con 0x563f31155b00 2024-05-08T05:32:53.251 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.235+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/39558421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe440a9a80 con 0x563f31155b00 2024-05-08T05:32:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe32ff5700 1 -- 172.21.15.59:0/39558421 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fbe4003b050 con 0x563f31155b00 2024-05-08T05:32:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe5fe9bb80 10 client.15876 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/39558421 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x563f31158d40 con 0x563f31155b00 2024-05-08T05:32:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe32ff5700 1 -- 172.21.15.59:0/39558421 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe440a9a80 con 0x563f31155b00 2024-05-08T05:32:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe32ff5700 1 -- 172.21.15.59:0/39558421 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fbe40003b50 con 0x563f31155b00 2024-05-08T05:32:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe5fe9bb80 10 client.15876 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:53.252 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe5fe9bb80 10 client.15876 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:32:53.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe5fe9bb80 20 client.15876 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:53.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563f31173ba0 0x563f311e1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:53.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe5fe9bb80 4 client.15876 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.05, "await_for": 42.0}] 2024-05-08T05:32:53.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/39558421 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.05, "await_for": 42.0}) -- 0x563f310af550 con 0x563f31173ba0 2024-05-08T05:32:53.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.236+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563f31173ba0 0x563f311e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:53.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.237+0000 7fbe4e7fc700 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563f31173ba0 0x563f311e1890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6e5cee09f4ada657 server_cookie=8027eb99791d7fd in_seq=0 out_seq=0 2024-05-08T05:32:53.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:53.237+0000 7fbe32ff5700 10 client.15876 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:53.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:53 smithi059 ceph-mon[30304]: pgmap v102: 129 pgs: 129 active+clean; 35 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-05-08T05:32:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:53 smithi143 ceph-mon[34337]: pgmap v102: 129 pgs: 129 active+clean; 35 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-05-08T05:32:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:53 smithi096 ceph-mon[34253]: pgmap v102: 129 pgs: 129 active+clean; 35 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-05-08T05:32:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:55 smithi143 ceph-mon[34337]: pgmap v103: 129 pgs: 129 active+clean; 35 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-05-08T05:32:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:55 smithi096 ceph-mon[34253]: pgmap v103: 129 pgs: 129 active+clean; 35 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-05-08T05:32:55.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:55 smithi059 ceph-mon[30304]: pgmap v103: 129 pgs: 129 active+clean; 35 MiB data, 661 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.1 MiB/s wr, 531 op/s 2024-05-08T05:32:55.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe32ff5700 1 -- 172.21.15.59:0/39558421 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563f310af550 con 0x563f31173ba0 2024-05-08T05:32:55.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe32ff5700 10 client.15876 handle_command_reply: tid=0 2024-05-08T05:32:55.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe5fe9bb80 1 client.15876 shutdown 2024-05-08T05:32:55.852 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/39558421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 msgr2=0x563f3116c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:55.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f3116c510 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7fbe40009c40 tx=0x7fbe4000f6f0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/39558421 shutdown_connections 2024-05-08T05:32:55.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563f31173ba0 0x563f311e1890 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563f3115b850 0x563f31157250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563f31155b00 0x563f3116c510 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe32ff5700 0 client.15876 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:55.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe5fe9bb80 1 --2- 172.21.15.59:0/39558421 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563f311564c0 0x563f3116ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.849+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/39558421 >> 172.21.15.59:0/39558421 conn(0x563f31081d00 msgr2=0x563f3115cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:55.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/39558421 shutdown_connections 2024-05-08T05:32:55.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe5fe9bb80 1 -- 172.21.15.59:0/39558421 wait complete. 2024-05-08T05:32:55.854 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe5fe9bb80 20 client.15876 trim_cache size 0 max 16384 2024-05-08T05:32:55.855 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:32:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 33, 2024-05-08T05:32:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:32:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 10, 2024-05-08T05:32:55.855 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:32:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "bc67f892": { 2024-05-08T05:32:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 10, 2024-05-08T05:32:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:32:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:32:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:32:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:32:55.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:32:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:32:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 81.0, 2024-05-08T05:32:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:32:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:32:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:32:55.857 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:32:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:32:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:32:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:32:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:32:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:32:55.858 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:32:55.859 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:32:55.859 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:32:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe55608700 1 -- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe380b2b20 msgr2=0x7fbe380b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe55608700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe380b2b20 0x7fbe380b4fe0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fbe4000f640 tx=0x7fbe40011000 comp rx=0 tx=0).stop 2024-05-08T05:32:55.860 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe55608700 1 -- 172.21.15.59:0/3729990980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe5006c430 msgr2=0x7fbe5019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe55608700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe5006c430 0x7fbe5019fbe0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbe4400e9a0 tx=0x7fbe4400ecb0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe55608700 1 -- 172.21.15.59:0/3729990980 shutdown_connections 2024-05-08T05:32:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe55608700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe380b2b20 0x7fbe380b4fe0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe55608700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe50109530 0x7fbe50199860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe55608700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe5006ba70 0x7fbe5019f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.861 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.850+0000 7fbe55608700 1 --2- 172.21.15.59:0/3729990980 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe5006c430 0x7fbe5019fbe0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.851+0000 7fbe55608700 1 -- 172.21.15.59:0/3729990980 >> 172.21.15.59:0/3729990980 conn(0x7fbe500793e0 msgr2=0x7fbe50100e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.851+0000 7fbe55608700 1 -- 172.21.15.59:0/3729990980 shutdown_connections 2024-05-08T05:32:55.862 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:32:55.851+0000 7fbe55608700 1 -- 172.21.15.59:0/3729990980 wait complete. 2024-05-08T05:32:56.702 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'bc67f892', quiesce took 9.7 seconds. Will release after: 21.05 2024-05-08T05:32:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:57 smithi143 ceph-mon[34337]: pgmap v104: 129 pgs: 129 active+clean; 39 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 693 op/s 2024-05-08T05:32:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:57 smithi096 ceph-mon[34253]: pgmap v104: 129 pgs: 129 active+clean; 39 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 693 op/s 2024-05-08T05:32:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:57 smithi059 ceph-mon[30304]: pgmap v104: 129 pgs: 129 active+clean; 39 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 4.2 MiB/s wr, 693 op/s 2024-05-08T05:32:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.182+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1695765400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 msgr2=0x7f46a0106c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.182+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1695765400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 0x7f46a0106c30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4690009900 tx=0x7f469002a410 comp rx=0 tx=0).stop 2024-05-08T05:32:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.182+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1695765400 shutdown_connections 2024-05-08T05:32:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.182+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1695765400 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a01025c0 0x7f46a010e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.182+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1695765400 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a0101c00 0x7f46a0102080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.182+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1695765400 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 0x7f46a0106c30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.182+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1695765400 >> 172.21.15.59:0/1695765400 conn(0x7f46a00776b0 msgr2=0x7f46a0077ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:58.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.182+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1695765400 shutdown_connections 2024-05-08T05:32:58.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.182+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1695765400 wait complete. 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.183+0000 7f46a4d0a640 1 Processor -- start 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.183+0000 7f46a4d0a640 1 -- start start 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.183+0000 7f46a4d0a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a0101c00 0x7f46a0199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.183+0000 7f46a4d0a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a01025c0 0x7f46a01995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.183+0000 7f46a4d0a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 0x7f46a019d920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.183+0000 7f46a4d0a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0111b30 con 0x7f46a0106850 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.183+0000 7f46a4d0a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01119b0 con 0x7f46a0101c00 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.183+0000 7f46a4d0a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f46a0111cb0 con 0x7f46a01025c0 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 0x7f46a019d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 0x7f46a019d920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34422/0 (socket says 172.21.15.59:34422) 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469ed9d640 1 -- 172.21.15.59:0/1562877283 learned_addr learned my addr 172.21.15.59:0/1562877283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469dd9b640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a01025c0 0x7f46a01995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469e59c640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a0101c00 0x7f46a0199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469ed9d640 1 -- 172.21.15.59:0/1562877283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a01025c0 msgr2=0x7f46a01995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469ed9d640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a01025c0 0x7f46a01995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469ed9d640 1 -- 172.21.15.59:0/1562877283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a0101c00 msgr2=0x7f46a0199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469ed9d640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a0101c00 0x7f46a0199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469ed9d640 1 -- 172.21.15.59:0/1562877283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46a019e050 con 0x7f46a0106850 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f469ed9d640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 0x7f46a019d920 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4698009fd0 tx=0x7f469800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f468f7fe640 1 -- 172.21.15.59:0/1562877283 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698017070 con 0x7f46a0106850 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.184+0000 7f468f7fe640 1 -- 172.21.15.59:0/1562877283 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4698003e20 con 0x7f46a0106850 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.185+0000 7f468f7fe640 1 -- 172.21.15.59:0/1562877283 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4698010460 con 0x7f46a0106850 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.185+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1562877283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46a019e340 con 0x7f46a0106850 2024-05-08T05:32:58.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.185+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1562877283 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46a01a5bd0 con 0x7f46a0106850 2024-05-08T05:32:58.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.187+0000 7f468f7fe640 1 -- 172.21.15.59:0/1562877283 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4698003970 con 0x7f46a0106850 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.187+0000 7f46a7a1e740 1 Processor -- start 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f46a7a1e740 1 -- start start 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12f11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a12fffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a1300500 0x55e7a13028f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e7a12565d0 con 0x55e7a1300500 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e7a120f890 con 0x55e7a12f0df0 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e7a1252590 con 0x55e7a12f1710 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f469dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12f11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f469dd9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12f11d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:58442/0 (socket says 172.21.15.59:58442) 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f469dd9b640 1 -- 172.21.15.59:0/2499272667 learned_addr learned my addr 172.21.15.59:0/2499272667 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2499272667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a1300500 0x55e7a13028f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f469e59c640 1 --2- 172.21.15.59:0/2499272667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a12fffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.188+0000 7f468f7fe640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f46880b2a10 0x7f46880b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f469dd9b640 1 -- 172.21.15.59:0/2499272667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 msgr2=0x55e7a12fffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2499272667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a12fffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f469dd9b640 1 -- 172.21.15.59:0/2499272667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a1300500 msgr2=0x55e7a13028f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2499272667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a1300500 0x55e7a13028f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f469dd9b640 1 -- 172.21.15.59:0/2499272667 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7a0f68210 con 0x55e7a12f0df0 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f469ed9d640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f46880b2a10 0x7f46880b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f468f7fe640 1 -- 172.21.15.59:0/1562877283 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f46980e2760 con 0x7f46a0106850 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2499272667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12f11d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4694011840 tx=0x7f46940097d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90cbbf5b1dd3415f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f468cff9640 1 -- 172.21.15.59:0/2499272667 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694004030 con 0x55e7a12f0df0 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f469ed9d640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f46880b2a10 0x7f46880b4ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46980a93d0 tx=0x7f469809df50 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f468cff9640 1 -- 172.21.15.59:0/2499272667 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469400f7c0 con 0x55e7a12f0df0 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.189+0000 7f468cff9640 1 -- 172.21.15.59:0/2499272667 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694003910 con 0x55e7a12f0df0 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2499272667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 msgr2=0x55e7a12f11d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:58.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2499272667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12f11d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f4694011840 tx=0x7f46940097d0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2499272667 shutdown_connections 2024-05-08T05:32:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2499272667 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a1300500 0x55e7a13028f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2499272667 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a12fffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2499272667 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12f11d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2499272667 >> 172.21.15.59:0/2499272667 conn(0x55e7a12ed420 msgr2=0x55e7a12ed800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2499272667 shutdown_connections 2024-05-08T05:32:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2499272667 wait complete. 2024-05-08T05:32:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 -- start start 2024-05-08T05:32:58.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.190+0000 7f46a7a1e740 1 Processor -- start 2024-05-08T05:32:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a12fd4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a12fda00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.198 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 0x55e7a12fdf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e7a1303130 con 0x55e7a12f0df0 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e7a1213370 con 0x55e7a1300500 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e7a13032b0 con 0x55e7a12f1710 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f46a7a1e740 10 client.0 _ll_register_callbacks cb 0x55e7a0f67a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a12fd4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a12fd4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34440/0 (socket says 172.21.15.59:34440) 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 -- 172.21.15.59:0/2723316486 learned_addr learned my addr 172.21.15.59:0/2723316486 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469e59c640 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a12fda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 0x55e7a12fdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 -- 172.21.15.59:0/2723316486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 msgr2=0x55e7a12fda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a12fda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 -- 172.21.15.59:0/2723316486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 msgr2=0x55e7a12fdf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:58.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 0x55e7a12fdf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 -- 172.21.15.59:0/2723316486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7a1034620 con 0x55e7a12f0df0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a12fd4c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4694009df0 tx=0x7f4694005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b6aea2182f1d91 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.191+0000 7f4685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f4685ffb640 1 -- 172.21.15.59:0/2723316486 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4694005460 con 0x55e7a12f0df0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f4685ffb640 1 -- 172.21.15.59:0/2723316486 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f46940046c0 con 0x55e7a12f0df0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2723316486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f469400f7c0 con 0x55e7a12f0df0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f4685ffb640 1 -- 172.21.15.59:0/2723316486 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4694032050 con 0x55e7a12f0df0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f46a7a1e740 10 client.15888 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2723316486 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e7a130f0b0 con 0x55e7a12f0df0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f4685ffb640 1 -- 172.21.15.59:0/2723316486 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f469400f7c0 con 0x55e7a12f0df0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f4685ffb640 1 -- 172.21.15.59:0/2723316486 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f469401f460 con 0x55e7a12f0df0 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f46a7a1e740 10 client.15888 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f46a7a1e740 10 client.15888 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:58.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f46a7a1e740 20 client.15888 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e7a1310400 0x55e7a13127f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f46a7a1e740 4 client.15888 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.192+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2723316486 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e7a12641e0 con 0x55e7a1310400 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.193+0000 7f469e59c640 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e7a1310400 0x55e7a13127f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.193+0000 7f4685ffb640 10 client.15888 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.193+0000 7f469e59c640 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e7a1310400 0x55e7a13127f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0c74d5376308519 server_cookie=d445c1110df8ec24 in_seq=0 out_seq=0 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f4685ffb640 1 -- 172.21.15.59:0/2723316486 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e7a12641e0 con 0x55e7a1310400 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f4685ffb640 10 client.15888 handle_command_reply: tid=0 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 client.15888 shutdown 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2723316486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 msgr2=0x55e7a12fd4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a12fd4c0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f4694009df0 tx=0x7f4694005fd0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2723316486 shutdown_connections 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e7a1310400 0x55e7a13127f0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 0x55e7a12fdf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a12fda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2723316486 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a12fd4c0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f4685ffb640 0 client.15888 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2723316486 >> 172.21.15.59:0/2723316486 conn(0x55e7a12ed420 msgr2=0x55e7a12f3290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2723316486 shutdown_connections 2024-05-08T05:32:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2723316486 wait complete. 2024-05-08T05:32:58.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.194+0000 7f46a7a1e740 20 client.15888 trim_cache size 0 max 16384 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f46a7a1e740 1 Processor -- start 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f46a7a1e740 1 -- start start 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12ff400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f1710 0x55e7a1318040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a1300500 0x55e7a1318580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01adab0 con 0x55e7a12f1710 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01af680 con 0x55e7a12f0df0 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01ae180 con 0x55e7a1300500 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f469e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f1710 0x55e7a1318040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f469e59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f1710 0x55e7a1318040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34454/0 (socket says 172.21.15.59:34454) 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f469e59c640 1 -- 172.21.15.59:0/2696870860 learned_addr learned my addr 172.21.15.59:0/2696870860 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:58.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.210+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2696870860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12ff400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f469e59c640 1 -- 172.21.15.59:0/2696870860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a1300500 msgr2=0x55e7a1318580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f469e59c640 1 --2- 172.21.15.59:0/2696870860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a1300500 0x55e7a1318580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f469e59c640 1 -- 172.21.15.59:0/2696870860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 msgr2=0x55e7a12ff400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f469e59c640 1 --2- 172.21.15.59:0/2696870860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12ff400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f469e59c640 1 -- 172.21.15.59:0/2696870860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e7a12f1540 con 0x55e7a12f1710 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f469dd9b640 1 --2- 172.21.15.59:0/2696870860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f469e59c640 1 --2- 172.21.15.59:0/2696870860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f1710 0x55e7a1318040 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4690005ec0 tx=0x7f4690005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aceb704e76ad05f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f4687fff640 1 -- 172.21.15.59:0/2696870860 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469004a070 con 0x55e7a12f1710 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f4687fff640 1 -- 172.21.15.59:0/2696870860 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4690004400 con 0x55e7a12f1710 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f4687fff640 1 -- 172.21.15.59:0/2696870860 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4690035040 con 0x55e7a12f1710 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2696870860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f1710 msgr2=0x55e7a1318040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.211+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2696870860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f1710 0x55e7a1318040 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f4690005ec0 tx=0x7f4690005e30 comp rx=0 tx=0).stop 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2696870860 shutdown_connections 2024-05-08T05:32:58.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2696870860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a1300500 0x55e7a1318580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2696870860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f1710 0x55e7a1318040 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/2696870860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a12f0df0 0x55e7a12ff400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2696870860 >> 172.21.15.59:0/2696870860 conn(0x55e7a12ed420 msgr2=0x55e7a12fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2696870860 shutdown_connections 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 -- 172.21.15.59:0/2696870860 wait complete. 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 -- start start 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 Processor -- start 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a1308810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a1305510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 0x55e7a1305a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01aef20 con 0x55e7a12f0df0 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01ae850 con 0x55e7a1300500 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f46a01aba70 con 0x55e7a12f1710 2024-05-08T05:32:58.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f46a7a1e740 10 client.0 _ll_register_callbacks cb 0x7f4694009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f469dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a1308810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f469dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a1308810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34456/0 (socket says 172.21.15.59:34456) 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f469dd9b640 1 -- 172.21.15.59:0/1668675978 learned_addr learned my addr 172.21.15.59:0/1668675978 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.212+0000 7f469e59c640 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a1305510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f469dd9b640 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 0x55e7a1305a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f469dd9b640 1 -- 172.21.15.59:0/1668675978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 msgr2=0x55e7a1305510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f469dd9b640 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a1305510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f469dd9b640 1 -- 172.21.15.59:0/1668675978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 msgr2=0x55e7a1305a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f469dd9b640 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 0x55e7a1305a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f469dd9b640 1 -- 172.21.15.59:0/1668675978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e7a1305fe0 con 0x55e7a12f0df0 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f469dd9b640 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a1308810 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f46940059f0 tx=0x7f4694020f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b744b88f2475b9a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f4685ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:32:58.219 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f4685ffb640 1 -- 172.21.15.59:0/1668675978 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469403c060 con 0x55e7a12f0df0 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f46a7a1e740 1 -- 172.21.15.59:0/1668675978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4690004400 con 0x55e7a12f0df0 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f4685ffb640 1 -- 172.21.15.59:0/1668675978 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f469401f3f0 con 0x55e7a12f0df0 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f4685ffb640 1 -- 172.21.15.59:0/1668675978 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f469401fca0 con 0x55e7a12f0df0 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f46a7a1e740 10 client.15894 fetch_fsmap learned FSMap version 33 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f46a7a1e740 1 -- 172.21.15.59:0/1668675978 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55e7a137b9d0 con 0x55e7a12f0df0 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.213+0000 7f4685ffb640 1 -- 172.21.15.59:0/1668675978 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4690004400 con 0x55e7a12f0df0 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f4685ffb640 1 -- 172.21.15.59:0/1668675978 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f46940037f0 con 0x55e7a12f0df0 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f46a7a1e740 10 client.15894 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f46a7a1e740 10 client.15894 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f46a7a1e740 20 client.15894 populate_metadata read hostname 'smithi059' 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e7a137c1b0 0x55e7a137e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f46a7a1e740 4 client.15894 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f46a7a1e740 1 -- 172.21.15.59:0/1668675978 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e7a125e050 con 0x55e7a137c1b0 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f469e59c640 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e7a137c1b0 0x55e7a137e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:32:58.220 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f469e59c640 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e7a137c1b0 0x55e7a137e5d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ff3e29ce97730fe server_cookie=33fa5ed624e7a479 in_seq=0 out_seq=0 2024-05-08T05:32:58.221 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.214+0000 7f4685ffb640 10 client.15894 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f4685ffb640 1 -- 172.21.15.59:0/1668675978 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e7a125e050 con 0x55e7a137c1b0 2024-05-08T05:32:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f4685ffb640 10 client.15894 handle_command_reply: tid=0 2024-05-08T05:32:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 client.15894 shutdown 2024-05-08T05:32:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 -- 172.21.15.59:0/1668675978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 msgr2=0x55e7a1308810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a1308810 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f46940059f0 tx=0x7f4694020f30 comp rx=0 tx=0).stop 2024-05-08T05:32:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 -- 172.21.15.59:0/1668675978 shutdown_connections 2024-05-08T05:32:58.266 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e7a137c1b0 0x55e7a137e5d0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e7a1300500 0x55e7a1305a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e7a12f1710 0x55e7a1305510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f4685ffb640 0 client.15894 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 --2- 172.21.15.59:0/1668675978 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e7a12f0df0 0x55e7a1308810 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 -- 172.21.15.59:0/1668675978 >> 172.21.15.59:0/1668675978 conn(0x55e7a12ed420 msgr2=0x55e7a12fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 -- 172.21.15.59:0/1668675978 shutdown_connections 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 1 -- 172.21.15.59:0/1668675978 wait complete. 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.264+0000 7f46a7a1e740 20 client.15894 trim_cache size 0 max 16384 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:32:58.267 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:32:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:32:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f46880b2a10 msgr2=0x7f46880b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f46880b2a10 0x7f46880b4ed0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f46980a93d0 tx=0x7f469809df50 comp rx=0 tx=0).stop 2024-05-08T05:32:58.268 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 msgr2=0x7f46a019d920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:32:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 0x7f46a019d920 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f4698009fd0 tx=0x7f469800b260 comp rx=0 tx=0).stop 2024-05-08T05:32:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1562877283 shutdown_connections 2024-05-08T05:32:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f46880b2a10 0x7f46880b4ed0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f46a0106850 0x7f46a019d920 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f46a01025c0 0x7f46a01995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 --2- 172.21.15.59:0/1562877283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f46a0101c00 0x7f46a0199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:32:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1562877283 >> 172.21.15.59:0/1562877283 conn(0x7f46a00776b0 msgr2=0x7f46a006b4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:32:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1562877283 shutdown_connections 2024-05-08T05:32:58.269 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:32:58.265+0000 7f46a4d0a640 1 -- 172.21.15.59:0/1562877283 wait complete. 2024-05-08T05:32:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:32:59 smithi143 ceph-mon[34337]: pgmap v105: 129 pgs: 129 active+clean; 39 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 439 op/s 2024-05-08T05:32:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:32:59 smithi096 ceph-mon[34253]: pgmap v105: 129 pgs: 129 active+clean; 39 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 439 op/s 2024-05-08T05:32:59.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:32:59 smithi059 ceph-mon[30304]: pgmap v105: 129 pgs: 129 active+clean; 39 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.6 MiB/s wr, 439 op/s 2024-05-08T05:33:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:01 smithi143 ceph-mon[34337]: pgmap v106: 129 pgs: 129 active+clean; 36 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 486 op/s 2024-05-08T05:33:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:01 smithi096 ceph-mon[34253]: pgmap v106: 129 pgs: 129 active+clean; 36 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 486 op/s 2024-05-08T05:33:01.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:01 smithi059 ceph-mon[30304]: pgmap v106: 129 pgs: 129 active+clean; 36 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 486 op/s 2024-05-08T05:33:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:03 smithi143 ceph-mon[34337]: pgmap v107: 129 pgs: 129 active+clean; 36 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-05-08T05:33:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:03 smithi096 ceph-mon[34253]: pgmap v107: 129 pgs: 129 active+clean; 36 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-05-08T05:33:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:03 smithi059 ceph-mon[30304]: pgmap v107: 129 pgs: 129 active+clean; 36 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-05-08T05:33:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:05 smithi143 ceph-mon[34337]: pgmap v108: 129 pgs: 129 active+clean; 36 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-05-08T05:33:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:05 smithi096 ceph-mon[34253]: pgmap v108: 129 pgs: 129 active+clean; 36 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-05-08T05:33:05.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:05 smithi059 ceph-mon[30304]: pgmap v108: 129 pgs: 129 active+clean; 36 MiB data, 704 MiB used, 1.0 TiB / 1.0 TiB avail; 832 KiB/s rd, 1.2 MiB/s wr, 209 op/s 2024-05-08T05:33:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:06 smithi143 ceph-mon[34337]: pgmap v109: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-05-08T05:33:07.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:06 smithi096 ceph-mon[34253]: pgmap v109: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-05-08T05:33:07.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:06 smithi059 ceph-mon[30304]: pgmap v109: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 846 KiB/s rd, 1.2 MiB/s wr, 229 op/s 2024-05-08T05:33:09.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:08 smithi143 ceph-mon[34337]: pgmap v110: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 99 KiB/s wr, 67 op/s 2024-05-08T05:33:09.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:08 smithi096 ceph-mon[34253]: pgmap v110: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 99 KiB/s wr, 67 op/s 2024-05-08T05:33:09.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:08 smithi059 ceph-mon[30304]: pgmap v110: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 93 KiB/s rd, 99 KiB/s wr, 67 op/s 2024-05-08T05:33:11.247 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:10 smithi143 ceph-mon[34337]: pgmap v111: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 102 KiB/s wr, 73 op/s 2024-05-08T05:33:11.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:10 smithi096 ceph-mon[34253]: pgmap v111: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 102 KiB/s wr, 73 op/s 2024-05-08T05:33:11.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:10 smithi059 ceph-mon[30304]: pgmap v111: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 97 KiB/s rd, 102 KiB/s wr, 73 op/s 2024-05-08T05:33:11.369 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.368+0000 7fc25d9e4640 1 -- 172.21.15.59:0/1017303002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581078e0 msgr2=0x7fc258107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.368+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/1017303002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581078e0 0x7fc258107cc0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7fc248009930 tx=0x7fc24802a420 comp rx=0 tx=0).stop 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.368+0000 7fc25d9e4640 1 -- 172.21.15.59:0/1017303002 shutdown_connections 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.368+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/1017303002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc258102290 0x7fc25810e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.368+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/1017303002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc2581018d0 0x7fc258101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.368+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/1017303002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581078e0 0x7fc258107cc0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.368+0000 7fc25d9e4640 1 -- 172.21.15.59:0/1017303002 >> 172.21.15.59:0/1017303002 conn(0x7fc258077c30 msgr2=0x7fc258078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.369+0000 7fc25d9e4640 1 -- 172.21.15.59:0/1017303002 shutdown_connections 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.369+0000 7fc25d9e4640 1 -- 172.21.15.59:0/1017303002 wait complete. 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.369+0000 7fc25d9e4640 1 Processor -- start 2024-05-08T05:33:11.370 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.369+0000 7fc25d9e4640 1 -- start start 2024-05-08T05:33:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc25d9e4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581018d0 0x7fc258199f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc25d9e4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc258102290 0x7fc25819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc25d9e4640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2581078e0 0x7fc2581a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc25d9e4640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc258111ba0 con 0x7fc2581018d0 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc25d9e4640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc258111a20 con 0x7fc258102290 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc25d9e4640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc258111d20 con 0x7fc2581078e0 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc256ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581018d0 0x7fc258199f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc256ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581018d0 0x7fc258199f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52144/0 (socket says 172.21.15.59:52144) 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc256ffd640 1 -- 172.21.15.59:0/611014907 learned_addr learned my addr 172.21.15.59:0/611014907 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc2577fe640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2581078e0 0x7fc2581a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc258102290 0x7fc25819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc256ffd640 1 -- 172.21.15.59:0/611014907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2581078e0 msgr2=0x7fc2581a4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc256ffd640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2581078e0 0x7fc2581a4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc256ffd640 1 -- 172.21.15.59:0/611014907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc258102290 msgr2=0x7fc25819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc256ffd640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc258102290 0x7fc25819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc256ffd640 1 -- 172.21.15.59:0/611014907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2581a5520 con 0x7fc2581018d0 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc258102290 0x7fc25819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.370+0000 7fc256ffd640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581018d0 0x7fc258199f60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc248005940 tx=0x7fc2480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:11.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.371+0000 7fc25c9e2640 1 -- 172.21.15.59:0/611014907 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc24803f030 con 0x7fc2581018d0 2024-05-08T05:33:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.371+0000 7fc25c9e2640 1 -- 172.21.15.59:0/611014907 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc248003cb0 con 0x7fc2581018d0 2024-05-08T05:33:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.371+0000 7fc25c9e2640 1 -- 172.21.15.59:0/611014907 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc248035070 con 0x7fc2581018d0 2024-05-08T05:33:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.371+0000 7fc25d9e4640 1 -- 172.21.15.59:0/611014907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2581a56f0 con 0x7fc2581018d0 2024-05-08T05:33:11.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.371+0000 7fc25d9e4640 1 -- 172.21.15.59:0/611014907 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2581a5b00 con 0x7fc2581018d0 2024-05-08T05:33:11.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.373+0000 7fc2607d9740 1 Processor -- start 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.373+0000 7fc25c9e2640 1 -- 172.21.15.59:0/611014907 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc248042030 con 0x7fc2581018d0 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.373+0000 7fc2607d9740 1 -- start start 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 0x561d699492f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d69953bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699564e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc2607d9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d698ae5d0 con 0x561d699540f0 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc2607d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561d69867890 con 0x561d69949830 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc2607d9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561d698aa590 con 0x561d69948f10 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc24e7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699564e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc24e7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699564e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52148/0 (socket says 172.21.15.59:52148) 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc24e7fc640 1 -- 172.21.15.59:0/1209332540 learned_addr learned my addr 172.21.15.59:0/1209332540 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:11.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc2577fe640 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 0x561d699492f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc256ffd640 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d69953bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc25c9e2640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc2440b2a10 0x7fc2440b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc24e7fc640 1 -- 172.21.15.59:0/1209332540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 msgr2=0x561d699492f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 0x561d699492f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc24e7fc640 1 -- 172.21.15.59:0/1209332540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 msgr2=0x561d69953bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d69953bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc24e7fc640 1 -- 172.21.15.59:0/1209332540 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d695c0210 con 0x561d699540f0 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc256ffd640 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d69953bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc2577fe640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc2440b2a10 0x7fc2440b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc25c9e2640 1 -- 172.21.15.59:0/611014907 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc248103170 con 0x7fc2581018d0 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.374+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699564e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc240009c20 tx=0x7fc24000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa70e099879234e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc24dffb640 1 -- 172.21.15.59:0/1209332540 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240016070 con 0x561d699540f0 2024-05-08T05:33:11.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc24dffb640 1 -- 172.21.15.59:0/1209332540 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc240011920 con 0x561d699540f0 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc2577fe640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc2440b2a10 0x7fc2440b4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc250005c40 tx=0x7fc250004a40 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc24dffb640 1 -- 172.21.15.59:0/1209332540 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2400107f0 con 0x561d699540f0 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc2607d9740 1 -- 172.21.15.59:0/1209332540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 msgr2=0x561d699564e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc2607d9740 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699564e0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7fc240009c20 tx=0x7fc24000e6c0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc2607d9740 1 -- 172.21.15.59:0/1209332540 shutdown_connections 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc2607d9740 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699564e0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc2607d9740 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d69953bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc2607d9740 1 --2- 172.21.15.59:0/1209332540 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 0x561d699492f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.375+0000 7fc2607d9740 1 -- 172.21.15.59:0/1209332540 >> 172.21.15.59:0/1209332540 conn(0x561d69945420 msgr2=0x561d69945800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 -- 172.21.15.59:0/1209332540 shutdown_connections 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 -- 172.21.15.59:0/1209332540 wait complete. 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 -- start start 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 Processor -- start 2024-05-08T05:33:11.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 0x561d69950ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d69949830 0x561d69951530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d699540f0 0x561d69951a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561d69956d20 con 0x561d69949830 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561d6986b370 con 0x561d69948f10 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561d69956ea0 con 0x561d699540f0 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.376+0000 7fc2607d9740 10 client.0 _ll_register_callbacks cb 0x561d695bfa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc24e7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d69949830 0x561d69951530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc24e7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d69949830 0x561d69951530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52158/0 (socket says 172.21.15.59:52158) 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc24e7fc640 1 -- 172.21.15.59:0/560110856 learned_addr learned my addr 172.21.15.59:0/560110856 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc256ffd640 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 0x561d69950ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc2577fe640 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d699540f0 0x561d69951a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc24e7fc640 1 -- 172.21.15.59:0/560110856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d699540f0 msgr2=0x561d69951a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:11.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d699540f0 0x561d69951a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc24e7fc640 1 -- 172.21.15.59:0/560110856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 msgr2=0x561d69950ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 0x561d69950ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc24e7fc640 1 -- 172.21.15.59:0/560110856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d6968c620 con 0x561d69949830 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc2577fe640 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d699540f0 0x561d69951a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc256ffd640 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 0x561d69950ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d69949830 0x561d69951530 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc24000ef30 tx=0x7fc2400097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7cb7eed82cd7fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc236ffd640 1 -- 172.21.15.59:0/560110856 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc240027020 con 0x561d69949830 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc236ffd640 1 -- 172.21.15.59:0/560110856 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc240004020 con 0x561d69949830 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc2607d9740 1 -- 172.21.15.59:0/560110856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc240011920 con 0x561d69949830 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc236ffd640 1 -- 172.21.15.59:0/560110856 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc24002f050 con 0x561d69949830 2024-05-08T05:33:11.385 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc2607d9740 10 client.15906 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.377+0000 7fc2607d9740 1 -- 172.21.15.59:0/560110856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561d69962ba0 con 0x561d69949830 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.378+0000 7fc236ffd640 1 -- 172.21.15.59:0/560110856 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc240011920 con 0x561d69949830 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.378+0000 7fc236ffd640 1 -- 172.21.15.59:0/560110856 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fc24000f040 con 0x561d69949830 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.378+0000 7fc2607d9740 10 client.15906 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.378+0000 7fc2607d9740 10 client.15906 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.378+0000 7fc2607d9740 20 client.15906 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.378+0000 7fc2607d9740 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561d69964000 0x561d699663f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.378+0000 7fc2607d9740 4 client.15906 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.378+0000 7fc2607d9740 1 -- 172.21.15.59:0/560110856 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d698bc1e0 con 0x561d69964000 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.378+0000 7fc256ffd640 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561d69964000 0x561d699663f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.386 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.379+0000 7fc256ffd640 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561d69964000 0x561d699663f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87b1772409463323 server_cookie=d9ea357dd7166070 in_seq=0 out_seq=0 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.379+0000 7fc236ffd640 10 client.15906 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.379+0000 7fc236ffd640 1 -- 172.21.15.59:0/560110856 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561d698bc1e0 con 0x561d69964000 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.379+0000 7fc236ffd640 10 client.15906 handle_command_reply: tid=0 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.379+0000 7fc2607d9740 1 client.15906 shutdown 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 -- 172.21.15.59:0/560110856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d69949830 msgr2=0x561d69951530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d69949830 0x561d69951530 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7fc24000ef30 tx=0x7fc2400097d0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 -- 172.21.15.59:0/560110856 shutdown_connections 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561d69964000 0x561d699663f0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d699540f0 0x561d69951a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.387 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d69949830 0x561d69951530 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc236ffd640 0 client.15906 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 --2- 172.21.15.59:0/560110856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 0x561d69950ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 -- 172.21.15.59:0/560110856 >> 172.21.15.59:0/560110856 conn(0x561d69945420 msgr2=0x561d6994b150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 -- 172.21.15.59:0/560110856 shutdown_connections 2024-05-08T05:33:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 1 -- 172.21.15.59:0/560110856 wait complete. 2024-05-08T05:33:11.388 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.380+0000 7fc2607d9740 20 client.15906 trim_cache size 0 max 16384 2024-05-08T05:33:11.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.395+0000 7fc2607d9740 1 Processor -- start 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.395+0000 7fc2607d9740 1 -- start start 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.395+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 0x561d6995c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.395+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d6996bae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.395+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d6996c020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.395+0000 7fc2607d9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581afc40 con 0x561d699540f0 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.395+0000 7fc2607d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581b1810 con 0x561d69949830 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.395+0000 7fc2607d9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581b0310 con 0x561d69948f10 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc2577fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d6996c020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc2577fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d6996c020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52168/0 (socket says 172.21.15.59:52168) 2024-05-08T05:33:11.404 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc2577fe640 1 -- 172.21.15.59:0/2856477217 learned_addr learned my addr 172.21.15.59:0/2856477217 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d6996bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc256ffd640 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 0x561d6995c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc2577fe640 1 -- 172.21.15.59:0/2856477217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 msgr2=0x561d6995c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc2577fe640 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 0x561d6995c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc2577fe640 1 -- 172.21.15.59:0/2856477217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 msgr2=0x561d6996bae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc2577fe640 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d6996bae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc2577fe640 1 -- 172.21.15.59:0/2856477217 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d69949620 con 0x561d699540f0 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d6996bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc256ffd640 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 0x561d6995c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc2577fe640 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d6996c020 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc25000f7c0 tx=0x7fc250019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7ce3a57897b63b4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc24d7fa640 1 -- 172.21.15.59:0/2856477217 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc25002b070 con 0x561d699540f0 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc24d7fa640 1 -- 172.21.15.59:0/2856477217 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25001bac0 con 0x561d699540f0 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.396+0000 7fc24d7fa640 1 -- 172.21.15.59:0/2856477217 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250026a10 con 0x561d699540f0 2024-05-08T05:33:11.405 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 -- 172.21.15.59:0/2856477217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 msgr2=0x561d6996c020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d6996c020 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc25000f7c0 tx=0x7fc250019000 comp rx=0 tx=0).stop 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 -- 172.21.15.59:0/2856477217 shutdown_connections 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d6996c020 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69949830 0x561d6996bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2856477217 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69948f10 0x561d6995c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 -- 172.21.15.59:0/2856477217 >> 172.21.15.59:0/2856477217 conn(0x561d69945420 msgr2=0x561d699530b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 -- 172.21.15.59:0/2856477217 shutdown_connections 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 -- 172.21.15.59:0/2856477217 wait complete. 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.397+0000 7fc2607d9740 1 -- start start 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2607d9740 1 Processor -- start 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 0x561d699c6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69949830 0x561d699c6d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2607d9740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699cb710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2607d9740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581b10b0 con 0x561d699540f0 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2607d9740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581b09e0 con 0x561d69948f10 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2607d9740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc2581adc00 con 0x561d69949830 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2607d9740 10 client.0 _ll_register_callbacks cb 0x7fc250009b30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2577fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2577fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699cb710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52178/0 (socket says 172.21.15.59:52178) 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2577fe640 1 -- 172.21.15.59:0/2825191562 learned_addr learned my addr 172.21.15.59:0/2825191562 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc24e7fc640 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69949830 0x561d699c6d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc256ffd640 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 0x561d699c6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2577fe640 1 -- 172.21.15.59:0/2825191562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69949830 msgr2=0x561d699c6d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2577fe640 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69949830 0x561d699c6d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2577fe640 1 -- 172.21.15.59:0/2825191562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 msgr2=0x561d699c6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2577fe640 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 0x561d699c6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.407 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.398+0000 7fc2577fe640 1 -- 172.21.15.59:0/2825191562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d699cbd50 con 0x561d699540f0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2577fe640 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699cb710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc250011fd0 tx=0x7fc250009f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f398587d5f6892d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc236ffd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc236ffd640 1 -- 172.21.15.59:0/2825191562 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc250033070 con 0x561d699540f0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2607d9740 1 -- 172.21.15.59:0/2825191562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc25001bac0 con 0x561d699540f0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc236ffd640 1 -- 172.21.15.59:0/2825191562 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc250004d70 con 0x561d699540f0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc236ffd640 1 -- 172.21.15.59:0/2825191562 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7fc25003f050 con 0x561d699540f0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2607d9740 10 client.15912 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2607d9740 1 -- 172.21.15.59:0/2825191562 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x561d699d3160 con 0x561d699540f0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc236ffd640 1 -- 172.21.15.59:0/2825191562 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc25001bac0 con 0x561d699540f0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc236ffd640 1 -- 172.21.15.59:0/2825191562 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7fc250026bb0 con 0x561d699540f0 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2607d9740 10 client.15912 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2607d9740 10 client.15912 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:33:11.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2607d9740 20 client.15912 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561d699d38a0 0x561d699d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2607d9740 4 client.15912 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:33:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.399+0000 7fc2607d9740 1 -- 172.21.15.59:0/2825191562 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561d698b6050 con 0x561d699d38a0 2024-05-08T05:33:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.400+0000 7fc256ffd640 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561d699d38a0 0x561d699d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.400+0000 7fc236ffd640 10 client.15912 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:11.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.400+0000 7fc256ffd640 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561d699d38a0 0x561d699d5cc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6540dacb74b7116e server_cookie=935be2029d38feeb in_seq=0 out_seq=0 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc236ffd640 1 -- 172.21.15.59:0/2825191562 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561d698b6050 con 0x561d699d38a0 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc236ffd640 10 client.15912 handle_command_reply: tid=0 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 client.15912 shutdown 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 -- 172.21.15.59:0/2825191562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 msgr2=0x561d699cb710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699cb710 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7fc250011fd0 tx=0x7fc250009f00 comp rx=0 tx=0).stop 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 -- 172.21.15.59:0/2825191562 shutdown_connections 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561d699d38a0 0x561d699d5cc0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561d699540f0 0x561d699cb710 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561d69949830 0x561d699c6d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.422 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 --2- 172.21.15.59:0/2825191562 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561d69948f10 0x561d699c6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc236ffd640 0 client.15912 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 -- 172.21.15.59:0/2825191562 >> 172.21.15.59:0/2825191562 conn(0x561d69945420 msgr2=0x561d69943cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 -- 172.21.15.59:0/2825191562 shutdown_connections 2024-05-08T05:33:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 1 -- 172.21.15.59:0/2825191562 wait complete. 2024-05-08T05:33:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.421+0000 7fc2607d9740 20 client.15912 trim_cache size 0 max 16384 2024-05-08T05:33:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:33:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:33:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:33:11.423 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:33:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 -- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc2440b2a10 msgr2=0x7fc2440b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc2440b2a10 0x7fc2440b4ed0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fc250005c40 tx=0x7fc250004a40 comp rx=0 tx=0).stop 2024-05-08T05:33:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 -- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581018d0 msgr2=0x7fc258199f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:11.424 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581018d0 0x7fc258199f60 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc248005940 tx=0x7fc2480042f0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 -- 172.21.15.59:0/611014907 shutdown_connections 2024-05-08T05:33:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc2440b2a10 0x7fc2440b4ed0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc2581078e0 0x7fc2581a4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc258102290 0x7fc25819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 --2- 172.21.15.59:0/611014907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc2581018d0 0x7fc258199f60 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 -- 172.21.15.59:0/611014907 >> 172.21.15.59:0/611014907 conn(0x7fc258077c30 msgr2=0x7fc2580ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 -- 172.21.15.59:0/611014907 shutdown_connections 2024-05-08T05:33:11.425 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:11.422+0000 7fc25d9e4640 1 -- 172.21.15.59:0/611014907 wait complete. 2024-05-08T05:33:13.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:12 smithi143 ceph-mon[34337]: pgmap v112: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-05-08T05:33:13.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:12 smithi096 ceph-mon[34253]: pgmap v112: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-05-08T05:33:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:12 smithi059 ceph-mon[30304]: pgmap v112: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-05-08T05:33:15.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:14 smithi143 ceph-mon[34337]: pgmap v113: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-05-08T05:33:15.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:14 smithi096 ceph-mon[34253]: pgmap v113: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-05-08T05:33:15.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:14 smithi059 ceph-mon[30304]: pgmap v113: 129 pgs: 129 active+clean; 36 MiB data, 708 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 27 KiB/s wr, 25 op/s 2024-05-08T05:33:17.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:16 smithi143 ceph-mon[34337]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 35 op/s 2024-05-08T05:33:17.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:16 smithi096 ceph-mon[34253]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 35 op/s 2024-05-08T05:33:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:16 smithi059 ceph-mon[30304]: pgmap v114: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 56 KiB/s wr, 35 op/s 2024-05-08T05:33:17.753 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:33:18.023 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:33:19.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:18 smithi143 ceph-mon[34337]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-05-08T05:33:19.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:18 smithi096 ceph-mon[34253]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-05-08T05:33:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:18 smithi059 ceph-mon[30304]: pgmap v115: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 32 KiB/s wr, 15 op/s 2024-05-08T05:33:19.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.770+0000 7eff1e7bc700 1 -- 172.21.15.59:0/35829541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806b180 msgr2=0x7eff1806b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:19.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.770+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/35829541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806b180 0x7eff1806b600 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7eff0c009990 tx=0x7eff0c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:33:19.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.770+0000 7eff1e7bc700 1 -- 172.21.15.59:0/35829541 shutdown_connections 2024-05-08T05:33:19.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.770+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/35829541 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff1806bb40 0x7eff18110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:19.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.770+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/35829541 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff18109510 0x7eff181098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:19.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.770+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/35829541 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806b180 0x7eff1806b600 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:19.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.770+0000 7eff1e7bc700 1 -- 172.21.15.59:0/35829541 >> 172.21.15.59:0/35829541 conn(0x7eff18079350 msgr2=0x7eff18079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.770+0000 7eff1e7bc700 1 -- 172.21.15.59:0/35829541 shutdown_connections 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.770+0000 7eff1e7bc700 1 -- 172.21.15.59:0/35829541 wait complete. 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.771+0000 7eff1e7bc700 1 Processor -- start 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.771+0000 7eff1e7bc700 1 -- start start 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.771+0000 7eff1e7bc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff1806b180 0x7eff1819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.771+0000 7eff1e7bc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806bb40 0x7eff1819fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.771+0000 7eff1e7bc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff18109510 0x7eff18199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff1e7bc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7eff18113640 con 0x7eff1806bb40 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff1e7bc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7eff181134c0 con 0x7eff18109510 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff1e7bc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7eff181137c0 con 0x7eff1806b180 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff177fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806bb40 0x7eff1819fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff177fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806bb40 0x7eff1819fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48094/0 (socket says 172.21.15.59:48094) 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff177fe700 1 -- 172.21.15.59:0/677213120 learned_addr learned my addr 172.21.15.59:0/677213120 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff17fff700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff1806b180 0x7eff1819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:19.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff1cd59700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff18109510 0x7eff18199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:19.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff177fe700 1 -- 172.21.15.59:0/677213120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff1806b180 msgr2=0x7eff1819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:19.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff177fe700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff1806b180 0x7eff1819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:19.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff177fe700 1 -- 172.21.15.59:0/677213120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff18109510 msgr2=0x7eff18199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:19.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff177fe700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff18109510 0x7eff18199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:19.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff177fe700 1 -- 172.21.15.59:0/677213120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff1819a0e0 con 0x7eff1806bb40 2024-05-08T05:33:19.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff177fe700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806bb40 0x7eff1819fd00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0057c0 tx=0x7eff0c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:19.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff157fa700 1 -- 172.21.15.59:0/677213120 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c027070 con 0x7eff1806bb40 2024-05-08T05:33:19.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff157fa700 1 -- 172.21.15.59:0/677213120 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff0c003f20 con 0x7eff1806bb40 2024-05-08T05:33:19.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff1819a370 con 0x7eff1806bb40 2024-05-08T05:33:19.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.772+0000 7eff157fa700 1 -- 172.21.15.59:0/677213120 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff0c022b30 con 0x7eff1806bb40 2024-05-08T05:33:19.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.773+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff181a6690 con 0x7eff1806bb40 2024-05-08T05:33:19.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.774+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff1806a190 con 0x7eff1806bb40 2024-05-08T05:33:19.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.775+0000 7eff157fa700 1 -- 172.21.15.59:0/677213120 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7eff0c02a070 con 0x7eff1806bb40 2024-05-08T05:33:19.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.776+0000 7eff157fa700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7eff000b29f0 0x7eff000b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:19.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.776+0000 7eff157fa700 1 -- 172.21.15.59:0/677213120 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7eff0c0e38a0 con 0x7eff1806bb40 2024-05-08T05:33:19.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.776+0000 7eff17fff700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7eff000b29f0 0x7eff000b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:19.777 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.776+0000 7eff17fff700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7eff000b29f0 0x7eff000b4eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7eff1819aac0 tx=0x7eff080073d0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:19.781 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:19.781+0000 7eff157fa700 1 -- 172.21.15.59:0/677213120 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7eff0c013070 con 0x7eff1806bb40 2024-05-08T05:33:20.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.045+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7eff1802e510 con 0x7eff1806bb40 2024-05-08T05:33:20.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.046+0000 7eff157fa700 1 -- 172.21.15.59:0/677213120 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11934 (secure 0 0 0) 0x7eff0c0aafa0 con 0x7eff1806bb40 2024-05-08T05:33:20.051 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:33:20.051 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:32:11.760389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7eff000b29f0 msgr2=0x7eff000b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7eff000b29f0 0x7eff000b4eb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7eff1819aac0 tx=0x7eff080073d0 comp rx=0 tx=0).stop 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806bb40 msgr2=0x7eff1819fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806bb40 0x7eff1819fd00 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7eff0c0057c0 tx=0x7eff0c0057f0 comp rx=0 tx=0).stop 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 shutdown_connections 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7eff000b29f0 0x7eff000b4eb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7eff18109510 0x7eff18199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7eff1806b180 0x7eff1819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 --2- 172.21.15.59:0/677213120 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7eff1806bb40 0x7eff1819fd00 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 >> 172.21.15.59:0/677213120 conn(0x7eff18079350 msgr2=0x7eff181010d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:20.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 shutdown_connections 2024-05-08T05:33:20.053 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:20.049+0000 7eff1e7bc700 1 -- 172.21.15.59:0/677213120 wait complete. 2024-05-08T05:33:20.053 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-05-08T05:33:20.855 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db --set-id bc67f892 --release --await' 2024-05-08T05:33:20.855 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db --set-id bc67f892 --release --await 2024-05-08T05:33:21.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:20 smithi059 ceph-mon[30304]: pgmap v116: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-05-08T05:33:21.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:20 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/677213120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:33:21.121 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:33:21.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:20 smithi143 ceph-mon[34337]: pgmap v116: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-05-08T05:33:21.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:20 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/677213120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:33:21.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:20 smithi096 ceph-mon[34253]: pgmap v116: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 34 KiB/s wr, 19 op/s 2024-05-08T05:33:21.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:20 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/677213120' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:33:22.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.878+0000 7f4332205700 1 -- 172.21.15.59:0/4203394802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c108870 msgr2=0x7f432c1103f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:22.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.878+0000 7f4332205700 1 --2- 172.21.15.59:0/4203394802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c108870 0x7f432c1103f0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f432400b0d0 tx=0x7f432400b3e0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.878+0000 7f4332205700 1 -- 172.21.15.59:0/4203394802 shutdown_connections 2024-05-08T05:33:22.879 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.878+0000 7f4332205700 1 --2- 172.21.15.59:0/4203394802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c108870 0x7f432c1103f0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.878+0000 7f4332205700 1 --2- 172.21.15.59:0/4203394802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432c103780 0x7f432c103b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.878+0000 7f4332205700 1 --2- 172.21.15.59:0/4203394802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f432c104130 0x7f432c108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.878+0000 7f4332205700 1 -- 172.21.15.59:0/4203394802 >> 172.21.15.59:0/4203394802 conn(0x7f432c07a610 msgr2=0x7f432c07aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:22.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.879+0000 7f4332205700 1 -- 172.21.15.59:0/4203394802 shutdown_connections 2024-05-08T05:33:22.880 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.879+0000 7f4332205700 1 -- 172.21.15.59:0/4203394802 wait complete. 2024-05-08T05:33:22.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.879+0000 7f4332205700 1 Processor -- start 2024-05-08T05:33:22.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f4332205700 1 -- start start 2024-05-08T05:33:22.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f4332205700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c103780 0x7f432c0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.881 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f4332205700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432c104130 0x7f432c075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f4332205700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f432c108870 0x7f432c076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f4332205700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f432c113660 con 0x7f432c103780 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f4332205700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1134e0 con 0x7f432c108870 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f4332205700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1137e0 con 0x7f432c104130 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f432b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c103780 0x7f432c0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f432b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c103780 0x7f432c0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48106/0 (socket says 172.21.15.59:48106) 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f432b7fe700 1 -- 172.21.15.59:0/3030790773 learned_addr learned my addr 172.21.15.59:0/3030790773 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f432bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f432c108870 0x7f432c076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.880+0000 7f432affd700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432c104130 0x7f432c075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f432b7fe700 1 -- 172.21.15.59:0/3030790773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432c104130 msgr2=0x7f432c075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f432b7fe700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432c104130 0x7f432c075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f432b7fe700 1 -- 172.21.15.59:0/3030790773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f432c108870 msgr2=0x7f432c076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f432b7fe700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f432c108870 0x7f432c076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.882 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f432b7fe700 1 -- 172.21.15.59:0/3030790773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f432c1a7fc0 con 0x7f432c103780 2024-05-08T05:33:22.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f432bfff700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f432c108870 0x7f432c076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:33:22.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f432b7fe700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c103780 0x7f432c0757a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f431c00d980 tx=0x7f431c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:22.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f4328ff9700 1 -- 172.21.15.59:0/3030790773 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c017070 con 0x7f432c103780 2024-05-08T05:33:22.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f4332205700 1 -- 172.21.15.59:0/3030790773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f432c1a82b0 con 0x7f432c103780 2024-05-08T05:33:22.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f4328ff9700 1 -- 172.21.15.59:0/3030790773 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f431c005020 con 0x7f432c103780 2024-05-08T05:33:22.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f4328ff9700 1 -- 172.21.15.59:0/3030790773 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431c00fa10 con 0x7f432c103780 2024-05-08T05:33:22.883 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.881+0000 7f4332205700 1 -- 172.21.15.59:0/3030790773 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f432c1a87f0 con 0x7f432c103780 2024-05-08T05:33:22.884 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.884+0000 7f4328ff9700 1 -- 172.21.15.59:0/3030790773 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f431c00fbb0 con 0x7f432c103780 2024-05-08T05:33:22.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f4328ff9700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43140b29f0 0x7f43140b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f433ca98b80 1 Processor -- start 2024-05-08T05:33:22.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f432affd700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43140b29f0 0x7f43140b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f4328ff9700 1 -- 172.21.15.59:0/3030790773 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f431c0e2890 con 0x7f432c103780 2024-05-08T05:33:22.885 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f433ca98b80 1 -- start start 2024-05-08T05:33:22.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c727c0 0x55f005c72ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c6cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c79580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f433ca98b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f005bbea40 con 0x55f005c6d430 2024-05-08T05:33:22.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f433ca98b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f005b23ea0 con 0x55f005c6ca70 2024-05-08T05:33:22.887 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f433ca98b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f005bbaa00 con 0x55f005c727c0 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f432affd700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43140b29f0 0x7f43140b4eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f432c103070 tx=0x7f432000a300 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f432bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c79580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f432bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c79580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48112/0 (socket says 172.21.15.59:48112) 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f432bfff700 1 -- 172.21.15.59:0/3081635609 learned_addr learned my addr 172.21.15.59:0/3081635609 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f432affd700 1 --2- 172.21.15.59:0/3081635609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c6cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.885+0000 7f432b7fe700 1 --2- 172.21.15.59:0/3081635609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c727c0 0x55f005c72ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.886+0000 7f432bfff700 1 -- 172.21.15.59:0/3081635609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c727c0 msgr2=0x55f005c72ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.886+0000 7f432bfff700 1 --2- 172.21.15.59:0/3081635609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c727c0 0x55f005c72ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.886+0000 7f432bfff700 1 -- 172.21.15.59:0/3081635609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 msgr2=0x55f005c6cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.886+0000 7f432bfff700 1 --2- 172.21.15.59:0/3081635609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c6cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.886+0000 7f432bfff700 1 -- 172.21.15.59:0/3081635609 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f005b0b9d0 con 0x55f005c6d430 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.886+0000 7f432bfff700 1 --2- 172.21.15.59:0/3081635609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c79580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4324009b80 tx=0x7f4324009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fab2ff20904f342f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.886+0000 7f4311ffb700 1 -- 172.21.15.59:0/3081635609 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324003c00 con 0x55f005c6d430 2024-05-08T05:33:22.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.886+0000 7f4311ffb700 1 -- 172.21.15.59:0/3081635609 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4324003da0 con 0x55f005c6d430 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.886+0000 7f4311ffb700 1 -- 172.21.15.59:0/3081635609 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43240053c0 con 0x55f005c6d430 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 -- 172.21.15.59:0/3081635609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 msgr2=0x55f005c79580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 --2- 172.21.15.59:0/3081635609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c79580 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f4324009b80 tx=0x7f4324009bb0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 -- 172.21.15.59:0/3081635609 shutdown_connections 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 --2- 172.21.15.59:0/3081635609 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c79580 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 --2- 172.21.15.59:0/3081635609 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c727c0 0x55f005c72ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 --2- 172.21.15.59:0/3081635609 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c6cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 -- 172.21.15.59:0/3081635609 >> 172.21.15.59:0/3081635609 conn(0x55f005b98c70 msgr2=0x55f005c677f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 -- 172.21.15.59:0/3081635609 shutdown_connections 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 -- 172.21.15.59:0/3081635609 wait complete. 2024-05-08T05:33:22.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 -- start start 2024-05-08T05:33:22.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.887+0000 7f433ca98b80 1 Processor -- start 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c79070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 0x55f005c76460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c769a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f433ca98b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f005c79ac0 con 0x55f005c727c0 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f433ca98b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f005b77fd0 con 0x55f005c6ca70 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f433ca98b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f005c79c40 con 0x55f005c6d430 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f433ca98b80 10 client.0 _ll_register_callbacks cb 0x55f005b526b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c769a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c769a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48118/0 (socket says 172.21.15.59:48118) 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432bfff700 1 -- 172.21.15.59:0/4063152039 learned_addr learned my addr 172.21.15.59:0/4063152039 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432b7fe700 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c79070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432affd700 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 0x55f005c76460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432bfff700 1 -- 172.21.15.59:0/4063152039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 msgr2=0x55f005c76460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432bfff700 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 0x55f005c76460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432bfff700 1 -- 172.21.15.59:0/4063152039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 msgr2=0x55f005c79070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432bfff700 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c79070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432bfff700 1 -- 172.21.15.59:0/4063152039 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f005b75860 con 0x55f005c727c0 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f432bfff700 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c769a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4324007da0 tx=0x7f4324007dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c7efd64f861b624 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.888+0000 7f430affd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f430affd700 1 -- 172.21.15.59:0/4063152039 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4324007870 con 0x55f005c727c0 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f430affd700 1 -- 172.21.15.59:0/4063152039 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f4324007a10 con 0x55f005c727c0 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f433ca98b80 1 -- 172.21.15.59:0/4063152039 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4324003da0 con 0x55f005c727c0 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f430affd700 1 -- 172.21.15.59:0/4063152039 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f43240050c0 con 0x55f005c727c0 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f433ca98b80 10 client.15930 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f433ca98b80 1 -- 172.21.15.59:0/4063152039 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f005c840f0 con 0x55f005c727c0 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f430affd700 1 -- 172.21.15.59:0/4063152039 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4324003da0 con 0x55f005c727c0 2024-05-08T05:33:22.894 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f430affd700 1 -- 172.21.15.59:0/4063152039 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f4324011070 con 0x55f005c727c0 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f433ca98b80 10 client.15930 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f433ca98b80 10 client.15930 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f433ca98b80 20 client.15930 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f433ca98b80 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f005c865c0 0x55f005c889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f433ca98b80 4 client.15930 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.889+0000 7f433ca98b80 1 -- 172.21.15.59:0/4063152039 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f005bcc420 con 0x55f005c865c0 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.890+0000 7f432b7fe700 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f005c865c0 0x55f005c889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.890+0000 7f430affd700 10 client.15930 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.890+0000 7f432b7fe700 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f005c865c0 0x55f005c889b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62dd00d82b527417 server_cookie=e660fceeb97ae6e3 in_seq=0 out_seq=0 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f430affd700 1 -- 172.21.15.59:0/4063152039 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f005bcc420 con 0x55f005c865c0 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f430affd700 10 client.15930 handle_command_reply: tid=0 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 client.15930 shutdown 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 -- 172.21.15.59:0/4063152039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 msgr2=0x55f005c769a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c769a0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f4324007da0 tx=0x7f4324007dd0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.895 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 -- 172.21.15.59:0/4063152039 shutdown_connections 2024-05-08T05:33:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f005c865c0 0x55f005c889b0 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c769a0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f430affd700 0 client.15930 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c79070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 --2- 172.21.15.59:0/4063152039 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 0x55f005c76460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 -- 172.21.15.59:0/4063152039 >> 172.21.15.59:0/4063152039 conn(0x55f005b98c70 msgr2=0x55f005c71b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 -- 172.21.15.59:0/4063152039 shutdown_connections 2024-05-08T05:33:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 1 -- 172.21.15.59:0/4063152039 wait complete. 2024-05-08T05:33:22.896 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.891+0000 7f433ca98b80 20 client.15930 trim_cache size 0 max 16384 2024-05-08T05:33:22.912 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f433ca98b80 1 Processor -- start 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f433ca98b80 1 -- start start 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6ca70 0x55f005c754a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c71240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c727c0 0x55f005c71780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f433ca98b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1b15a0 con 0x55f005c6d430 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f433ca98b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1b3170 con 0x55f005c727c0 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f433ca98b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1b1c70 con 0x55f005c6ca70 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f432affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c71240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f432affd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c71240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48130/0 (socket says 172.21.15.59:48130) 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.911+0000 7f432affd700 1 -- 172.21.15.59:0/476572536 learned_addr learned my addr 172.21.15.59:0/476572536 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f432b7fe700 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6ca70 0x55f005c754a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.913 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f432bfff700 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c727c0 0x55f005c71780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f432affd700 1 -- 172.21.15.59:0/476572536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6ca70 msgr2=0x55f005c754a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f432affd700 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6ca70 0x55f005c754a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f432affd700 1 -- 172.21.15.59:0/476572536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c727c0 msgr2=0x55f005c71780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f432affd700 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c727c0 0x55f005c71780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f432affd700 1 -- 172.21.15.59:0/476572536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f005c62440 con 0x55f005c6d430 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f432bfff700 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c727c0 0x55f005c71780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f432affd700 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c71240 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f432c077230 tx=0x7f4320012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b80108ebb7eb6ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f43117fa700 1 -- 172.21.15.59:0/476572536 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320016070 con 0x55f005c6d430 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f43117fa700 1 -- 172.21.15.59:0/476572536 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4320004830 con 0x55f005c6d430 2024-05-08T05:33:22.914 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.912+0000 7f43117fa700 1 -- 172.21.15.59:0/476572536 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320005280 con 0x55f005c6d430 2024-05-08T05:33:22.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 -- 172.21.15.59:0/476572536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 msgr2=0x55f005c71240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:22.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c71240 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f432c077230 tx=0x7f4320012000 comp rx=0 tx=0).stop 2024-05-08T05:33:22.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 -- 172.21.15.59:0/476572536 shutdown_connections 2024-05-08T05:33:22.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c727c0 0x55f005c71780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6ca70 0x55f005c754a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 --2- 172.21.15.59:0/476572536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c6d430 0x55f005c71240 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 -- 172.21.15.59:0/476572536 >> 172.21.15.59:0/476572536 conn(0x55f005b98c70 msgr2=0x55f005c74d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:22.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 -- 172.21.15.59:0/476572536 shutdown_connections 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 -- 172.21.15.59:0/476572536 wait complete. 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.913+0000 7f433ca98b80 1 -- start start 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f433ca98b80 1 Processor -- start 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c8b9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 0x55f005c8bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f433ca98b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c889b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f433ca98b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1b2a10 con 0x55f005c727c0 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f433ca98b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1b2340 con 0x55f005c6ca70 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f433ca98b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f432c1af520 con 0x55f005c6d430 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f433ca98b80 10 client.0 _ll_register_callbacks cb 0x55f005c76350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c889b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c889b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48140/0 (socket says 172.21.15.59:48140) 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432bfff700 1 -- 172.21.15.59:0/785328526 learned_addr learned my addr 172.21.15.59:0/785328526 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:22.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432b7fe700 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c8b9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432affd700 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 0x55f005c8bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432bfff700 1 -- 172.21.15.59:0/785328526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 msgr2=0x55f005c8bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432bfff700 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 0x55f005c8bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432bfff700 1 -- 172.21.15.59:0/785328526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 msgr2=0x55f005c8b9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432bfff700 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c8b9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.914+0000 7f432bfff700 1 -- 172.21.15.59:0/785328526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f005c89060 con 0x55f005c727c0 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f432bfff700 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c889b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4324031ff0 tx=0x7f4324007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af8e09306df00f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f430affd700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f430affd700 1 -- 172.21.15.59:0/785328526 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432402f680 con 0x55f005c727c0 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f430affd700 1 -- 172.21.15.59:0/785328526 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f432400f040 con 0x55f005c727c0 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f433ca98b80 1 -- 172.21.15.59:0/785328526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4320004830 con 0x55f005c727c0 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f430affd700 1 -- 172.21.15.59:0/785328526 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f4324007870 con 0x55f005c727c0 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f433ca98b80 10 client.15936 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:22.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f433ca98b80 1 -- 172.21.15.59:0/785328526 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55f005cf1430 con 0x55f005c727c0 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f430affd700 1 -- 172.21.15.59:0/785328526 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4320004830 con 0x55f005c727c0 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f430affd700 1 -- 172.21.15.59:0/785328526 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f4324011070 con 0x55f005c727c0 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f433ca98b80 10 client.15936 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f433ca98b80 10 client.15936 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f433ca98b80 20 client.15936 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f433ca98b80 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f005c865c0 0x55f005cf3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f433ca98b80 4 client.15936 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bc67f892", "release": true, "await": true}] 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.915+0000 7f433ca98b80 1 -- 172.21.15.59:0/785328526 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bc67f892", "release": true, "await": true}) -- 0x55f005bc64c0 con 0x55f005c865c0 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.916+0000 7f432b7fe700 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f005c865c0 0x55f005cf3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.916+0000 7f430affd700 10 client.15936 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:22.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:22.916+0000 7f432b7fe700 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f005c865c0 0x55f005cf3a20 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70275f37911ca7d4 server_cookie=f2a9ae47793ec237 in_seq=0 out_seq=0 2024-05-08T05:33:23.140 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:22 smithi059 ceph-mon[30304]: pgmap v117: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-05-08T05:33:23.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.139+0000 7f430affd700 1 -- 172.21.15.59:0/785328526 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f005bc64c0 con 0x55f005c865c0 2024-05-08T05:33:23.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.139+0000 7f430affd700 10 client.15936 handle_command_reply: tid=0 2024-05-08T05:33:23.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 client.15936 shutdown 2024-05-08T05:33:23.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 -- 172.21.15.59:0/785328526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 msgr2=0x55f005c889b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:23.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c889b0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f4324031ff0 tx=0x7f4324007f30 comp rx=0 tx=0).stop 2024-05-08T05:33:23.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 -- 172.21.15.59:0/785328526 shutdown_connections 2024-05-08T05:33:23.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f005c865c0 0x55f005cf3a20 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:23.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f005c727c0 0x55f005c889b0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:23.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f005c6ca70 0x55f005c8b9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 --2- 172.21.15.59:0/785328526 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f005c6d430 0x55f005c8bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 -- 172.21.15.59:0/785328526 >> 172.21.15.59:0/785328526 conn(0x55f005b98c70 msgr2=0x55f005c71240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f430affd700 0 client.15936 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 -- 172.21.15.59:0/785328526 shutdown_connections 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 1 -- 172.21.15.59:0/785328526 wait complete. 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.140+0000 7f433ca98b80 20 client.15936 trim_cache size 0 max 16384 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 33, 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 12, 2024-05-08T05:33:23.142 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "bc67f892": { 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 12, 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 81.0, 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:33:23.143 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:33:23.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:23.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:23.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:23.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:23.144 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:23.144 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.141+0000 7f4332205700 1 -- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43140b29f0 msgr2=0x7f43140b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.141+0000 7f4332205700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43140b29f0 0x7f43140b4eb0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f432c103070 tx=0x7f432000a300 comp rx=0 tx=0).stop 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.141+0000 7f4332205700 1 -- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c103780 msgr2=0x7f432c0757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.141+0000 7f4332205700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c103780 0x7f432c0757a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f431c00d980 tx=0x7f431c00dc90 comp rx=0 tx=0).stop 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.142+0000 7f4332205700 1 -- 172.21.15.59:0/3030790773 shutdown_connections 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.142+0000 7f4332205700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43140b29f0 0x7f43140b4eb0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.142+0000 7f4332205700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f432c108870 0x7f432c076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.142+0000 7f4332205700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f432c103780 0x7f432c0757a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.142+0000 7f4332205700 1 --2- 172.21.15.59:0/3030790773 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f432c104130 0x7f432c075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.142+0000 7f4332205700 1 -- 172.21.15.59:0/3030790773 >> 172.21.15.59:0/3030790773 conn(0x7f432c07a610 msgr2=0x7f432c1016c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:23.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.142+0000 7f4332205700 1 -- 172.21.15.59:0/3030790773 shutdown_connections 2024-05-08T05:33:23.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:23.142+0000 7f4332205700 1 -- 172.21.15.59:0/3030790773 wait complete. 2024-05-08T05:33:23.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:22 smithi143 ceph-mon[34337]: pgmap v117: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-05-08T05:33:23.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:22 smithi096 ceph-mon[34253]: pgmap v117: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-05-08T05:33:24.044 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'bc67f892', seconds elapsed: 37.1 2024-05-08T05:33:24.044 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.05 2024-05-08T05:33:24.044 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:33:24.344 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:33:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.530+0000 7efd4d472640 1 -- 172.21.15.59:0/3332131252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 msgr2=0x7efd48101330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.530+0000 7efd4d472640 1 --2- 172.21.15.59:0/3332131252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 0x7efd48101330 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7efd3c009930 tx=0x7efd3c02a420 comp rx=0 tx=0).stop 2024-05-08T05:33:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.530+0000 7efd4d472640 1 -- 172.21.15.59:0/3332131252 shutdown_connections 2024-05-08T05:33:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.530+0000 7efd4d472640 1 --2- 172.21.15.59:0/3332131252 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd48101870 0x7efd4810f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.530+0000 7efd4d472640 1 --2- 172.21.15.59:0/3332131252 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 0x7efd48101330 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.530+0000 7efd4d472640 1 --2- 172.21.15.59:0/3332131252 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd48107990 0x7efd48107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.530+0000 7efd4d472640 1 -- 172.21.15.59:0/3332131252 >> 172.21.15.59:0/3332131252 conn(0x7efd48077e40 msgr2=0x7efd480ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.530+0000 7efd4d472640 1 -- 172.21.15.59:0/3332131252 shutdown_connections 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.530+0000 7efd4d472640 1 -- 172.21.15.59:0/3332131252 wait complete. 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd4d472640 1 Processor -- start 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd4d472640 1 -- start start 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd4d472640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 0x7efd48102090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd4d472640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd48101870 0x7efd481025d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd4d472640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd48107990 0x7efd48102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd4d472640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd48112d10 con 0x7efd48100eb0 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd4d472640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd48112b90 con 0x7efd48101870 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd4d472640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efd48112e90 con 0x7efd48107990 2024-05-08T05:33:24.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd46ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 0x7efd48102090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd46ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 0x7efd48102090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48154/0 (socket says 172.21.15.59:48154) 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd46ffd640 1 -- 172.21.15.59:0/4143611851 learned_addr learned my addr 172.21.15.59:0/4143611851 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd467fc640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd48101870 0x7efd481025d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.531+0000 7efd477fe640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd48107990 0x7efd48102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd46ffd640 1 -- 172.21.15.59:0/4143611851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd48107990 msgr2=0x7efd48102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd46ffd640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd48107990 0x7efd48102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd46ffd640 1 -- 172.21.15.59:0/4143611851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd48101870 msgr2=0x7efd481025d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd46ffd640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd48101870 0x7efd481025d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd46ffd640 1 -- 172.21.15.59:0/4143611851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd481a7670 con 0x7efd48100eb0 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd477fe640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd48107990 0x7efd48102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd46ffd640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 0x7efd48102090 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efd3800e9f0 tx=0x7efd3800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd2ffff640 1 -- 172.21.15.59:0/4143611851 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3800ccc0 con 0x7efd48100eb0 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd2ffff640 1 -- 172.21.15.59:0/4143611851 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd380108e0 con 0x7efd48100eb0 2024-05-08T05:33:24.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd4d472640 1 -- 172.21.15.59:0/4143611851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd481a7960 con 0x7efd48100eb0 2024-05-08T05:33:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd2ffff640 1 -- 172.21.15.59:0/4143611851 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd38018880 con 0x7efd48100eb0 2024-05-08T05:33:24.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.532+0000 7efd4d472640 1 -- 172.21.15.59:0/4143611851 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd481a7e80 con 0x7efd48100eb0 2024-05-08T05:33:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.535+0000 7efd5018f740 1 Processor -- start 2024-05-08T05:33:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.535+0000 7efd5018f740 1 -- start start 2024-05-08T05:33:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.535+0000 7efd2ffff640 1 -- 172.21.15.59:0/4143611851 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efd38010430 con 0x7efd48100eb0 2024-05-08T05:33:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.535+0000 7efd5018f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58caaf10 0x562e58cab2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.535+0000 7efd5018f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58cb5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.538 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.535+0000 7efd5018f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cb60f0 0x562e58cb84e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.535+0000 7efd5018f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562e58c105d0 con 0x562e58cab830 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.535+0000 7efd5018f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e58bc9890 con 0x562e58cb60f0 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.535+0000 7efd5018f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562e58c0c590 con 0x562e58caaf10 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd477fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58caaf10 0x562e58cab2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd46ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58cb5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd467fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cb60f0 0x562e58cb84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd46ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58cb5bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48160/0 (socket says 172.21.15.59:48160) 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd46ffd640 1 -- 172.21.15.59:0/3813049002 learned_addr learned my addr 172.21.15.59:0/3813049002 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd477fe640 1 --2- 172.21.15.59:0/3813049002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58caaf10 0x562e58cab2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:40310/0 (socket says 172.21.15.59:40310) 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd467fc640 1 -- 172.21.15.59:0/3813049002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58caaf10 msgr2=0x562e58cab2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd467fc640 1 --2- 172.21.15.59:0/3813049002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58caaf10 0x562e58cab2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd467fc640 1 -- 172.21.15.59:0/3813049002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 msgr2=0x562e58cb5bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd467fc640 1 --2- 172.21.15.59:0/3813049002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58cb5bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.539 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd467fc640 1 -- 172.21.15.59:0/3813049002 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e58922210 con 0x562e58cb60f0 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd467fc640 1 --2- 172.21.15.59:0/3813049002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cb60f0 0x562e58cb84e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd3c031440 tx=0x7efd3c005be0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d64e1807e8e4009 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd2ffff640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efd300b2a40 0x7efd300b4f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd477fe640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efd300b2a40 0x7efd300b4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.536+0000 7efd2d7fa640 1 -- 172.21.15.59:0/3813049002 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c040070 con 0x562e58cb60f0 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.537+0000 7efd2d7fa640 1 -- 172.21.15.59:0/3813049002 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd3c004030 con 0x562e58cb60f0 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.537+0000 7efd2ffff640 1 -- 172.21.15.59:0/4143611851 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efd380e2c80 con 0x7efd48100eb0 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.537+0000 7efd2d7fa640 1 -- 172.21.15.59:0/3813049002 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c035040 con 0x562e58cb60f0 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 -- 172.21.15.59:0/3813049002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cb60f0 msgr2=0x562e58cb84e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd477fe640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efd300b2a40 0x7efd300b4f00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd40005fd0 tx=0x7efd40009500 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 --2- 172.21.15.59:0/3813049002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cb60f0 0x562e58cb84e0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd3c031440 tx=0x7efd3c005be0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 -- 172.21.15.59:0/3813049002 shutdown_connections 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 --2- 172.21.15.59:0/3813049002 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cb60f0 0x562e58cb84e0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.540 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 --2- 172.21.15.59:0/3813049002 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58cb5bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 --2- 172.21.15.59:0/3813049002 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58caaf10 0x562e58cab2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 -- 172.21.15.59:0/3813049002 >> 172.21.15.59:0/3813049002 conn(0x562e58ca7420 msgr2=0x562e58ca7800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 -- 172.21.15.59:0/3813049002 shutdown_connections 2024-05-08T05:33:24.541 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 -- 172.21.15.59:0/3813049002 wait complete. 2024-05-08T05:33:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 -- start start 2024-05-08T05:33:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.538+0000 7efd5018f740 1 Processor -- start 2024-05-08T05:33:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd5018f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cb3040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd5018f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cb3580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.548 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd5018f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58cb3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd5018f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562e58cb8cc0 con 0x562e58caaf10 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd5018f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562e58bcd370 con 0x562e58cab830 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd5018f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562e58cb8e40 con 0x562e58cb60f0 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd467fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cb3580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd467fc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cb3580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:50568/0 (socket says 172.21.15.59:50568) 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd467fc640 1 -- 172.21.15.59:0/1314205755 learned_addr learned my addr 172.21.15.59:0/1314205755 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd46ffd640 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cb3040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd5018f740 10 client.0 _ll_register_callbacks cb 0x562e58921a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd477fe640 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58cb3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd467fc640 1 -- 172.21.15.59:0/1314205755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 msgr2=0x562e58cb3ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd467fc640 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58cb3ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd467fc640 1 -- 172.21.15.59:0/1314205755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 msgr2=0x562e58cb3040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd467fc640 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cb3040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd467fc640 1 -- 172.21.15.59:0/1314205755 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e589ee620 con 0x562e58cab830 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd46ffd640 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cb3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd467fc640 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cb3580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd3c000c00 tx=0x7efd3c005090 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86cccfb7458105b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.539+0000 7efd2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd2aff5640 1 -- 172.21.15.59:0/1314205755 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c052060 con 0x562e58cab830 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd2aff5640 1 -- 172.21.15.59:0/1314205755 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efd3c0043f0 con 0x562e58cab830 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd5018f740 1 -- 172.21.15.59:0/1314205755 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd3c004030 con 0x562e58cab830 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd2aff5640 1 -- 172.21.15.59:0/1314205755 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efd3c059050 con 0x562e58cab830 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd5018f740 10 client.25568 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd5018f740 1 -- 172.21.15.59:0/1314205755 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562e58cc4b60 con 0x562e58cab830 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd2aff5640 1 -- 172.21.15.59:0/1314205755 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd3c004030 con 0x562e58cab830 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd2aff5640 1 -- 172.21.15.59:0/1314205755 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7efd3c0047a0 con 0x562e58cab830 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd5018f740 10 client.25568 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:24.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd5018f740 10 client.25568 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd5018f740 20 client.25568 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd5018f740 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562e58cc5eb0 0x562e58cc82a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd5018f740 4 client.25568 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.540+0000 7efd5018f740 1 -- 172.21.15.59:0/1314205755 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562e58c1e1e0 con 0x562e58cc5eb0 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.541+0000 7efd46ffd640 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562e58cc5eb0 0x562e58cc82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.541+0000 7efd2aff5640 10 client.25568 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.541+0000 7efd46ffd640 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562e58cc5eb0 0x562e58cc82a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a18a4a06f5dd6c44 server_cookie=2356f445b622d423 in_seq=0 out_seq=0 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.541+0000 7efd2aff5640 1 -- 172.21.15.59:0/1314205755 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562e58c1e1e0 con 0x562e58cc5eb0 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd2aff5640 10 client.25568 handle_command_reply: tid=0 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 client.25568 shutdown 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 -- 172.21.15.59:0/1314205755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 msgr2=0x562e58cb3580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cb3580 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7efd3c000c00 tx=0x7efd3c005090 comp rx=0 tx=0).stop 2024-05-08T05:33:24.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 -- 172.21.15.59:0/1314205755 shutdown_connections 2024-05-08T05:33:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562e58cc5eb0 0x562e58cc82a0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58cb3ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd2aff5640 0 client.25568 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cb3580 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 --2- 172.21.15.59:0/1314205755 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cb3040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 -- 172.21.15.59:0/1314205755 >> 172.21.15.59:0/1314205755 conn(0x562e58ca7420 msgr2=0x562e58cad1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 -- 172.21.15.59:0/1314205755 shutdown_connections 2024-05-08T05:33:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 1 -- 172.21.15.59:0/1314205755 wait complete. 2024-05-08T05:33:24.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.542+0000 7efd5018f740 20 client.25568 trim_cache size 0 max 16384 2024-05-08T05:33:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.560+0000 7efd5018f740 1 Processor -- start 2024-05-08T05:33:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd5018f740 1 -- start start 2024-05-08T05:33:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd5018f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58caaf10 0x562e58cad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd5018f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58ccd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd5018f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58ccde10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd5018f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b1f30 con 0x562e58cab830 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd5018f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b3b00 con 0x562e58caaf10 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd5018f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b2600 con 0x562e58cb60f0 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd467fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58ccd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd477fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58ccde10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd46ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58caaf10 0x562e58cad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd467fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58ccd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48180/0 (socket says 172.21.15.59:48180) 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd467fc640 1 -- 172.21.15.59:0/3014444209 learned_addr learned my addr 172.21.15.59:0/3014444209 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd477fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58ccde10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:40328/0 (socket says 172.21.15.59:40328) 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd467fc640 1 -- 172.21.15.59:0/3014444209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 msgr2=0x562e58ccde10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd467fc640 1 --2- 172.21.15.59:0/3014444209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58ccde10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd467fc640 1 -- 172.21.15.59:0/3014444209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58caaf10 msgr2=0x562e58cad840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd467fc640 1 --2- 172.21.15.59:0/3014444209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58caaf10 0x562e58cad840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd467fc640 1 -- 172.21.15.59:0/3014444209 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562e58cab620 con 0x562e58cab830 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd46ffd640 1 --2- 172.21.15.59:0/3014444209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58caaf10 0x562e58cad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:33:24.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.561+0000 7efd477fe640 1 --2- 172.21.15.59:0/3014444209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58ccde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:33:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.562+0000 7efd467fc640 1 --2- 172.21.15.59:0/3014444209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58ccd8d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efd3c02a460 tx=0x7efd3c001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daca201f2ce0d885 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.562+0000 7efd2cff9640 1 -- 172.21.15.59:0/3014444209 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c052070 con 0x562e58cab830 2024-05-08T05:33:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.562+0000 7efd2cff9640 1 -- 172.21.15.59:0/3014444209 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd3c0044b0 con 0x562e58cab830 2024-05-08T05:33:24.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.562+0000 7efd2cff9640 1 -- 172.21.15.59:0/3014444209 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd3c040070 con 0x562e58cab830 2024-05-08T05:33:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 -- 172.21.15.59:0/3014444209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 msgr2=0x562e58ccd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 --2- 172.21.15.59:0/3014444209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58ccd8d0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7efd3c02a460 tx=0x7efd3c001e70 comp rx=0 tx=0).stop 2024-05-08T05:33:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 -- 172.21.15.59:0/3014444209 shutdown_connections 2024-05-08T05:33:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 --2- 172.21.15.59:0/3014444209 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58ccde10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 --2- 172.21.15.59:0/3014444209 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58cab830 0x562e58ccd8d0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 --2- 172.21.15.59:0/3014444209 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58caaf10 0x562e58cad840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 -- 172.21.15.59:0/3014444209 >> 172.21.15.59:0/3014444209 conn(0x562e58ca7420 msgr2=0x562e58cb5100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 -- 172.21.15.59:0/3014444209 shutdown_connections 2024-05-08T05:33:24.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 -- 172.21.15.59:0/3014444209 wait complete. 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 -- start start 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.563+0000 7efd5018f740 1 Processor -- start 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd5018f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cbe090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd5018f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cbe5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd5018f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58cbb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd5018f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b33a0 con 0x562e58caaf10 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd5018f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efd481b2cd0 con 0x562e58cab830 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd5018f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efd481afef0 con 0x562e58cb60f0 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd5018f740 10 client.0 _ll_register_callbacks cb 0x7efd380e6640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd46ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cbe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd46ffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cbe090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48188/0 (socket says 172.21.15.59:48188) 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd46ffd640 1 -- 172.21.15.59:0/2995512976 learned_addr learned my addr 172.21.15.59:0/2995512976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd477fe640 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58cbb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd467fc640 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cbe5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd46ffd640 1 -- 172.21.15.59:0/2995512976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 msgr2=0x562e58cbb060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:24.568 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd46ffd640 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58cbb060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd46ffd640 1 -- 172.21.15.59:0/2995512976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 msgr2=0x562e58cbe5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd46ffd640 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cbe5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd46ffd640 1 -- 172.21.15.59:0/2995512976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562e58cbb5d0 con 0x562e58caaf10 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd467fc640 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cbe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd46ffd640 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cbe090 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efd380a9b80 tx=0x7efd3800ce60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677ca02a2ffd7c89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.564+0000 7efd2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd2aff5640 1 -- 172.21.15.59:0/2995512976 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd380a9360 con 0x562e58caaf10 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd2aff5640 1 -- 172.21.15.59:0/2995512976 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efd380a5e80 con 0x562e58caaf10 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd5018f740 1 -- 172.21.15.59:0/2995512976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efd3c0044b0 con 0x562e58caaf10 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd2aff5640 1 -- 172.21.15.59:0/2995512976 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efd380a5380 con 0x562e58caaf10 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd5018f740 10 client.15954 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd5018f740 1 -- 172.21.15.59:0/2995512976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x562e58d30d00 con 0x562e58caaf10 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd2aff5640 1 -- 172.21.15.59:0/2995512976 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd3c0044b0 con 0x562e58caaf10 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd2aff5640 1 -- 172.21.15.59:0/2995512976 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7efd380e02e0 con 0x562e58caaf10 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd5018f740 10 client.15954 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:24.569 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd5018f740 10 client.15954 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:33:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd5018f740 20 client.15954 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd5018f740 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562e58cc5eb0 0x562e58d33550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd5018f740 4 client.15954 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:33:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.565+0000 7efd5018f740 1 -- 172.21.15.59:0/2995512976 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562e58c18050 con 0x562e58cc5eb0 2024-05-08T05:33:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.566+0000 7efd467fc640 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562e58cc5eb0 0x562e58d33550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.566+0000 7efd2aff5640 10 client.15954 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:24.570 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.566+0000 7efd467fc640 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562e58cc5eb0 0x562e58d33550 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc0b1ff68c952ed4 server_cookie=49eddaff15d31d24 in_seq=0 out_seq=0 2024-05-08T05:33:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.579+0000 7efd2aff5640 1 -- 172.21.15.59:0/2995512976 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562e58c18050 con 0x562e58cc5eb0 2024-05-08T05:33:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.579+0000 7efd2aff5640 10 client.15954 handle_command_reply: tid=0 2024-05-08T05:33:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.579+0000 7efd5018f740 1 client.15954 shutdown 2024-05-08T05:33:24.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 -- 172.21.15.59:0/2995512976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 msgr2=0x562e58cbe090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cbe090 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7efd380a9b80 tx=0x7efd3800ce60 comp rx=0 tx=0).stop 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 -- 172.21.15.59:0/2995512976 shutdown_connections 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562e58cc5eb0 0x562e58d33550 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562e58cb60f0 0x562e58cbb060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562e58cab830 0x562e58cbe5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd2aff5640 0 client.15954 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 --2- 172.21.15.59:0/2995512976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562e58caaf10 0x562e58cbe090 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 -- 172.21.15.59:0/2995512976 >> 172.21.15.59:0/2995512976 conn(0x562e58ca7420 msgr2=0x562e58cb3000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 -- 172.21.15.59:0/2995512976 shutdown_connections 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 1 -- 172.21.15.59:0/2995512976 wait complete. 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.580+0000 7efd5018f740 20 client.15954 trim_cache size 0 max 16384 2024-05-08T05:33:24.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:33:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:33:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:33:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:33:24.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 -- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efd300b2a40 msgr2=0x7efd300b4f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efd300b2a40 0x7efd300b4f00 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7efd40005fd0 tx=0x7efd40009500 comp rx=0 tx=0).stop 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 -- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 msgr2=0x7efd48102090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 0x7efd48102090 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7efd3800e9f0 tx=0x7efd3800edf0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 -- 172.21.15.59:0/4143611851 shutdown_connections 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efd300b2a40 0x7efd300b4f00 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efd48107990 0x7efd48102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efd48101870 0x7efd481025d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 --2- 172.21.15.59:0/4143611851 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efd48100eb0 0x7efd48102090 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 -- 172.21.15.59:0/4143611851 >> 172.21.15.59:0/4143611851 conn(0x7efd48077e40 msgr2=0x7efd480ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 -- 172.21.15.59:0/4143611851 shutdown_connections 2024-05-08T05:33:24.583 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:24.581+0000 7efd4d472640 1 -- 172.21.15.59:0/4143611851 wait complete. 2024-05-08T05:33:25.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:24 smithi059 ceph-mon[30304]: pgmap v118: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-05-08T05:33:25.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:24 smithi143 ceph-mon[34337]: pgmap v118: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-05-08T05:33:25.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:24 smithi096 ceph-mon[34253]: pgmap v118: 129 pgs: 129 active+clean; 37 MiB data, 709 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-05-08T05:33:26.192 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.191+0000 7f604e4a1700 1 -- 172.21.15.59:0/3977941055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6048109520 msgr2=0x7f6048109900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.191+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3977941055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6048109520 0x7f6048109900 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f6038009990 tx=0x7f6038009ca0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.191+0000 7f604e4a1700 1 -- 172.21.15.59:0/3977941055 shutdown_connections 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.191+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3977941055 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048102450 0x7f6048110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.191+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3977941055 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6048109520 0x7f6048109900 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.191+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3977941055 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6048101a90 0x7f6048101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.191+0000 7f604e4a1700 1 -- 172.21.15.59:0/3977941055 >> 172.21.15.59:0/3977941055 conn(0x7f6048078a10 msgr2=0x7f60480ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.192+0000 7f604e4a1700 1 -- 172.21.15.59:0/3977941055 shutdown_connections 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.192+0000 7f604e4a1700 1 -- 172.21.15.59:0/3977941055 wait complete. 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.192+0000 7f604e4a1700 1 Processor -- start 2024-05-08T05:33:26.193 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.192+0000 7f604e4a1700 1 -- start start 2024-05-08T05:33:26.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f604e4a1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6048101a90 0x7f604819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:26.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f604e4a1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048102450 0x7f604819fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:26.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f604e4a1700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6048109520 0x7f6048199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:26.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f604e4a1700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f6048113650 con 0x7f6048101a90 2024-05-08T05:33:26.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f604e4a1700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f60481134d0 con 0x7f6048102450 2024-05-08T05:33:26.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f604e4a1700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f60481137d0 con 0x7f6048109520 2024-05-08T05:33:26.195 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f6047fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6048101a90 0x7f604819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f60477fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048102450 0x7f604819fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f604ca3e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6048109520 0x7f6048199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f60477fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048102450 0x7f604819fdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:50598/0 (socket says 172.21.15.59:50598) 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f60477fe700 1 -- 172.21.15.59:0/3202505468 learned_addr learned my addr 172.21.15.59:0/3202505468 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.193+0000 7f6047fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6048101a90 0x7f604819f890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48202/0 (socket says 172.21.15.59:48202) 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f604ca3e700 1 -- 172.21.15.59:0/3202505468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048102450 msgr2=0x7f604819fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f604ca3e700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048102450 0x7f604819fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f604ca3e700 1 -- 172.21.15.59:0/3202505468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6048101a90 msgr2=0x7f604819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f604ca3e700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6048101a90 0x7f604819f890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f604ca3e700 1 -- 172.21.15.59:0/3202505468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604819a1b0 con 0x7f6048109520 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f60477fe700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048102450 0x7f604819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f604ca3e700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6048109520 0x7f6048199a50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f604000efc0 tx=0x7f604000c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:26.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f60457fa700 1 -- 172.21.15.59:0/3202505468 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400092e0 con 0x7f6048109520 2024-05-08T05:33:26.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f60457fa700 1 -- 172.21.15.59:0/3202505468 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6040004bb0 con 0x7f6048109520 2024-05-08T05:33:26.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f60457fa700 1 -- 172.21.15.59:0/3202505468 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040004020 con 0x7f6048109520 2024-05-08T05:33:26.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604819a440 con 0x7f6048109520 2024-05-08T05:33:26.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.194+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60481a6760 con 0x7f6048109520 2024-05-08T05:33:26.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.195+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f604810ba00 con 0x7f6048109520 2024-05-08T05:33:26.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.197+0000 7f60457fa700 1 -- 172.21.15.59:0/3202505468 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f604001b030 con 0x7f6048109520 2024-05-08T05:33:26.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.198+0000 7f60457fa700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f60300b2a70 0x7f60300b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:26.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.198+0000 7f6047fff700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f60300b2a70 0x7f60300b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:26.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.198+0000 7f60457fa700 1 -- 172.21.15.59:0/3202505468 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f60400e2d30 con 0x7f6048109520 2024-05-08T05:33:26.198 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.198+0000 7f6047fff700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f60300b2a70 0x7f60300b4f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f603800b2e0 tx=0x7f6038005960 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:26.201 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.201+0000 7f60457fa700 1 -- 172.21.15.59:0/3202505468 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f6040022020 con 0x7f6048109520 2024-05-08T05:33:26.520 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.520+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f604819af80 con 0x7f6048109520 2024-05-08T05:33:26.521 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.521+0000 7f60457fa700 1 -- 172.21.15.59:0/3202505468 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11934 (secure 0 0 0) 0x7f6040010040 con 0x7f6048109520 2024-05-08T05:33:26.521 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:33:26.522 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:32:11.760389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:33:26.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.523+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f60300b2a70 msgr2=0x7f60300b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:26.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.523+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f60300b2a70 0x7f60300b4f30 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f603800b2e0 tx=0x7f6038005960 comp rx=0 tx=0).stop 2024-05-08T05:33:26.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6048109520 msgr2=0x7f6048199a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:26.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6048109520 0x7f6048199a50 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f604000efc0 tx=0x7f604000c3a0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 shutdown_connections 2024-05-08T05:33:26.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f60300b2a70 0x7f60300b4f30 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f6048109520 0x7f6048199a50 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f6048101a90 0x7f604819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 --2- 172.21.15.59:0/3202505468 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f6048102450 0x7f604819fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:26.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 >> 172.21.15.59:0/3202505468 conn(0x7f6048078a10 msgr2=0x7f60481002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:26.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 shutdown_connections 2024-05-08T05:33:26.527 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:26.524+0000 7f604e4a1700 1 -- 172.21.15.59:0/3202505468 wait complete. 2024-05-08T05:33:26.528 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-05-08T05:33:27.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:26 smithi143 ceph-mon[34337]: pgmap v119: 129 pgs: 129 active+clean; 38 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 462 KiB/s wr, 83 op/s 2024-05-08T05:33:27.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:26 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3202505468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:33:27.275 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:26 smithi059 ceph-mon[30304]: pgmap v119: 129 pgs: 129 active+clean; 38 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 462 KiB/s wr, 83 op/s 2024-05-08T05:33:27.275 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:26 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3202505468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:33:27.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:26 smithi096 ceph-mon[34253]: pgmap v119: 129 pgs: 129 active+clean; 38 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 386 KiB/s rd, 462 KiB/s wr, 83 op/s 2024-05-08T05:33:27.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:26 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3202505468' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:33:27.313 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db / --timeout 60 --expiration 81.05 --await-for 42.0' 2024-05-08T05:33:27.314 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db / --timeout 60 --expiration 81.05 --await-for 42.0 2024-05-08T05:33:27.624 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:33:29.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:28 smithi059 ceph-mon[30304]: pgmap v120: 129 pgs: 129 active+clean; 38 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 433 KiB/s wr, 74 op/s 2024-05-08T05:33:29.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:28 smithi143 ceph-mon[34337]: pgmap v120: 129 pgs: 129 active+clean; 38 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 433 KiB/s wr, 74 op/s 2024-05-08T05:33:29.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:28 smithi096 ceph-mon[34253]: pgmap v120: 129 pgs: 129 active+clean; 38 MiB data, 720 MiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 433 KiB/s wr, 74 op/s 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.751+0000 7f9eab73c700 1 -- 172.21.15.59:0/1297224473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ea4103ee0 msgr2=0x7f9ea4110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.751+0000 7f9eab73c700 1 --2- 172.21.15.59:0/1297224473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ea4103ee0 0x7f9ea4110360 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00b0d0 tx=0x7f9e9c00b3e0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.751+0000 7f9eab73c700 1 -- 172.21.15.59:0/1297224473 shutdown_connections 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.751+0000 7f9eab73c700 1 --2- 172.21.15.59:0/1297224473 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ea4103ee0 0x7f9ea4110360 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.751+0000 7f9eab73c700 1 --2- 172.21.15.59:0/1297224473 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ea4109590 0x7f9ea4109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.751+0000 7f9eab73c700 1 --2- 172.21.15.59:0/1297224473 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ea4103520 0x7f9ea41039a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.752+0000 7f9eab73c700 1 -- 172.21.15.59:0/1297224473 >> 172.21.15.59:0/1297224473 conn(0x7f9ea407a5c0 msgr2=0x7f9ea407a9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.752+0000 7f9eab73c700 1 -- 172.21.15.59:0/1297224473 shutdown_connections 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.752+0000 7f9eab73c700 1 -- 172.21.15.59:0/1297224473 wait complete. 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.752+0000 7f9eab73c700 1 Processor -- start 2024-05-08T05:33:29.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.752+0000 7f9eab73c700 1 -- start start 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9eab73c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ea4103520 0x7f9ea419f660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9eab73c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ea4103ee0 0x7f9ea419fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9eab73c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ea4109590 0x7f9ea41997e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9eab73c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41136e0 con 0x7f9ea4103520 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9eab73c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea4113560 con 0x7f9ea4109590 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9eab73c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea4113860 con 0x7f9ea4103ee0 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea94d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ea4103520 0x7f9ea419f660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea8cd7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ea4103ee0 0x7f9ea419fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea9cd9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ea4109590 0x7f9ea41997e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea94d8700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ea4103520 0x7f9ea419f660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59506/0 (socket says 172.21.15.59:59506) 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea8cd7700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ea4103ee0 0x7f9ea419fba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:34594/0 (socket says 172.21.15.59:34594) 2024-05-08T05:33:29.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea94d8700 1 -- 172.21.15.59:0/116600724 learned_addr learned my addr 172.21.15.59:0/116600724 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea9cd9700 1 -- 172.21.15.59:0/116600724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ea4103ee0 msgr2=0x7f9ea419fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea9cd9700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ea4103ee0 0x7f9ea419fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea9cd9700 1 -- 172.21.15.59:0/116600724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ea4103520 msgr2=0x7f9ea419f660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea9cd9700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ea4103520 0x7f9ea419f660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea9cd9700 1 -- 172.21.15.59:0/116600724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ea4199fc0 con 0x7f9ea4109590 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ea4103ee0 0x7f9ea419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.753+0000 7f9ea9cd9700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ea4109590 0x7f9ea41997e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00b770 tx=0x7f9e9c007980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.754+0000 7f9e9a7fc700 1 -- 172.21.15.59:0/116600724 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c01f070 con 0x7f9ea4109590 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.754+0000 7f9e9a7fc700 1 -- 172.21.15.59:0/116600724 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c007dc0 con 0x7f9ea4109590 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.754+0000 7f9eab73c700 1 -- 172.21.15.59:0/116600724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ea419a250 con 0x7f9ea4109590 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.754+0000 7f9e9a7fc700 1 -- 172.21.15.59:0/116600724 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c01b430 con 0x7f9ea4109590 2024-05-08T05:33:29.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.754+0000 7f9eab73c700 1 -- 172.21.15.59:0/116600724 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ea41a64c0 con 0x7f9ea4109590 2024-05-08T05:33:29.757 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.757+0000 7f9e9a7fc700 1 -- 172.21.15.59:0/116600724 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9e9c022070 con 0x7f9ea4109590 2024-05-08T05:33:29.758 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.758+0000 7f9e9a7fc700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9e900b2aa0 0x7f9e900b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.758+0000 7f9e9a7fc700 1 -- 172.21.15.59:0/116600724 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f9e9c00f020 con 0x7f9ea4109590 2024-05-08T05:33:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.758+0000 7f9ea94d8700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9e900b2aa0 0x7f9e900b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.758+0000 7f9eb5fcfb80 1 Processor -- start 2024-05-08T05:33:29.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9eb5fcfb80 1 -- start start 2024-05-08T05:33:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270d8820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea94d8700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9e900b2aa0 0x7f9e900b4f60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009d20 tx=0x7f9ea0009260 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270d3c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d4260 0x55ba270d46c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ba270219f0 con 0x55ba270d4260 2024-05-08T05:33:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ba26fc0950 con 0x55ba270d8d60 2024-05-08T05:33:29.760 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ba2701d9b0 con 0x55ba270d8440 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea94d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270d8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea8cd7700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270d3c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea94d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270d8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:34600/0 (socket says 172.21.15.59:34600) 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea94d8700 1 -- 172.21.15.59:0/2040884246 learned_addr learned my addr 172.21.15.59:0/2040884246 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea9cd9700 1 --2- 172.21.15.59:0/2040884246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d4260 0x55ba270d46c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea8cd7700 1 -- 172.21.15.59:0/2040884246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 msgr2=0x55ba270d8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/2040884246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270d8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea8cd7700 1 -- 172.21.15.59:0/2040884246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d4260 msgr2=0x55ba270d46c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/2040884246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d4260 0x55ba270d46c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.759+0000 7f9ea8cd7700 1 -- 172.21.15.59:0/2040884246 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba26f6ea60 con 0x55ba270d8d60 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.760+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/2040884246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270d3c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e940114c0 tx=0x7f9e940117d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a3f55c02f304d73d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.760+0000 7f9e8f7fe700 1 -- 172.21.15.59:0/2040884246 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94004310 con 0x55ba270d8d60 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.760+0000 7f9e8f7fe700 1 -- 172.21.15.59:0/2040884246 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e940044b0 con 0x55ba270d8d60 2024-05-08T05:33:29.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.760+0000 7f9e8f7fe700 1 -- 172.21.15.59:0/2040884246 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94003680 con 0x55ba270d8d60 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.761+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/2040884246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 msgr2=0x55ba270d3c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.761+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/2040884246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270d3c40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9e940114c0 tx=0x7f9e940117d0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.761+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/2040884246 shutdown_connections 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.761+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/2040884246 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d4260 0x55ba270d46c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.761+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/2040884246 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270d8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.761+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/2040884246 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270d3c40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.761+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/2040884246 >> 172.21.15.59:0/2040884246 conn(0x55ba26ffb290 msgr2=0x55ba270ca070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.761+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/2040884246 shutdown_connections 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.761+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/2040884246 wait complete. 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.762+0000 7f9eb5fcfb80 1 -- start start 2024-05-08T05:33:29.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.762+0000 7f9eb5fcfb80 1 Processor -- start 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.762+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dfba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.762+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 0x55ba270dcf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270dd480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ba270e0930 con 0x55ba270d8440 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ba26f87dc0 con 0x55ba270d8d60 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ba270e0ab0 con 0x55ba270d4260 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea94d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dfba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea94d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dfba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:34614/0 (socket says 172.21.15.59:34614) 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea94d8700 1 -- 172.21.15.59:0/1636272113 learned_addr learned my addr 172.21.15.59:0/1636272113 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9eb5fcfb80 10 client.0 _ll_register_callbacks cb 0x55ba26fb9ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 0x55ba270dcf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea9cd9700 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270dd480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea94d8700 1 -- 172.21.15.59:0/1636272113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 msgr2=0x55ba270dd480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:29.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea94d8700 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270dd480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea94d8700 1 -- 172.21.15.59:0/1636272113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 msgr2=0x55ba270dcf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea94d8700 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 0x55ba270dcf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea94d8700 1 -- 172.21.15.59:0/1636272113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba26fbd820 con 0x55ba270d4260 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 0x55ba270dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9ea94d8700 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dfba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009f70 tx=0x7f9ea00047c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9928c7c62406bb87 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.763+0000 7f9e8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9e8cff9700 1 -- 172.21.15.59:0/1636272113 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ea002b060 con 0x55ba270d4260 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9e8cff9700 1 -- 172.21.15.59:0/1636272113 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f9ea00265f0 con 0x55ba270d4260 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/1636272113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e940044b0 con 0x55ba270d4260 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9e8cff9700 1 -- 172.21.15.59:0/1636272113 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9ea0032050 con 0x55ba270d4260 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9eb5fcfb80 10 client.25537 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:29.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/1636272113 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ba270eb1b0 con 0x55ba270d4260 2024-05-08T05:33:29.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9e8cff9700 1 -- 172.21.15.59:0/1636272113 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e940044b0 con 0x55ba270d4260 2024-05-08T05:33:29.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9e8cff9700 1 -- 172.21.15.59:0/1636272113 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f9ea0025030 con 0x55ba270d4260 2024-05-08T05:33:29.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9eb5fcfb80 10 client.25537 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:29.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9eb5fcfb80 10 client.25537 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:33:29.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9eb5fcfb80 20 client.25537 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:29.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ba270ed890 0x55ba270efce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.768 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9eb5fcfb80 4 client.25537 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.764+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/1636272113 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba2709b7e0 con 0x55ba270ed890 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.765+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ba270ed890 0x55ba270efce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.765+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ba270ed890 0x55ba270efce0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b9b40120ca9f0237 server_cookie=df56c9f778737d9f in_seq=0 out_seq=0 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.765+0000 7f9e8cff9700 10 client.25537 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9e8cff9700 1 -- 172.21.15.59:0/1636272113 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ba2709b7e0 con 0x55ba270ed890 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9e8cff9700 10 client.25537 handle_command_reply: tid=0 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 client.25537 shutdown 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/1636272113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 msgr2=0x55ba270dfba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dfba0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009f70 tx=0x7f9ea00047c0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/1636272113 shutdown_connections 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ba270ed890 0x55ba270efce0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270dd480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dfba0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9e8cff9700 0 client.25537 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:29.769 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/1636272113 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 0x55ba270dcf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/1636272113 >> 172.21.15.59:0/1636272113 conn(0x55ba26ffb290 msgr2=0x55ba270d9a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:29.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/1636272113 shutdown_connections 2024-05-08T05:33:29.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/1636272113 wait complete. 2024-05-08T05:33:29.770 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.766+0000 7f9eb5fcfb80 20 client.25537 trim_cache size 0 max 16384 2024-05-08T05:33:29.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9eb5fcfb80 1 Processor -- start 2024-05-08T05:33:29.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9eb5fcfb80 1 -- start start 2024-05-08T05:33:29.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 0x55ba270df190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270f55b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41ad010 con 0x55ba270d8440 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41aebe0 con 0x55ba270d8d60 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41ad6e0 con 0x55ba270d4260 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9ea9cd9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270f55b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9ea94d8700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9ea8cd7700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 0x55ba270df190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9ea9cd9700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270f55b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:52634/0 (socket says 172.21.15.59:52634) 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9ea9cd9700 1 -- 172.21.15.59:0/3502335761 learned_addr learned my addr 172.21.15.59:0/3502335761 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9ea9cd9700 1 -- 172.21.15.59:0/3502335761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 msgr2=0x55ba270dec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.790+0000 7f9ea9cd9700 1 --2- 172.21.15.59:0/3502335761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.791+0000 7f9ea9cd9700 1 -- 172.21.15.59:0/3502335761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 msgr2=0x55ba270df190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.791+0000 7f9ea9cd9700 1 --2- 172.21.15.59:0/3502335761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 0x55ba270df190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.791+0000 7f9ea9cd9700 1 -- 172.21.15.59:0/3502335761 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba270c4fd0 con 0x55ba270d8d60 2024-05-08T05:33:29.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.791+0000 7f9ea9cd9700 1 --2- 172.21.15.59:0/3502335761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270f55b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0a7b40 tx=0x7f9e9c0b1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4ac99f2dcabd9ef6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:29.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.791+0000 7f9e8effd700 1 -- 172.21.15.59:0/3502335761 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0aa3e0 con 0x55ba270d8d60 2024-05-08T05:33:29.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.791+0000 7f9e8effd700 1 -- 172.21.15.59:0/3502335761 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c0aafd0 con 0x55ba270d8d60 2024-05-08T05:33:29.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.791+0000 7f9e8effd700 1 -- 172.21.15.59:0/3502335761 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e9c0b10b0 con 0x55ba270d8d60 2024-05-08T05:33:29.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/3502335761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 msgr2=0x55ba270f55b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:29.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/3502335761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270f55b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c0a7b40 tx=0x7f9e9c0b1040 comp rx=0 tx=0).stop 2024-05-08T05:33:29.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/3502335761 shutdown_connections 2024-05-08T05:33:29.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/3502335761 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d8d60 0x55ba270f55b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/3502335761 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d4260 0x55ba270dec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/3502335761 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8440 0x55ba270df190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/3502335761 >> 172.21.15.59:0/3502335761 conn(0x55ba26ffb290 msgr2=0x55ba270de540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:29.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/3502335761 shutdown_connections 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/3502335761 wait complete. 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.792+0000 7f9eb5fcfb80 1 -- start start 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9eb5fcfb80 1 Processor -- start 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d4260 0x55ba270e5d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270e6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9eb5fcfb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8d60 0x55ba270d91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41ae480 con 0x55ba270d8d60 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41addb0 con 0x55ba270d4260 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9eb5fcfb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9ea41aaf90 con 0x55ba270d8440 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9eb5fcfb80 10 client.0 _ll_register_callbacks cb 0x55ba26fb9ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9ea9cd9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8d60 0x55ba270d91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9ea9cd9700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8d60 0x55ba270d91e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59536/0 (socket says 172.21.15.59:59536) 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9ea9cd9700 1 -- 172.21.15.59:0/77109131 learned_addr learned my addr 172.21.15.59:0/77109131 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:29.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270e6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.793+0000 7f9ea94d8700 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d4260 0x55ba270e5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9ea8cd7700 1 -- 172.21.15.59:0/77109131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d4260 msgr2=0x55ba270e5d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d4260 0x55ba270e5d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9ea8cd7700 1 -- 172.21.15.59:0/77109131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8d60 msgr2=0x55ba270d91e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8d60 0x55ba270d91e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9ea8cd7700 1 -- 172.21.15.59:0/77109131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba270d8090 con 0x55ba270d8440 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9ea8cd7700 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270e6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400a010 tx=0x7f9e94005bd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4d4652de52bd25ae server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9e8cff9700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9e8cff9700 1 -- 172.21.15.59:0/77109131 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e94036070 con 0x55ba270d8440 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/77109131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e9c0aafd0 con 0x55ba270d8440 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.794+0000 7f9e8cff9700 1 -- 172.21.15.59:0/77109131 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f9e940054e0 con 0x55ba270d8440 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9e8cff9700 1 -- 172.21.15.59:0/77109131 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7f9e94042050 con 0x55ba270d8440 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9eb5fcfb80 10 client.25543 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/77109131 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55ba270cc810 con 0x55ba270d8440 2024-05-08T05:33:29.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9e8cff9700 1 -- 172.21.15.59:0/77109131 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e9c0aafd0 con 0x55ba270d8440 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9e8cff9700 1 -- 172.21.15.59:0/77109131 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7f9e94018070 con 0x55ba270d8440 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9eb5fcfb80 10 client.25543 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9eb5fcfb80 10 client.25543 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9eb5fcfb80 20 client.25543 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ba270cce30 0x55ba270cf250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9eb5fcfb80 4 client.25543 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.05, "await_for": 42.0}] 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/77109131 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 81.05, "await_for": 42.0}) -- 0x55ba2702f3d0 con 0x55ba270cce30 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.795+0000 7f9ea94d8700 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ba270cce30 0x55ba270cf250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.796+0000 7f9e8cff9700 10 client.25543 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:29.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:29.796+0000 7f9ea94d8700 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ba270cce30 0x55ba270cf250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb22c20d615bcd8c server_cookie=dc87f02c4f35a510 in_seq=0 out_seq=0 2024-05-08T05:33:30.727 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.726+0000 7f9e8cff9700 1 -- 172.21.15.59:0/77109131 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55ba2702f3d0 con 0x55ba270cce30 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.726+0000 7f9e8cff9700 10 client.25543 handle_command_reply: tid=0 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.726+0000 7f9eb5fcfb80 1 client.25543 shutdown 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/77109131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 msgr2=0x55ba270e6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270e6280 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9e9400a010 tx=0x7f9e94005bd0 comp rx=0 tx=0).stop 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/77109131 shutdown_connections 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ba270cce30 0x55ba270cf250 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ba270d8d60 0x55ba270d91e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ba270d4260 0x55ba270e5d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 --2- 172.21.15.59:0/77109131 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ba270d8440 0x55ba270e6280 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9e8cff9700 0 client.25543 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/77109131 >> 172.21.15.59:0/77109131 conn(0x55ba26ffb290 msgr2=0x55ba270d7910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/77109131 shutdown_connections 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 1 -- 172.21.15.59:0/77109131 wait complete. 2024-05-08T05:33:30.728 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.727+0000 7f9eb5fcfb80 20 client.25543 trim_cache size 0 max 16384 2024-05-08T05:33:30.729 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:33:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 33, 2024-05-08T05:33:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:33:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 14, 2024-05-08T05:33:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:33:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "cc2da2d4": { 2024-05-08T05:33:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 14, 2024-05-08T05:33:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:33:30.729 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 81.0, 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:30.730 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:30.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:30.731 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:33:30.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 -- 172.21.15.59:0/116600724 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9e900b2aa0 msgr2=0x7f9e900b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:30.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9e900b2aa0 0x7f9e900b4f60 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9ea0009d20 tx=0x7f9ea0009260 comp rx=0 tx=0).stop 2024-05-08T05:33:30.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 -- 172.21.15.59:0/116600724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ea4109590 msgr2=0x7f9ea41997e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:30.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ea4109590 0x7f9ea41997e0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f9e9c00b770 tx=0x7f9e9c007980 comp rx=0 tx=0).stop 2024-05-08T05:33:30.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 -- 172.21.15.59:0/116600724 shutdown_connections 2024-05-08T05:33:30.731 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9e900b2aa0 0x7f9e900b4f60 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:30.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9ea4109590 0x7f9ea41997e0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:30.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9ea4103520 0x7f9ea419f660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:30.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 --2- 172.21.15.59:0/116600724 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9ea4103ee0 0x7f9ea419fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:30.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 -- 172.21.15.59:0/116600724 >> 172.21.15.59:0/116600724 conn(0x7f9ea407a5c0 msgr2=0x7f9ea4101de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:30.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 -- 172.21.15.59:0/116600724 shutdown_connections 2024-05-08T05:33:30.732 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:30.728+0000 7f9eab73c700 1 -- 172.21.15.59:0/116600724 wait complete. 2024-05-08T05:33:31.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:30 smithi143 ceph-mon[34337]: pgmap v121: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 252 op/s 2024-05-08T05:33:31.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:30 smithi096 ceph-mon[34253]: pgmap v121: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 252 op/s 2024-05-08T05:33:31.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:30 smithi059 ceph-mon[30304]: pgmap v121: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 252 op/s 2024-05-08T05:33:31.657 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'cc2da2d4', quiesce took 7.6 seconds. Will release after: 21.05 2024-05-08T05:33:33.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:32 smithi143 ceph-mon[34337]: pgmap v122: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-05-08T05:33:33.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:32 smithi096 ceph-mon[34253]: pgmap v122: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-05-08T05:33:33.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:32 smithi059 ceph-mon[30304]: pgmap v122: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-05-08T05:33:35.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:34 smithi143 ceph-mon[34337]: pgmap v123: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-05-08T05:33:35.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:34 smithi096 ceph-mon[34253]: pgmap v123: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-05-08T05:33:35.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:34 smithi059 ceph-mon[30304]: pgmap v123: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 247 op/s 2024-05-08T05:33:37.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:36 smithi096 ceph-mon[34253]: pgmap v124: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 253 op/s 2024-05-08T05:33:37.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:36 smithi059 ceph-mon[30304]: pgmap v124: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 253 op/s 2024-05-08T05:33:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:36 smithi143 ceph-mon[34337]: pgmap v124: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 253 op/s 2024-05-08T05:33:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 -- 172.21.15.59:0/958254436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20103c60 msgr2=0x7ffa20104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 --2- 172.21.15.59:0/958254436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20103c60 0x7ffa20104040 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009900 tx=0x7ffa1002a480 comp rx=0 tx=0).stop 2024-05-08T05:33:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 -- 172.21.15.59:0/958254436 shutdown_connections 2024-05-08T05:33:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 --2- 172.21.15.59:0/958254436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa2010d030 0x7ffa2010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 --2- 172.21.15.59:0/958254436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa20104580 0x7ffa2010ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 --2- 172.21.15.59:0/958254436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20103c60 0x7ffa20104040 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 -- 172.21.15.59:0/958254436 >> 172.21.15.59:0/958254436 conn(0x7ffa20077c30 msgr2=0x7ffa20078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:37.687 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 -- 172.21.15.59:0/958254436 shutdown_connections 2024-05-08T05:33:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 -- 172.21.15.59:0/958254436 wait complete. 2024-05-08T05:33:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.686+0000 7ffa25672640 1 Processor -- start 2024-05-08T05:33:37.688 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa25672640 1 -- start start 2024-05-08T05:33:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa25672640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa20103c60 0x7ffa20199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa25672640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20104580 0x7ffa201996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.689 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa25672640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa2010d030 0x7ffa2019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa25672640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20112c70 con 0x7ffa20104580 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa25672640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20112af0 con 0x7ffa2010d030 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa25672640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffa20112df0 con 0x7ffa20103c60 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa1e7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20104580 0x7ffa201996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa1f7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa2010d030 0x7ffa2019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa1e7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20104580 0x7ffa201996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59554/0 (socket says 172.21.15.59:59554) 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa1e7fc640 1 -- 172.21.15.59:0/3273881802 learned_addr learned my addr 172.21.15.59:0/3273881802 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.687+0000 7ffa1effd640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa20103c60 0x7ffa20199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa1e7fc640 1 -- 172.21.15.59:0/3273881802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa20103c60 msgr2=0x7ffa20199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa1e7fc640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa20103c60 0x7ffa20199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa1e7fc640 1 -- 172.21.15.59:0/3273881802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa2010d030 msgr2=0x7ffa2019d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:37.690 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa1e7fc640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa2010d030 0x7ffa2019d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa1e7fc640 1 -- 172.21.15.59:0/3273881802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa2019e110 con 0x7ffa20104580 2024-05-08T05:33:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa1e7fc640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20104580 0x7ffa201996b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400d770 tx=0x7ffa1400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa07fff640 1 -- 172.21.15.59:0/3273881802 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14017070 con 0x7ffa20104580 2024-05-08T05:33:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa25672640 1 -- 172.21.15.59:0/3273881802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa2019e400 con 0x7ffa20104580 2024-05-08T05:33:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa07fff640 1 -- 172.21.15.59:0/3273881802 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa140054d0 con 0x7ffa20104580 2024-05-08T05:33:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa07fff640 1 -- 172.21.15.59:0/3273881802 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa14004c30 con 0x7ffa20104580 2024-05-08T05:33:37.691 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.688+0000 7ffa25672640 1 -- 172.21.15.59:0/3273881802 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa201a5c90 con 0x7ffa20104580 2024-05-08T05:33:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa07fff640 1 -- 172.21.15.59:0/3273881802 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ffa14004e10 con 0x7ffa20104580 2024-05-08T05:33:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa2838d740 1 Processor -- start 2024-05-08T05:33:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa2838d740 1 -- start start 2024-05-08T05:33:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf2f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf39bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 0x55aacaf3c4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa2838d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55aacae945d0 con 0x55aacaf2f830 2024-05-08T05:33:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa2838d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aacae4d890 con 0x55aacaf2ef10 2024-05-08T05:33:37.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa2838d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55aacae90590 con 0x55aacaf3a0f0 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa1effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf39bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa1effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf39bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59566/0 (socket says 172.21.15.59:59566) 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa1effd640 1 -- 172.21.15.59:0/4235409167 learned_addr learned my addr 172.21.15.59:0/4235409167 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.691+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf2f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa1e7fc640 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 0x55aacaf3c4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa1effd640 1 -- 172.21.15.59:0/4235409167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 msgr2=0x55aacaf3c4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa1effd640 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 0x55aacaf3c4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa1effd640 1 -- 172.21.15.59:0/4235409167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 msgr2=0x55aacaf2f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa1effd640 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf2f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa07fff640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ffa080b2a10 0x7ffa080b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa1effd640 1 -- 172.21.15.59:0/4235409167 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacaba6210 con 0x55aacaf2f830 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf2f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa1e7fc640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ffa080b2a10 0x7ffa080b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa1effd640 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf39bb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009900 tx=0x7ffa10032e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5d4fb39cc22b898 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa07fff640 1 -- 172.21.15.59:0/3273881802 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ffa140e23f0 con 0x7ffa20104580 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa057fa640 1 -- 172.21.15.59:0/4235409167 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10040070 con 0x55aacaf2f830 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa057fa640 1 -- 172.21.15.59:0/4235409167 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa100316a0 con 0x55aacaf2f830 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.692+0000 7ffa057fa640 1 -- 172.21.15.59:0/4235409167 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10035040 con 0x55aacaf2f830 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 -- 172.21.15.59:0/4235409167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 msgr2=0x55aacaf39bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa1e7fc640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ffa080b2a10 0x7ffa080b4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55aacaf2c5b0 tx=0x7ffa140e7040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf39bb0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009900 tx=0x7ffa10032e60 comp rx=0 tx=0).stop 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 -- 172.21.15.59:0/4235409167 shutdown_connections 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 0x55aacaf3c4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf39bb0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4235409167 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf2f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 -- 172.21.15.59:0/4235409167 >> 172.21.15.59:0/4235409167 conn(0x55aacaf2b420 msgr2=0x55aacaf2b800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 -- 172.21.15.59:0/4235409167 shutdown_connections 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 -- 172.21.15.59:0/4235409167 wait complete. 2024-05-08T05:33:37.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.693+0000 7ffa2838d740 1 -- start start 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa2838d740 1 Processor -- start 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf37030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf37570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 0x55aacaf37ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa2838d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55aacaf3ccc0 con 0x55aacaf2f830 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa2838d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aacae51370 con 0x55aacaf2ef10 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa2838d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55aacaf3ce40 con 0x55aacaf3a0f0 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa2838d740 10 client.0 _ll_register_callbacks cb 0x55aacaba5a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf37570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf37570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59570/0 (socket says 172.21.15.59:59570) 2024-05-08T05:33:37.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1effd640 1 -- 172.21.15.59:0/4277822500 learned_addr learned my addr 172.21.15.59:0/4277822500 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf37030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 0x55aacaf37ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1effd640 1 -- 172.21.15.59:0/4277822500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 msgr2=0x55aacaf37ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1effd640 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 0x55aacaf37ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1effd640 1 -- 172.21.15.59:0/4277822500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 msgr2=0x55aacaf37030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1effd640 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf37030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1effd640 1 -- 172.21.15.59:0/4277822500 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacac72620 con 0x55aacaf2f830 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.694+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 0x55aacaf37ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa1effd640 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf37570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffa10031a80 tx=0x7ffa1002a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2a3f9a1ec268133 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa02ff5640 1 -- 172.21.15.59:0/4277822500 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa1004c060 con 0x55aacaf2f830 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa02ff5640 1 -- 172.21.15.59:0/4277822500 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ffa10004230 con 0x55aacaf2f830 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa2838d740 1 -- 172.21.15.59:0/4277822500 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa100316a0 con 0x55aacaf2f830 2024-05-08T05:33:37.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa02ff5640 1 -- 172.21.15.59:0/4277822500 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffa10053050 con 0x55aacaf2f830 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa2838d740 10 client.15990 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa2838d740 1 -- 172.21.15.59:0/4277822500 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aacaf48b40 con 0x55aacaf2f830 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa02ff5640 1 -- 172.21.15.59:0/4277822500 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa100316a0 con 0x55aacaf2f830 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa02ff5640 1 -- 172.21.15.59:0/4277822500 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7ffa1003c440 con 0x55aacaf2f830 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa2838d740 10 client.15990 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa2838d740 10 client.15990 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa2838d740 20 client.15990 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aacaf49e90 0x55aacaf4c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa2838d740 4 client.15990 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.695+0000 7ffa2838d740 1 -- 172.21.15.59:0/4277822500 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aacaea21e0 con 0x55aacaf49e90 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.696+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aacaf49e90 0x55aacaf4c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.696+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aacaf49e90 0x55aacaf4c280 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67b822b7b4ffa22d server_cookie=eea0c43abee66f51 in_seq=0 out_seq=0 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.696+0000 7ffa02ff5640 10 client.15990 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa02ff5640 1 -- 172.21.15.59:0/4277822500 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aacaea21e0 con 0x55aacaf49e90 2024-05-08T05:33:37.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa02ff5640 10 client.15990 handle_command_reply: tid=0 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 client.15990 shutdown 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 -- 172.21.15.59:0/4277822500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 msgr2=0x55aacaf37570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf37570 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7ffa10031a80 tx=0x7ffa1002a480 comp rx=0 tx=0).stop 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 -- 172.21.15.59:0/4277822500 shutdown_connections 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aacaf49e90 0x55aacaf4c280 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf3a0f0 0x55aacaf37ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2f830 0x55aacaf37570 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa02ff5640 0 client.15990 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 --2- 172.21.15.59:0/4277822500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf37030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 -- 172.21.15.59:0/4277822500 >> 172.21.15.59:0/4277822500 conn(0x55aacaf2b420 msgr2=0x55aacaf311a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 -- 172.21.15.59:0/4277822500 shutdown_connections 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.697+0000 7ffa2838d740 1 -- 172.21.15.59:0/4277822500 wait complete. 2024-05-08T05:33:37.704 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.698+0000 7ffa2838d740 20 client.15990 trim_cache size 0 max 16384 2024-05-08T05:33:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa2838d740 1 Processor -- start 2024-05-08T05:33:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa2838d740 1 -- start start 2024-05-08T05:33:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf38f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf51910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf3a0f0 0x55aacaf51e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa2838d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201afd00 con 0x55aacaf3a0f0 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa2838d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201b18d0 con 0x55aacaf2ef10 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa2838d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201b03d0 con 0x55aacaf2f830 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf3a0f0 0x55aacaf51e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf3a0f0 0x55aacaf51e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59578/0 (socket says 172.21.15.59:59578) 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1f7fe640 1 -- 172.21.15.59:0/2826039890 learned_addr learned my addr 172.21.15.59:0/2826039890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf38f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1effd640 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf51910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1f7fe640 1 -- 172.21.15.59:0/2826039890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 msgr2=0x55aacaf51910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf51910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1f7fe640 1 -- 172.21.15.59:0/2826039890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 msgr2=0x55aacaf38f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf38f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.713+0000 7ffa1f7fe640 1 -- 172.21.15.59:0/2826039890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aacaf31840 con 0x55aacaf3a0f0 2024-05-08T05:33:37.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.714+0000 7ffa1effd640 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf51910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.714+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf3a0f0 0x55aacaf51e50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800aef0 tx=0x7ffa1800a2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547f5acf9849cbc0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.714+0000 7ffa04ff9640 1 -- 172.21.15.59:0/2826039890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18027070 con 0x55aacaf3a0f0 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.714+0000 7ffa04ff9640 1 -- 172.21.15.59:0/2826039890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1800ee40 con 0x55aacaf3a0f0 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.714+0000 7ffa04ff9640 1 -- 172.21.15.59:0/2826039890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18005440 con 0x55aacaf3a0f0 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.714+0000 7ffa2838d740 1 -- 172.21.15.59:0/2826039890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf3a0f0 msgr2=0x55aacaf51e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.714+0000 7ffa2838d740 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf3a0f0 0x55aacaf51e50 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800aef0 tx=0x7ffa1800a2c0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 -- 172.21.15.59:0/2826039890 shutdown_connections 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf3a0f0 0x55aacaf51e50 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf51910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 --2- 172.21.15.59:0/2826039890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf2ef10 0x55aacaf38f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 -- 172.21.15.59:0/2826039890 >> 172.21.15.59:0/2826039890 conn(0x55aacaf2b420 msgr2=0x55aacaf38860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 -- 172.21.15.59:0/2826039890 shutdown_connections 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 -- 172.21.15.59:0/2826039890 wait complete. 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 -- start start 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 Processor -- start 2024-05-08T05:33:37.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2ef10 0x55aacaf42090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf425d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf3a0f0 0x55aacaf3f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201b1170 con 0x55aacaf2ef10 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201b0aa0 con 0x55aacaf3a0f0 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ffa201adcc0 con 0x55aacaf2f830 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa2838d740 10 client.0 _ll_register_callbacks cb 0x7ffa10032db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa1f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2ef10 0x55aacaf42090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa1f7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2ef10 0x55aacaf42090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:59586/0 (socket says 172.21.15.59:59586) 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa1f7fe640 1 -- 172.21.15.59:0/757310749 learned_addr learned my addr 172.21.15.59:0/757310749 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.715+0000 7ffa1effd640 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf425d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa1f7fe640 1 -- 172.21.15.59:0/757310749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 msgr2=0x55aacaf425d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf425d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa1f7fe640 1 -- 172.21.15.59:0/757310749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf3a0f0 msgr2=0x55aacaf3f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf3a0f0 0x55aacaf3f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa1f7fe640 1 -- 172.21.15.59:0/757310749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aacaf3f6a0 con 0x55aacaf2ef10 2024-05-08T05:33:37.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa1effd640 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa1f7fe640 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2ef10 0x55aacaf42090 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800a290 tx=0x7ffa1801f850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a851bccdfb40b764 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa02ff5640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa02ff5640 1 -- 172.21.15.59:0/757310749 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa18027070 con 0x55aacaf2ef10 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa02ff5640 1 -- 172.21.15.59:0/757310749 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ffa18005000 con 0x55aacaf2ef10 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa2838d740 1 -- 172.21.15.59:0/757310749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa1800ee40 con 0x55aacaf2ef10 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa02ff5640 1 -- 172.21.15.59:0/757310749 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7ffa1801fc80 con 0x55aacaf2ef10 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa2838d740 10 client.15996 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa2838d740 1 -- 172.21.15.59:0/757310749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55aacafb4d50 con 0x55aacaf2ef10 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa02ff5640 1 -- 172.21.15.59:0/757310749 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa1800ee40 con 0x55aacaf2ef10 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.716+0000 7ffa02ff5640 1 -- 172.21.15.59:0/757310749 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7ffa1800f780 con 0x55aacaf2ef10 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.717+0000 7ffa2838d740 10 client.15996 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.717+0000 7ffa2838d740 10 client.15996 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.717+0000 7ffa2838d740 20 client.15996 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.717+0000 7ffa2838d740 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aacafb5420 0x55aacafb7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:37.722 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.717+0000 7ffa2838d740 4 client.15996 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:33:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.717+0000 7ffa2838d740 1 -- 172.21.15.59:0/757310749 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55aacae9c050 con 0x55aacafb5420 2024-05-08T05:33:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.717+0000 7ffa1effd640 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aacafb5420 0x55aacafb7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.717+0000 7ffa1effd640 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aacafb5420 0x55aacafb7840 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88ecac9130ba6ce8 server_cookie=a94f80651fb1870e in_seq=0 out_seq=0 2024-05-08T05:33:37.723 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.717+0000 7ffa02ff5640 10 client.15996 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:37.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.732+0000 7ffa02ff5640 1 -- 172.21.15.59:0/757310749 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55aacae9c050 con 0x55aacafb5420 2024-05-08T05:33:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa02ff5640 10 client.15996 handle_command_reply: tid=0 2024-05-08T05:33:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 client.15996 shutdown 2024-05-08T05:33:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 -- 172.21.15.59:0/757310749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2ef10 msgr2=0x55aacaf42090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2ef10 0x55aacaf42090 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ffa1800a290 tx=0x7ffa1801f850 comp rx=0 tx=0).stop 2024-05-08T05:33:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 -- 172.21.15.59:0/757310749 shutdown_connections 2024-05-08T05:33:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aacafb5420 0x55aacafb7840 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aacaf3a0f0 0x55aacaf3f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa02ff5640 0 client.15996 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aacaf2f830 0x55aacaf425d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 --2- 172.21.15.59:0/757310749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aacaf2ef10 0x55aacaf42090 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 -- 172.21.15.59:0/757310749 >> 172.21.15.59:0/757310749 conn(0x55aacaf2b420 msgr2=0x55aacaf310c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 -- 172.21.15.59:0/757310749 shutdown_connections 2024-05-08T05:33:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 1 -- 172.21.15.59:0/757310749 wait complete. 2024-05-08T05:33:37.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.733+0000 7ffa2838d740 20 client.15996 trim_cache size 0 max 16384 2024-05-08T05:33:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:33:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:33:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:33:37.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:33:37.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 -- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ffa080b2a10 msgr2=0x7ffa080b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ffa080b2a10 0x7ffa080b4ed0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55aacaf2c5b0 tx=0x7ffa140e7040 comp rx=0 tx=0).stop 2024-05-08T05:33:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 -- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20104580 msgr2=0x7ffa201996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20104580 0x7ffa201996b0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400d770 tx=0x7ffa1400dc30 comp rx=0 tx=0).stop 2024-05-08T05:33:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 -- 172.21.15.59:0/3273881802 shutdown_connections 2024-05-08T05:33:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ffa080b2a10 0x7ffa080b4ed0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ffa2010d030 0x7ffa2019d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.739 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ffa20104580 0x7ffa201996b0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 --2- 172.21.15.59:0/3273881802 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ffa20103c60 0x7ffa20199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 -- 172.21.15.59:0/3273881802 >> 172.21.15.59:0/3273881802 conn(0x7ffa20077c30 msgr2=0x7ffa200fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 -- 172.21.15.59:0/3273881802 shutdown_connections 2024-05-08T05:33:37.740 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:37.734+0000 7ffa25672640 1 -- 172.21.15.59:0/3273881802 wait complete. 2024-05-08T05:33:39.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:38 smithi096 ceph-mon[34253]: pgmap v125: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 183 op/s 2024-05-08T05:33:39.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:38 smithi059 ceph-mon[30304]: pgmap v125: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 183 op/s 2024-05-08T05:33:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:38 smithi143 ceph-mon[34337]: pgmap v125: 129 pgs: 129 active+clean; 40 MiB data, 766 MiB used, 1.0 TiB / 1.0 TiB avail; 987 KiB/s rd, 1.1 MiB/s wr, 183 op/s 2024-05-08T05:33:41.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:41 smithi096 ceph-mon[34253]: pgmap v126: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-05-08T05:33:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:41 smithi059 ceph-mon[30304]: pgmap v126: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-05-08T05:33:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:41 smithi143 ceph-mon[34337]: pgmap v126: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 994 KiB/s rd, 1.1 MiB/s wr, 190 op/s 2024-05-08T05:33:42.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:42 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:33:42.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:42 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:33:42.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:42 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:33:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:43 smithi096 ceph-mon[34253]: pgmap v127: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-08T05:33:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:43 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:33:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:43 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:33:43.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:43 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:33:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:43 smithi059 ceph-mon[30304]: pgmap v127: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-08T05:33:43.441 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:43 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:33:43.441 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:43 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:33:43.441 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:43 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:33:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:43 smithi143 ceph-mon[34337]: pgmap v127: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-08T05:33:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:43 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:33:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:43 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:33:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:43 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:33:45.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:45 smithi096 ceph-mon[34253]: pgmap v128: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-08T05:33:45.295 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:45 smithi143 ceph-mon[34337]: pgmap v128: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-08T05:33:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:45 smithi059 ceph-mon[30304]: pgmap v128: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 11 KiB/s wr, 13 op/s 2024-05-08T05:33:47.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:47 smithi096 ceph-mon[34253]: pgmap v129: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-05-08T05:33:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:47 smithi059 ceph-mon[30304]: pgmap v129: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-05-08T05:33:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:47 smithi143 ceph-mon[34337]: pgmap v129: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 11 KiB/s wr, 17 op/s 2024-05-08T05:33:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:49 smithi096 ceph-mon[34253]: pgmap v130: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-05-08T05:33:49.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:49 smithi059 ceph-mon[30304]: pgmap v130: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-05-08T05:33:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:49 smithi143 ceph-mon[34337]: pgmap v130: 129 pgs: 129 active+clean; 36 MiB data, 772 MiB used, 1.0 TiB / 1.0 TiB avail; 9.0 KiB/s rd, 7.1 KiB/s wr, 11 op/s 2024-05-08T05:33:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.837+0000 7efeb0c93640 1 -- 172.21.15.59:0/2809879580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 msgr2=0x7efeac10e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.837+0000 7efeb0c93640 1 --2- 172.21.15.59:0/2809879580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 0x7efeac10e760 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7efea4009930 tx=0x7efea402a420 comp rx=0 tx=0).stop 2024-05-08T05:33:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.837+0000 7efeb0c93640 1 -- 172.21.15.59:0/2809879580 shutdown_connections 2024-05-08T05:33:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.837+0000 7efeb0c93640 1 --2- 172.21.15.59:0/2809879580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 0x7efeac10e760 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.837+0000 7efeb0c93640 1 --2- 172.21.15.59:0/2809879580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeac1018e0 0x7efeac101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.837+0000 7efeb0c93640 1 --2- 172.21.15.59:0/2809879580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeac1078a0 0x7efeac107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.837+0000 7efeb0c93640 1 -- 172.21.15.59:0/2809879580 >> 172.21.15.59:0/2809879580 conn(0x7efeac078a30 msgr2=0x7efeac078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.837+0000 7efeb0c93640 1 -- 172.21.15.59:0/2809879580 shutdown_connections 2024-05-08T05:33:50.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.837+0000 7efeb0c93640 1 -- 172.21.15.59:0/2809879580 wait complete. 2024-05-08T05:33:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeb0c93640 1 Processor -- start 2024-05-08T05:33:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeb0c93640 1 -- start start 2024-05-08T05:33:50.840 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeb0c93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeac1018e0 0x7efeac198040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeb0c93640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 0x7efeac198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeb0c93640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeac1078a0 0x7efeac19c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeb0c93640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efeac111b60 con 0x7efeac1022a0 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeb0c93640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efeac1119e0 con 0x7efeac1018e0 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeb0c93640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efeac111ce0 con 0x7efeac1078a0 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efea9d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 0x7efeac198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efea9d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 0x7efeac198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50740/0 (socket says 172.21.15.59:50740) 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efea9d9b640 1 -- 172.21.15.59:0/1807134797 learned_addr learned my addr 172.21.15.59:0/1807134797 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeaa59c640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeac1018e0 0x7efeac198040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.838+0000 7efeaad9d640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeac1078a0 0x7efeac19c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efea9d9b640 1 -- 172.21.15.59:0/1807134797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeac1078a0 msgr2=0x7efeac19c860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efea9d9b640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeac1078a0 0x7efeac19c860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efea9d9b640 1 -- 172.21.15.59:0/1807134797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeac1018e0 msgr2=0x7efeac198040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:50.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efea9d9b640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeac1018e0 0x7efeac198040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efea9d9b640 1 -- 172.21.15.59:0/1807134797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efeac19cf90 con 0x7efeac1022a0 2024-05-08T05:33:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efea9d9b640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 0x7efeac198580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efea000d950 tx=0x7efea000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efe9b7fe640 1 -- 172.21.15.59:0/1807134797 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0017030 con 0x7efeac1022a0 2024-05-08T05:33:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efeb0c93640 1 -- 172.21.15.59:0/1807134797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efeac19d280 con 0x7efeac1022a0 2024-05-08T05:33:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efeb0c93640 1 -- 172.21.15.59:0/1807134797 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efeac1a4b10 con 0x7efeac1022a0 2024-05-08T05:33:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efe9b7fe640 1 -- 172.21.15.59:0/1807134797 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea0004020 con 0x7efeac1022a0 2024-05-08T05:33:50.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.839+0000 7efe9b7fe640 1 -- 172.21.15.59:0/1807134797 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea0004cb0 con 0x7efeac1022a0 2024-05-08T05:33:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efeb39b2740 1 Processor -- start 2024-05-08T05:33:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efeb39b2740 1 -- start start 2024-05-08T05:33:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efe9b7fe640 1 -- 172.21.15.59:0/1807134797 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efea0005020 con 0x7efeac1022a0 2024-05-08T05:33:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 0x56431f438020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f4385f0 0x56431f438a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f445240 0x56431f447630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efeb39b2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56431f39b5d0 con 0x56431f445240 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efeb39b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56431f354890 con 0x56431f4385f0 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efeb39b2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56431f397590 con 0x56431f4397d0 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efea9d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f445240 0x56431f447630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efea9d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f445240 0x56431f447630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50752/0 (socket says 172.21.15.59:50752) 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.842+0000 7efea9d9b640 1 -- 172.21.15.59:0/999420504 learned_addr learned my addr 172.21.15.59:0/999420504 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efeaa59c640 1 --2- 172.21.15.59:0/999420504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f4385f0 0x56431f438a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efeaad9d640 1 --2- 172.21.15.59:0/999420504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 0x56431f438020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efeaa59c640 1 -- 172.21.15.59:0/999420504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 msgr2=0x56431f438020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efeaa59c640 1 --2- 172.21.15.59:0/999420504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 0x56431f438020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efeaa59c640 1 -- 172.21.15.59:0/999420504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f445240 msgr2=0x56431f447630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efeaa59c640 1 --2- 172.21.15.59:0/999420504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f445240 0x56431f447630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efeaa59c640 1 -- 172.21.15.59:0/999420504 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431f0ad210 con 0x56431f4385f0 2024-05-08T05:33:50.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efe9b7fe640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efe940b2a90 0x7efe940b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efeaa59c640 1 --2- 172.21.15.59:0/999420504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f4385f0 0x56431f438a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009f30 tx=0x7efe9c010800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f3e3deed9af5c575 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efeaad9d640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efe940b2a90 0x7efe940b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efe98ff9640 1 -- 172.21.15.59:0/999420504 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c016070 con 0x56431f4385f0 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efe98ff9640 1 -- 172.21.15.59:0/999420504 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe9c004d20 con 0x56431f4385f0 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efe98ff9640 1 -- 172.21.15.59:0/999420504 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c005350 con 0x56431f4385f0 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.843+0000 7efe9b7fe640 1 -- 172.21.15.59:0/1807134797 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efea00e21b0 con 0x7efeac1022a0 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeb39b2740 1 -- 172.21.15.59:0/999420504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f4385f0 msgr2=0x56431f438a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeaad9d640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efe940b2a90 0x7efe940b4f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efea4005ab0 tx=0x7efea40058d0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeb39b2740 1 --2- 172.21.15.59:0/999420504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f4385f0 0x56431f438a70 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7efe9c009f30 tx=0x7efe9c010800 comp rx=0 tx=0).stop 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeb39b2740 1 -- 172.21.15.59:0/999420504 shutdown_connections 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeb39b2740 1 --2- 172.21.15.59:0/999420504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f445240 0x56431f447630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeb39b2740 1 --2- 172.21.15.59:0/999420504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f4385f0 0x56431f438a70 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeb39b2740 1 --2- 172.21.15.59:0/999420504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 0x56431f438020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeb39b2740 1 -- 172.21.15.59:0/999420504 >> 172.21.15.59:0/999420504 conn(0x56431f434630 msgr2=0x56431f434a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeb39b2740 1 -- 172.21.15.59:0/999420504 shutdown_connections 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.844+0000 7efeb39b2740 1 -- 172.21.15.59:0/999420504 wait complete. 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeb39b2740 1 -- start start 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeb39b2740 1 Processor -- start 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4385f0 0x56431f442100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f442640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f442b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeb39b2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56431f4480d0 con 0x56431f4397d0 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeb39b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x56431f358370 con 0x56431f445240 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeb39b2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x56431f448250 con 0x56431f4385f0 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeb39b2740 10 client.0 _ll_register_callbacks cb 0x56431f0aca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efea9d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f442640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efea9d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f442640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50764/0 (socket says 172.21.15.59:50764) 2024-05-08T05:33:50.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efea9d9b640 1 -- 172.21.15.59:0/761858445 learned_addr learned my addr 172.21.15.59:0/761858445 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeaa59c640 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4385f0 0x56431f442100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efeaad9d640 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f442b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efea9d9b640 1 -- 172.21.15.59:0/761858445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4385f0 msgr2=0x56431f442100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efea9d9b640 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4385f0 0x56431f442100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efea9d9b640 1 -- 172.21.15.59:0/761858445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 msgr2=0x56431f442b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efea9d9b640 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f442b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.845+0000 7efea9d9b640 1 -- 172.21.15.59:0/761858445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431f179620 con 0x56431f4397d0 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efea9d9b640 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f442640 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efea0009610 tx=0x7efea00a4fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aee39c9e06f2e041 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efe91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efe91ffb640 1 -- 172.21.15.59:0/761858445 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00a8890 con 0x56431f4397d0 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efe91ffb640 1 -- 172.21.15.59:0/761858445 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efea00041d0 con 0x56431f4397d0 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efeb39b2740 1 -- 172.21.15.59:0/761858445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe9c004d20 con 0x56431f4397d0 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efe91ffb640 1 -- 172.21.15.59:0/761858445 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efea00dfa20 con 0x56431f4397d0 2024-05-08T05:33:50.853 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efeb39b2740 10 client.16008 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efeb39b2740 1 -- 172.21.15.59:0/761858445 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56431f454090 con 0x56431f4397d0 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efe91ffb640 1 -- 172.21.15.59:0/761858445 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe9c004d20 con 0x56431f4397d0 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efe91ffb640 1 -- 172.21.15.59:0/761858445 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7efea00dfce0 con 0x56431f4397d0 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efeb39b2740 10 client.16008 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efeb39b2740 10 client.16008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efeb39b2740 20 client.16008 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efeb39b2740 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56431f4555c0 0x56431f457a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efeb39b2740 4 client.16008 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.846+0000 7efeb39b2740 1 -- 172.21.15.59:0/761858445 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56431f3a91e0 con 0x56431f4555c0 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.847+0000 7efeaa59c640 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56431f4555c0 0x56431f457a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.847+0000 7efeaa59c640 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56431f4555c0 0x56431f457a00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35de429bce55cfa6 server_cookie=99a7f9975f491573 in_seq=0 out_seq=0 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.847+0000 7efe91ffb640 10 client.16008 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efe91ffb640 1 -- 172.21.15.59:0/761858445 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x56431f3a91e0 con 0x56431f4555c0 2024-05-08T05:33:50.854 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efe91ffb640 10 client.16008 handle_command_reply: tid=0 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 client.16008 shutdown 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 -- 172.21.15.59:0/761858445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 msgr2=0x56431f442640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f442640 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7efea0009610 tx=0x7efea00a4fe0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 -- 172.21.15.59:0/761858445 shutdown_connections 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56431f4555c0 0x56431f457a00 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f442b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f442640 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 --2- 172.21.15.59:0/761858445 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4385f0 0x56431f442100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 -- 172.21.15.59:0/761858445 >> 172.21.15.59:0/761858445 conn(0x56431f434630 msgr2=0x56431f43a4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efe91ffb640 0 client.16008 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 -- 172.21.15.59:0/761858445 shutdown_connections 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 1 -- 172.21.15.59:0/761858445 wait complete. 2024-05-08T05:33:50.855 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.848+0000 7efeb39b2740 20 client.16008 trim_cache size 0 max 16384 2024-05-08T05:33:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efeb39b2740 1 Processor -- start 2024-05-08T05:33:50.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efeb39b2740 1 -- start start 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4385f0 0x56431f433820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f42d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f433dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efeb39b2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c00fe80 con 0x56431f4397d0 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efeb39b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c011a50 con 0x56431f445240 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efeb39b2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c010550 con 0x56431f4385f0 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efea9d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f42d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efea9d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f42d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50776/0 (socket says 172.21.15.59:50776) 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efea9d9b640 1 -- 172.21.15.59:0/2770299771 learned_addr learned my addr 172.21.15.59:0/2770299771 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efea9d9b640 1 -- 172.21.15.59:0/2770299771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4385f0 msgr2=0x56431f433820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efea9d9b640 1 --2- 172.21.15.59:0/2770299771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4385f0 0x56431f433820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efea9d9b640 1 -- 172.21.15.59:0/2770299771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 msgr2=0x56431f433dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efea9d9b640 1 --2- 172.21.15.59:0/2770299771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f433dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efea9d9b640 1 -- 172.21.15.59:0/2770299771 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56431f42c960 con 0x56431f4397d0 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.864+0000 7efea9d9b640 1 --2- 172.21.15.59:0/2770299771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f42d3e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efea0005d80 tx=0x7efea00a5110 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf4b7a666b76a68a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efe93fff640 1 -- 172.21.15.59:0/2770299771 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00a8890 con 0x56431f4397d0 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efe93fff640 1 -- 172.21.15.59:0/2770299771 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea0004e50 con 0x56431f4397d0 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efe93fff640 1 -- 172.21.15.59:0/2770299771 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efea00a8ef0 con 0x56431f4397d0 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 -- 172.21.15.59:0/2770299771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 msgr2=0x56431f42d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 --2- 172.21.15.59:0/2770299771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f42d3e0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7efea0005d80 tx=0x7efea00a5110 comp rx=0 tx=0).stop 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 -- 172.21.15.59:0/2770299771 shutdown_connections 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 --2- 172.21.15.59:0/2770299771 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f433dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 --2- 172.21.15.59:0/2770299771 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4397d0 0x56431f42d3e0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 --2- 172.21.15.59:0/2770299771 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4385f0 0x56431f433820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 -- 172.21.15.59:0/2770299771 >> 172.21.15.59:0/2770299771 conn(0x56431f434630 msgr2=0x56431f434a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 -- 172.21.15.59:0/2770299771 shutdown_connections 2024-05-08T05:33:50.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 -- 172.21.15.59:0/2770299771 wait complete. 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.865+0000 7efeb39b2740 1 -- start start 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeb39b2740 1 Processor -- start 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4385f0 0x56431f45cc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 0x56431f45d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeb39b2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f45d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeb39b2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c0112f0 con 0x56431f4385f0 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeb39b2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c010c20 con 0x56431f445240 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeb39b2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efe8c00de40 con 0x56431f4397d0 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeb39b2740 10 client.0 _ll_register_callbacks cb 0x7efea0005cd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaa59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4385f0 0x56431f45cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaa59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4385f0 0x56431f45cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50786/0 (socket says 172.21.15.59:50786) 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaa59c640 1 -- 172.21.15.59:0/1667447611 learned_addr learned my addr 172.21.15.59:0/1667447611 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:50.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaad9d640 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f45d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efea9d9b640 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 0x56431f45d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaa59c640 1 -- 172.21.15.59:0/1667447611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 msgr2=0x56431f45d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaa59c640 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 0x56431f45d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaa59c640 1 -- 172.21.15.59:0/1667447611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 msgr2=0x56431f45d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaa59c640 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f45d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaa59c640 1 -- 172.21.15.59:0/1667447611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56431f4bf020 con 0x56431f4385f0 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaad9d640 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f45d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efeaa59c640 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4385f0 0x56431f45cc30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005c40 tx=0x7efe9c012740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61d73db56ccd21ec server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.866+0000 7efe91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:50.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efe91ffb640 1 -- 172.21.15.59:0/1667447611 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe9c016040 con 0x56431f4385f0 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efe91ffb640 1 -- 172.21.15.59:0/1667447611 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efe9c003970 con 0x56431f4385f0 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efeb39b2740 1 -- 172.21.15.59:0/1667447611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efea0004e50 con 0x56431f4385f0 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efe91ffb640 1 -- 172.21.15.59:0/1667447611 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7efe9c012cc0 con 0x56431f4385f0 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efeb39b2740 10 client.16014 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efeb39b2740 1 -- 172.21.15.59:0/1667447611 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x56431f4c02d0 con 0x56431f4385f0 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efe91ffb640 1 -- 172.21.15.59:0/1667447611 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efea0004e50 con 0x56431f4385f0 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efe91ffb640 1 -- 172.21.15.59:0/1667447611 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7efe9c00f040 con 0x56431f4385f0 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efeb39b2740 10 client.16014 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efeb39b2740 10 client.16014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efeb39b2740 20 client.16014 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efeb39b2740 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56431f4c0880 0x56431f4c2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efeb39b2740 4 client.16014 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efeb39b2740 1 -- 172.21.15.59:0/1667447611 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56431f3a3050 con 0x56431f4c0880 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.867+0000 7efea9d9b640 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56431f4c0880 0x56431f4c2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:50.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.868+0000 7efea9d9b640 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56431f4c0880 0x56431f4c2ca0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe6ee5164b4fa6f server_cookie=9cfe313874ac9069 in_seq=0 out_seq=0 2024-05-08T05:33:50.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.868+0000 7efe91ffb640 10 client.16014 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efe91ffb640 1 -- 172.21.15.59:0/1667447611 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56431f3a3050 con 0x56431f4c0880 2024-05-08T05:33:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efe91ffb640 10 client.16014 handle_command_reply: tid=0 2024-05-08T05:33:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 client.16014 shutdown 2024-05-08T05:33:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 -- 172.21.15.59:0/1667447611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4385f0 msgr2=0x56431f45cc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:50.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4385f0 0x56431f45cc30 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7efe9c005c40 tx=0x7efe9c012740 comp rx=0 tx=0).stop 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 -- 172.21.15.59:0/1667447611 shutdown_connections 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56431f4c0880 0x56431f4c2ca0 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x56431f445240 0x56431f45d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x56431f4397d0 0x56431f45d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efe91ffb640 0 client.16014 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 --2- 172.21.15.59:0/1667447611 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x56431f4385f0 0x56431f45cc30 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 -- 172.21.15.59:0/1667447611 >> 172.21.15.59:0/1667447611 conn(0x56431f434630 msgr2=0x56431f42d3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 -- 172.21.15.59:0/1667447611 shutdown_connections 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.882+0000 7efeb39b2740 1 -- 172.21.15.59:0/1667447611 wait complete. 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb39b2740 20 client.16014 trim_cache size 0 max 16384 2024-05-08T05:33:50.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:33:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:33:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:33:50.885 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 -- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efe940b2a90 msgr2=0x7efe940b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efe940b2a90 0x7efe940b4f50 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7efea4005ab0 tx=0x7efea40058d0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 -- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 msgr2=0x7efeac198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 0x7efeac198580 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7efea000d950 tx=0x7efea000de10 comp rx=0 tx=0).stop 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 -- 172.21.15.59:0/1807134797 shutdown_connections 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efe940b2a90 0x7efe940b4f50 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efeac1078a0 0x7efeac19c860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efeac1022a0 0x7efeac198580 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 --2- 172.21.15.59:0/1807134797 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efeac1018e0 0x7efeac198040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 -- 172.21.15.59:0/1807134797 >> 172.21.15.59:0/1807134797 conn(0x7efeac078a30 msgr2=0x7efeac104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 -- 172.21.15.59:0/1807134797 shutdown_connections 2024-05-08T05:33:50.886 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:33:50.883+0000 7efeb0c93640 1 -- 172.21.15.59:0/1807134797 wait complete. 2024-05-08T05:33:51.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:51 smithi096 ceph-mon[34253]: pgmap v131: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-05-08T05:33:51.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:51 smithi059 ceph-mon[30304]: pgmap v131: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-05-08T05:33:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:51 smithi143 ceph-mon[34337]: pgmap v131: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 15 KiB/s wr, 16 op/s 2024-05-08T05:33:52.708 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:33:52.975 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:33:53.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:53 smithi096 ceph-mon[34253]: pgmap v132: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-05-08T05:33:53.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:53 smithi059 ceph-mon[30304]: pgmap v132: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-05-08T05:33:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:53 smithi143 ceph-mon[34337]: pgmap v132: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-05-08T05:33:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.769+0000 7f938d8b3700 1 -- 172.21.15.59:0/2913841083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388102450 msgr2=0x7f9388110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.769+0000 7f938d8b3700 1 --2- 172.21.15.59:0/2913841083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388102450 0x7f9388110340 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f938000b0a0 tx=0x7f938000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:33:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.769+0000 7f938d8b3700 1 -- 172.21.15.59:0/2913841083 shutdown_connections 2024-05-08T05:33:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.769+0000 7f938d8b3700 1 --2- 172.21.15.59:0/2913841083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388102450 0x7f9388110340 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.769+0000 7f938d8b3700 1 --2- 172.21.15.59:0/2913841083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388109520 0x7f9388109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.769+0000 7f938d8b3700 1 --2- 172.21.15.59:0/2913841083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9388101a90 0x7f9388101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.769+0000 7f938d8b3700 1 -- 172.21.15.59:0/2913841083 >> 172.21.15.59:0/2913841083 conn(0x7f9388078a10 msgr2=0x7f93880ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.769+0000 7f938d8b3700 1 -- 172.21.15.59:0/2913841083 shutdown_connections 2024-05-08T05:33:54.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.769+0000 7f938d8b3700 1 -- 172.21.15.59:0/2913841083 wait complete. 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f938d8b3700 1 Processor -- start 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f938d8b3700 1 -- start start 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f938d8b3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9388101a90 0x7f9388102c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f938d8b3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388102450 0x7f93881031b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f938d8b3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388109520 0x7f93881036f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f938d8b3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9388113630 con 0x7f9388109520 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f938d8b3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93881134b0 con 0x7f9388101a90 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f938d8b3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f93881137b0 con 0x7f9388102450 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f93877fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388109520 0x7f93881036f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f93877fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388109520 0x7f93881036f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50802/0 (socket says 172.21.15.59:50802) 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f93877fe700 1 -- 172.21.15.59:0/3842463149 learned_addr learned my addr 172.21.15.59:0/3842463149 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.770+0000 7f93867fc700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388102450 0x7f93881031b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:54.773 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f9386ffd700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9388101a90 0x7f9388102c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f93877fe700 1 -- 172.21.15.59:0/3842463149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388102450 msgr2=0x7f93881031b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f93877fe700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388102450 0x7f93881031b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f93877fe700 1 -- 172.21.15.59:0/3842463149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9388101a90 msgr2=0x7f9388102c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f93877fe700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9388101a90 0x7f9388102c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f93877fe700 1 -- 172.21.15.59:0/3842463149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93881a7ff0 con 0x7f9388109520 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f93877fe700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388109520 0x7f93881036f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f938000b830 tx=0x7f9380004ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f938c8b1700 1 -- 172.21.15.59:0/3842463149 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9380021070 con 0x7f9388109520 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93881a8280 con 0x7f9388109520 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f938c8b1700 1 -- 172.21.15.59:0/3842463149 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9380007e30 con 0x7f9388109520 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f938c8b1700 1 -- 172.21.15.59:0/3842463149 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93800047d0 con 0x7f9388109520 2024-05-08T05:33:54.774 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.771+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93881a86e0 con 0x7f9388109520 2024-05-08T05:33:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.772+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f938806a190 con 0x7f9388109520 2024-05-08T05:33:54.775 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.774+0000 7f938c8b1700 1 -- 172.21.15.59:0/3842463149 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9380007980 con 0x7f9388109520 2024-05-08T05:33:54.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.775+0000 7f938c8b1700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f93700b29f0 0x7f93700b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:54.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.775+0000 7f9386ffd700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f93700b29f0 0x7f93700b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:54.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.775+0000 7f938c8b1700 1 -- 172.21.15.59:0/3842463149 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f93800e3a00 con 0x7f9388109520 2024-05-08T05:33:54.776 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.776+0000 7f9386ffd700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f93700b29f0 0x7f93700b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9378005950 tx=0x7f93780058e0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:54.779 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:54.779+0000 7f938c8b1700 1 -- 172.21.15.59:0/3842463149 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f93800ab100 con 0x7f9388109520 2024-05-08T05:33:55.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.045+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f938802e6d0 con 0x7f9388109520 2024-05-08T05:33:55.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.046+0000 7f938c8b1700 1 -- 172.21.15.59:0/3842463149 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) ==== 94+0+11934 (secure 0 0 0) 0x7f93800aaa00 con 0x7f9388109520 2024-05-08T05:33:55.046 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:33:55.046 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":33,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":33,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:32:11.760389+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[1,2,4],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f93700b29f0 msgr2=0x7f93700b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f93700b29f0 0x7f93700b4eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f9378005950 tx=0x7f93780058e0 comp rx=0 tx=0).stop 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388109520 msgr2=0x7f93881036f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388109520 0x7f93881036f0 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f938000b830 tx=0x7f9380004ce0 comp rx=0 tx=0).stop 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 shutdown_connections 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f93700b29f0 0x7f93700b4eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9388109520 0x7f93881036f0 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9388101a90 0x7f9388102c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 --2- 172.21.15.59:0/3842463149 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9388102450 0x7f93881031b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:55.049 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.048+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 >> 172.21.15.59:0/3842463149 conn(0x7f9388078a10 msgr2=0x7f9388100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:55.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.049+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 shutdown_connections 2024-05-08T05:33:55.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:55.049+0000 7f938d8b3700 1 -- 172.21.15.59:0/3842463149 wait complete. 2024-05-08T05:33:55.050 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 33 2024-05-08T05:33:55.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:55 smithi096 ceph-mon[34253]: pgmap v133: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-05-08T05:33:55.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:55 smithi059 ceph-mon[30304]: pgmap v133: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-05-08T05:33:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:55 smithi143 ceph-mon[34337]: pgmap v133: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 8.4 KiB/s wr, 9 op/s 2024-05-08T05:33:55.860 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db --set-id cc2da2d4 --release --await' 2024-05-08T05:33:55.860 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db --set-id cc2da2d4 --release --await 2024-05-08T05:33:56.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:56 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3842463149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:33:56.121 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:33:56.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:56 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3842463149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:33:56.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:56 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3842463149' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:33:57.334 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:57 smithi059 ceph-mon[30304]: pgmap v134: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-05-08T05:33:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:57 smithi143 ceph-mon[34337]: pgmap v134: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-05-08T05:33:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:57 smithi096 ceph-mon[34253]: pgmap v134: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 15 KiB/s wr, 15 op/s 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.914+0000 7faaa9015700 1 -- 172.21.15.59:0/174535248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa4109510 msgr2=0x7faaa41098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.914+0000 7faaa9015700 1 --2- 172.21.15.59:0/174535248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa4109510 0x7faaa41098f0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7faa9400b0a0 tx=0x7faa9400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.914+0000 7faaa9015700 1 -- 172.21.15.59:0/174535248 shutdown_connections 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.914+0000 7faaa9015700 1 --2- 172.21.15.59:0/174535248 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaa406bb40 0x7faaa4110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.914+0000 7faaa9015700 1 --2- 172.21.15.59:0/174535248 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa4109510 0x7faaa41098f0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.914+0000 7faaa9015700 1 --2- 172.21.15.59:0/174535248 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaa406b180 0x7faaa406b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.914+0000 7faaa9015700 1 -- 172.21.15.59:0/174535248 >> 172.21.15.59:0/174535248 conn(0x7faaa4079350 msgr2=0x7faaa4079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.914+0000 7faaa9015700 1 -- 172.21.15.59:0/174535248 shutdown_connections 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.914+0000 7faaa9015700 1 -- 172.21.15.59:0/174535248 wait complete. 2024-05-08T05:33:57.915 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa9015700 1 Processor -- start 2024-05-08T05:33:57.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa9015700 1 -- start start 2024-05-08T05:33:57.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa9015700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaa406b180 0x7faaa4102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa9015700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa406bb40 0x7faaa41029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa9015700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaa4109510 0x7faaa4102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa9015700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faaa4113670 con 0x7faaa406bb40 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa9015700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faaa41134f0 con 0x7faaa406b180 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa9015700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7faaa41137f0 con 0x7faaa4109510 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa406bb40 0x7faaa41029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa259c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa406bb40 0x7faaa41029b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50826/0 (socket says 172.21.15.59:50826) 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.915+0000 7faaa259c700 1 -- 172.21.15.59:0/3170890270 learned_addr learned my addr 172.21.15.59:0/3170890270 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaa406b180 0x7faaa4102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa359e700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaa4109510 0x7faaa4102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa259c700 1 -- 172.21.15.59:0/3170890270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaa4109510 msgr2=0x7faaa4102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa259c700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaa4109510 0x7faaa4102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa259c700 1 -- 172.21.15.59:0/3170890270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaa406b180 msgr2=0x7faaa4102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa259c700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaa406b180 0x7faaa4102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa259c700 1 -- 172.21.15.59:0/3170890270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faaa41a80d0 con 0x7faaa406bb40 2024-05-08T05:33:57.918 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa259c700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa406bb40 0x7faaa41029b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faa9800d950 tx=0x7faa9800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:57.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faa8bfff700 1 -- 172.21.15.59:0/3170890270 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa98016070 con 0x7faaa406bb40 2024-05-08T05:33:57.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa9015700 1 -- 172.21.15.59:0/3170890270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faaa41a83c0 con 0x7faaa406bb40 2024-05-08T05:33:57.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faa8bfff700 1 -- 172.21.15.59:0/3170890270 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa98009df0 con 0x7faaa406bb40 2024-05-08T05:33:57.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faa8bfff700 1 -- 172.21.15.59:0/3170890270 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9800f3c0 con 0x7faaa406bb40 2024-05-08T05:33:57.919 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.916+0000 7faaa9015700 1 -- 172.21.15.59:0/3170890270 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faaa41a8950 con 0x7faaa406bb40 2024-05-08T05:33:57.920 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.920+0000 7faa8bfff700 1 -- 172.21.15.59:0/3170890270 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faa98019070 con 0x7faaa406bb40 2024-05-08T05:33:57.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.920+0000 7faa8bfff700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faa8c0b29f0 0x7faa8c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faa8c0b29f0 0x7faa8c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faab38a8b80 1 Processor -- start 2024-05-08T05:33:57.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faa8bfff700 1 -- 172.21.15.59:0/3170890270 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7faa980e25a0 con 0x7faaa406bb40 2024-05-08T05:33:57.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faab38a8b80 1 -- start start 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49ffa0 0x55bada4a0380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 0x55bada49b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 0x55bada49c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faab38a8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bada3ed960 con 0x55bada49ffa0 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faab38a8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bada38c8c0 con 0x55bada49bdc0 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faab38a8b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bada3e9920 con 0x55bada4a08c0 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faa8c0b29f0 0x7faa8c0b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa94009fd0 tx=0x7faa94006210 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faaa2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49ffa0 0x55bada4a0380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faaa2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49ffa0 0x55bada4a0380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50832/0 (socket says 172.21.15.59:50832) 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faaa2d9d700 1 -- 172.21.15.59:0/1511405010 learned_addr learned my addr 172.21.15.59:0/1511405010 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:57.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faaa359e700 1 --2- 172.21.15.59:0/1511405010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 0x55bada49c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.921+0000 7faaa259c700 1 --2- 172.21.15.59:0/1511405010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 0x55bada49b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.922+0000 7faaa2d9d700 1 -- 172.21.15.59:0/1511405010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 msgr2=0x55bada49b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.922+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/1511405010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 0x55bada49b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.922+0000 7faaa2d9d700 1 -- 172.21.15.59:0/1511405010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 msgr2=0x55bada49c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.922+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/1511405010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 0x55bada49c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.922+0000 7faaa2d9d700 1 -- 172.21.15.59:0/1511405010 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bada33a9d0 con 0x55bada49ffa0 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.922+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/1511405010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49ffa0 0x55bada4a0380 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faa94015000 tx=0x7faa9401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d86c7ec214546cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.922+0000 7faa897fa700 1 -- 172.21.15.59:0/1511405010 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9402e070 con 0x55bada49ffa0 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.922+0000 7faa897fa700 1 -- 172.21.15.59:0/1511405010 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa94007b90 con 0x55bada49ffa0 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.922+0000 7faa897fa700 1 -- 172.21.15.59:0/1511405010 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa940057f0 con 0x55bada49ffa0 2024-05-08T05:33:57.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 -- 172.21.15.59:0/1511405010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49ffa0 msgr2=0x55bada4a0380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 --2- 172.21.15.59:0/1511405010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49ffa0 0x55bada4a0380 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7faa94015000 tx=0x7faa9401d000 comp rx=0 tx=0).stop 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 -- 172.21.15.59:0/1511405010 shutdown_connections 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 --2- 172.21.15.59:0/1511405010 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 0x55bada49c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 --2- 172.21.15.59:0/1511405010 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49ffa0 0x55bada4a0380 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 --2- 172.21.15.59:0/1511405010 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 0x55bada49b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 -- 172.21.15.59:0/1511405010 >> 172.21.15.59:0/1511405010 conn(0x55bada3c7200 msgr2=0x55bada495fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 -- 172.21.15.59:0/1511405010 shutdown_connections 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 -- 172.21.15.59:0/1511405010 wait complete. 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 -- start start 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 Processor -- start 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4a7630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49ffa0 0x55bada4a49d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 0x55bada4a4f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55bada4a8430 con 0x55bada49bdc0 2024-05-08T05:33:57.927 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.923+0000 7faab38a8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55bada353d30 con 0x55bada49ffa0 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faab38a8b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55bada4a85b0 con 0x55bada4a08c0 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4a7630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faab38a8b80 10 client.0 _ll_register_callbacks cb 0x55bada385a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4a7630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50842/0 (socket says 172.21.15.59:50842) 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa2d9d700 1 -- 172.21.15.59:0/2130699347 learned_addr learned my addr 172.21.15.59:0/2130699347 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa359e700 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 0x55bada4a4f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa259c700 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49ffa0 0x55bada4a49d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa2d9d700 1 -- 172.21.15.59:0/2130699347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 msgr2=0x55bada4a4f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 0x55bada4a4f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa2d9d700 1 -- 172.21.15.59:0/2130699347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49ffa0 msgr2=0x55bada4a49d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49ffa0 0x55bada4a49d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa2d9d700 1 -- 172.21.15.59:0/2130699347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bada389790 con 0x55bada49bdc0 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4a7630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faa94009c50 tx=0x7faa940091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea74cb1f09b75697 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:57.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faa86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:57.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faa86ff5700 1 -- 172.21.15.59:0/2130699347 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9402e030 con 0x55bada49bdc0 2024-05-08T05:33:57.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faa86ff5700 1 -- 172.21.15.59:0/2130699347 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7faa94007910 con 0x55bada49bdc0 2024-05-08T05:33:57.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.924+0000 7faab38a8b80 1 -- 172.21.15.59:0/2130699347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa94007b90 con 0x55bada49bdc0 2024-05-08T05:33:57.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faa86ff5700 1 -- 172.21.15.59:0/2130699347 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faa94043050 con 0x55bada49bdc0 2024-05-08T05:33:57.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faab38a8b80 10 client.16032 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:57.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faab38a8b80 1 -- 172.21.15.59:0/2130699347 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bada4b2be0 con 0x55bada49bdc0 2024-05-08T05:33:57.930 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faa86ff5700 1 -- 172.21.15.59:0/2130699347 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa94007b90 con 0x55bada49bdc0 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faa86ff5700 1 -- 172.21.15.59:0/2130699347 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7faa94034070 con 0x55bada49bdc0 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faab38a8b80 10 client.16032 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faab38a8b80 10 client.16032 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faab38a8b80 20 client.16032 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faab38a8b80 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55bada4b50b0 0x55bada4b74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faab38a8b80 4 client.16032 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faab38a8b80 1 -- 172.21.15.59:0/2130699347 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bada467750 con 0x55bada4b50b0 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.925+0000 7faaa259c700 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55bada4b50b0 0x55bada4b74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.926+0000 7faaa259c700 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55bada4b50b0 0x55bada4b74a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43a5e8c0f41156c7 server_cookie=850ff82fa2a06043 in_seq=0 out_seq=0 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.926+0000 7faa86ff5700 10 client.16032 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.926+0000 7faa86ff5700 1 -- 172.21.15.59:0/2130699347 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55bada467750 con 0x55bada4b50b0 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.926+0000 7faa86ff5700 10 client.16032 handle_command_reply: tid=0 2024-05-08T05:33:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.926+0000 7faab38a8b80 1 client.16032 shutdown 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 -- 172.21.15.59:0/2130699347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 msgr2=0x55bada4a7630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4a7630 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7faa94009c50 tx=0x7faa940091c0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 -- 172.21.15.59:0/2130699347 shutdown_connections 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55bada4b50b0 0x55bada4b74a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada4a08c0 0x55bada4a4f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4a7630 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faa86ff5700 0 client.16032 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 --2- 172.21.15.59:0/2130699347 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49ffa0 0x55bada4a49d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 -- 172.21.15.59:0/2130699347 >> 172.21.15.59:0/2130699347 conn(0x55bada3c7200 msgr2=0x55bada494da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 -- 172.21.15.59:0/2130699347 shutdown_connections 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 1 -- 172.21.15.59:0/2130699347 wait complete. 2024-05-08T05:33:57.932 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.927+0000 7faab38a8b80 20 client.16032 trim_cache size 0 max 16384 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faab38a8b80 1 Processor -- start 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faab38a8b80 1 -- start start 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 0x55bada4a1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada4a08c0 0x55bada4b0870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faab38a8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faaa41b1690 con 0x55bada4a08c0 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faab38a8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faaa41b3260 con 0x55bada49bdc0 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faab38a8b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7faaa41b1d60 con 0x55bada49ffa0 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faaa359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada4a08c0 0x55bada4b0870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faaa359e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada4a08c0 0x55bada4b0870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50850/0 (socket says 172.21.15.59:50850) 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.947+0000 7faaa359e700 1 -- 172.21.15.59:0/26215179 learned_addr learned my addr 172.21.15.59:0/26215179 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:57.952 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 0x55bada4a1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa259c700 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa359e700 1 -- 172.21.15.59:0/26215179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 msgr2=0x55bada4a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa359e700 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa359e700 1 -- 172.21.15.59:0/26215179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 msgr2=0x55bada4a1cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa359e700 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 0x55bada4a1cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa359e700 1 -- 172.21.15.59:0/26215179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bada49fbf0 con 0x55bada4a08c0 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa259c700 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 0x55bada4a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faaa359e700 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada4a08c0 0x55bada4b0870 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00dd20 tx=0x7faa9c012450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83f6cffd71f9e3f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faa88ff9700 1 -- 172.21.15.59:0/26215179 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c01f020 con 0x55bada4a08c0 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faa88ff9700 1 -- 172.21.15.59:0/26215179 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa9c01b3f0 con 0x55bada4a08c0 2024-05-08T05:33:57.953 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.948+0000 7faa88ff9700 1 -- 172.21.15.59:0/26215179 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9c010630 con 0x55bada4a08c0 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 -- 172.21.15.59:0/26215179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada4a08c0 msgr2=0x55bada4b0870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada4a08c0 0x55bada4b0870 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7faa9c00dd20 tx=0x7faa9c012450 comp rx=0 tx=0).stop 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 -- 172.21.15.59:0/26215179 shutdown_connections 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada4a08c0 0x55bada4b0870 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada49bdc0 0x55bada4a1cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 --2- 172.21.15.59:0/26215179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 -- 172.21.15.59:0/26215179 >> 172.21.15.59:0/26215179 conn(0x55bada3c7200 msgr2=0x55bada4a32c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 -- 172.21.15.59:0/26215179 shutdown_connections 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 -- 172.21.15.59:0/26215179 wait complete. 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 -- start start 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.949+0000 7faab38a8b80 1 Processor -- start 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4b85b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faab38a8b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada4a08c0 0x55bada4b9030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.954 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faab38a8b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faaa41b2b00 con 0x55bada49bdc0 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faab38a8b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faaa41b2430 con 0x55bada4a08c0 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faab38a8b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7faaa41af610 con 0x55bada49ffa0 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faab38a8b80 10 client.0 _ll_register_callbacks cb 0x55bada385a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4b85b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa2d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4b85b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50856/0 (socket says 172.21.15.59:50856) 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa2d9d700 1 -- 172.21.15.59:0/565829359 learned_addr learned my addr 172.21.15.59:0/565829359 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa259c700 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa359e700 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada4a08c0 0x55bada4b9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa2d9d700 1 -- 172.21.15.59:0/565829359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 msgr2=0x55bada4b8af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4b8af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa2d9d700 1 -- 172.21.15.59:0/565829359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada4a08c0 msgr2=0x55bada4b9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada4a08c0 0x55bada4b9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa2d9d700 1 -- 172.21.15.59:0/565829359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bada527df0 con 0x55bada49bdc0 2024-05-08T05:33:57.955 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa259c700 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa359e700 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada4a08c0 0x55bada4b9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faaa2d9d700 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4b85b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7faa94000f80 tx=0x7faa940050b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=946ce426a03c0a6b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faa86ff5700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faa86ff5700 1 -- 172.21.15.59:0/565829359 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa9402e070 con 0x55bada49bdc0 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faa86ff5700 1 -- 172.21.15.59:0/565829359 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7faa94007910 con 0x55bada49bdc0 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.950+0000 7faab38a8b80 1 -- 172.21.15.59:0/565829359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa9c01b3f0 con 0x55bada49bdc0 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faa86ff5700 1 -- 172.21.15.59:0/565829359 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=33) ==== 24+0+0 (secure 0 0 0) 0x7faa94050050 con 0x55bada49bdc0 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faab38a8b80 10 client.16038 fetch_fsmap learned FSMap version 33 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faab38a8b80 1 -- 172.21.15.59:0/565829359 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=33,monmap=4+}) -- 0x55bada529040 con 0x55bada49bdc0 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faa86ff5700 1 -- 172.21.15.59:0/565829359 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa9c01b3f0 con 0x55bada49bdc0 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faa86ff5700 1 -- 172.21.15.59:0/565829359 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 33) ==== 8374+0+0 (secure 0 0 0) 0x7faa94034040 con 0x55bada49bdc0 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faab38a8b80 10 client.16038 fetch_fsmap finished waiting for FSMap version 33 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faab38a8b80 10 client.16038 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faab38a8b80 20 client.16038 populate_metadata read hostname 'smithi059' 2024-05-08T05:33:57.956 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faab38a8b80 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55bada529690 0x55bada52baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:33:57.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faab38a8b80 4 client.16038 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "cc2da2d4", "release": true, "await": true}] 2024-05-08T05:33:57.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faab38a8b80 1 -- 172.21.15.59:0/565829359 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "cc2da2d4", "release": true, "await": true}) -- 0x55bada496540 con 0x55bada529690 2024-05-08T05:33:57.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.951+0000 7faaa259c700 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55bada529690 0x55bada52baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:33:57.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.952+0000 7faa86ff5700 10 client.16038 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:57.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:57.952+0000 7faaa259c700 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55bada529690 0x55bada52baa0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4687d267e3ef175c server_cookie=b64b91e05d80d493 in_seq=0 out_seq=0 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.168+0000 7faa86ff5700 1 -- 172.21.15.59:0/565829359 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55bada496540 con 0x55bada529690 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.168+0000 7faa86ff5700 10 client.16038 handle_command_reply: tid=0 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 client.16038 shutdown 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 -- 172.21.15.59:0/565829359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 msgr2=0x55bada4b85b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4b85b0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7faa94000f80 tx=0x7faa940050b0 comp rx=0 tx=0).stop 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 -- 172.21.15.59:0/565829359 shutdown_connections 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55bada529690 0x55bada52baa0 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55bada4a08c0 0x55bada4b9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55bada49bdc0 0x55bada4b85b0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 --2- 172.21.15.59:0/565829359 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55bada49ffa0 0x55bada4b8af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 -- 172.21.15.59:0/565829359 >> 172.21.15.59:0/565829359 conn(0x55bada3c7200 msgr2=0x55bada49ebe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faa86ff5700 0 client.16038 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 -- 172.21.15.59:0/565829359 shutdown_connections 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 1 -- 172.21.15.59:0/565829359 wait complete. 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.169+0000 7faab38a8b80 20 client.16038 trim_cache size 0 max 16384 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 33, 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 16, 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stdout: "cc2da2d4": { 2024-05-08T05:33:58.171 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 16, 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 81.0, 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:33:58.172 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:33:58.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:58.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:58.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:58.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:58.173 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:33:58.173 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 -- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faa8c0b29f0 msgr2=0x7faa8c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faa8c0b29f0 0x7faa8c0b4eb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7faa94009fd0 tx=0x7faa94006210 comp rx=0 tx=0).stop 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 -- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa406bb40 msgr2=0x7faaa41029b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa406bb40 0x7faaa41029b0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7faa9800d950 tx=0x7faa9800dc60 comp rx=0 tx=0).stop 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 -- 172.21.15.59:0/3170890270 shutdown_connections 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faa8c0b29f0 0x7faa8c0b4eb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faaa4109510 0x7faaa4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faaa406b180 0x7faaa4102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 --2- 172.21.15.59:0/3170890270 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faaa406bb40 0x7faaa41029b0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 -- 172.21.15.59:0/3170890270 >> 172.21.15.59:0/3170890270 conn(0x7faaa4079350 msgr2=0x7faaa41011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 -- 172.21.15.59:0/3170890270 shutdown_connections 2024-05-08T05:33:58.174 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:33:58.170+0000 7faaa9015700 1 -- 172.21.15.59:0/3170890270 wait complete. 2024-05-08T05:33:58.955 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'cc2da2d4', seconds elapsed: 34.9 2024-05-08T05:33:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:33:59 smithi059 ceph-mon[30304]: pgmap v135: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-05-08T05:33:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:33:59 smithi143 ceph-mon[34337]: pgmap v135: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-05-08T05:33:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:33:59 smithi096 ceph-mon[34253]: pgmap v135: 129 pgs: 129 active+clean; 36 MiB data, 773 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 15 KiB/s wr, 11 op/s 2024-05-08T05:34:01.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:01 smithi059 ceph-mon[30304]: pgmap v136: 129 pgs: 129 active+clean; 37 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 412 KiB/s wr, 76 op/s 2024-05-08T05:34:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:01 smithi143 ceph-mon[34337]: pgmap v136: 129 pgs: 129 active+clean; 37 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 412 KiB/s wr, 76 op/s 2024-05-08T05:34:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:01 smithi096 ceph-mon[34253]: pgmap v136: 129 pgs: 129 active+clean; 37 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 253 KiB/s rd, 412 KiB/s wr, 76 op/s 2024-05-08T05:34:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:03 smithi059 ceph-mon[30304]: pgmap v137: 129 pgs: 129 active+clean; 37 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 403 KiB/s wr, 71 op/s 2024-05-08T05:34:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:03 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:34:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:03 smithi143 ceph-mon[34337]: pgmap v137: 129 pgs: 129 active+clean; 37 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 403 KiB/s wr, 71 op/s 2024-05-08T05:34:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:03 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:34:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:03 smithi096 ceph-mon[34253]: pgmap v137: 129 pgs: 129 active+clean; 37 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 403 KiB/s wr, 71 op/s 2024-05-08T05:34:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:03 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 -- 172.21.15.59:0/3387737729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee0102990 msgr2=0x7efee0102e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 --2- 172.21.15.59:0/3387737729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee0102990 0x7efee0102e10 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7efed4009930 tx=0x7efed402a420 comp rx=0 tx=0).stop 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 -- 172.21.15.59:0/3387737729 shutdown_connections 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 --2- 172.21.15.59:0/3387737729 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0103350 0x7efee010f860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 --2- 172.21.15.59:0/3387737729 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee0102990 0x7efee0102e10 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 --2- 172.21.15.59:0/3387737729 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee01089a0 0x7efee0108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 -- 172.21.15.59:0/3387737729 >> 172.21.15.59:0/3387737729 conn(0x7efee00fe460 msgr2=0x7efee0100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 -- 172.21.15.59:0/3387737729 shutdown_connections 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 -- 172.21.15.59:0/3387737729 wait complete. 2024-05-08T05:34:03.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.990+0000 7efee694e640 1 Processor -- start 2024-05-08T05:34:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efee694e640 1 -- start start 2024-05-08T05:34:03.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efee694e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee0102990 0x7efee0074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efee694e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0103350 0x7efee00751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efee694e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee01089a0 0x7efee0075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efee694e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112c60 con 0x7efee0102990 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efee694e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112ae0 con 0x7efee0103350 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efee694e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efee0112de0 con 0x7efee01089a0 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efedffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee0102990 0x7efee0074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0103350 0x7efee00751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0103350 0x7efee00751f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:54044/0 (socket says 172.21.15.59:54044) 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efedf7fe640 1 -- 172.21.15.59:0/404741799 learned_addr learned my addr 172.21.15.59:0/404741799 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:03.997 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.991+0000 7efee4eeb640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee01089a0 0x7efee0075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedffff640 1 -- 172.21.15.59:0/404741799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee01089a0 msgr2=0x7efee0075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedffff640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee01089a0 0x7efee0075730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedffff640 1 -- 172.21.15.59:0/404741799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0103350 msgr2=0x7efee00751f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedffff640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0103350 0x7efee00751f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedffff640 1 -- 172.21.15.59:0/404741799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee01a7680 con 0x7efee0102990 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedf7fe640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0103350 0x7efee00751f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedffff640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee0102990 0x7efee0074cb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efed000e9f0 tx=0x7efed000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedd7fa640 1 -- 172.21.15.59:0/404741799 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed000ccc0 con 0x7efee0102990 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedd7fa640 1 -- 172.21.15.59:0/404741799 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed0005020 con 0x7efee0102990 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efedd7fa640 1 -- 172.21.15.59:0/404741799 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed000ce60 con 0x7efee0102990 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efee694e640 1 -- 172.21.15.59:0/404741799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee01a7970 con 0x7efee0102990 2024-05-08T05:34:03.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.992+0000 7efee694e640 1 -- 172.21.15.59:0/404741799 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee01a7e90 con 0x7efee0102990 2024-05-08T05:34:04.011 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.995+0000 7efedd7fa640 1 -- 172.21.15.59:0/404741799 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7efed00051d0 con 0x7efee0102990 2024-05-08T05:34:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.995+0000 7efee9660740 1 Processor -- start 2024-05-08T05:34:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.995+0000 7efee9660740 1 -- start start 2024-05-08T05:34:04.012 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee9660740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d842f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee9660740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d84830 0x5560e8d8ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee9660740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d914e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee9660740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5560e8ce95d0 con 0x5560e8d83f10 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee9660740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5560e8ca2890 con 0x5560e8d84830 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee9660740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5560e8ce5590 con 0x5560e8d8f0f0 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedd7fa640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efec80b2a90 0x7efec80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d84830 0x5560e8d8ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee4eeb640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d842f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee4eeb640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d842f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34002/0 (socket says 172.21.15.59:34002) 2024-05-08T05:34:04.013 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee4eeb640 1 -- 172.21.15.59:0/2119605583 learned_addr learned my addr 172.21.15.59:0/2119605583 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d84830 0x5560e8d8ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:54056/0 (socket says 172.21.15.59:54056) 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedf7fe640 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d914e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee4eeb640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efec80b2a90 0x7efec80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedd7fa640 1 -- 172.21.15.59:0/404741799 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efed00e2810 con 0x7efee0102990 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedf7fe640 1 -- 172.21.15.59:0/2119605583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d84830 msgr2=0x5560e8d8ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedf7fe640 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d84830 0x5560e8d8ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedf7fe640 1 -- 172.21.15.59:0/2119605583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 msgr2=0x5560e8d842f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedf7fe640 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d842f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedf7fe640 1 -- 172.21.15.59:0/2119605583 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e89fb210 con 0x5560e8d8f0f0 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee4eeb640 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedffff640 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d84830 0x5560e8d8ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efee4eeb640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efec80b2a90 0x7efec80b4f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efed8005c40 tx=0x7efed8009500 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.996+0000 7efedf7fe640 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d914e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efed4004770 tx=0x7efed4032ed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6074476fe22c5fe9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:04.014 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efec67fc640 1 -- 172.21.15.59:0/2119605583 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4035040 con 0x5560e8d8f0f0 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efec67fc640 1 -- 172.21.15.59:0/2119605583 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed40317a0 con 0x5560e8d8f0f0 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efec67fc640 1 -- 172.21.15.59:0/2119605583 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed40047a0 con 0x5560e8d8f0f0 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efee9660740 1 -- 172.21.15.59:0/2119605583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 msgr2=0x5560e8d914e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efee9660740 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d914e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7efed4004770 tx=0x7efed4032ed0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efee9660740 1 -- 172.21.15.59:0/2119605583 shutdown_connections 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efee9660740 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d914e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efee9660740 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d84830 0x5560e8d8ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efee9660740 1 --2- 172.21.15.59:0/2119605583 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d842f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efee9660740 1 -- 172.21.15.59:0/2119605583 >> 172.21.15.59:0/2119605583 conn(0x5560e8d80420 msgr2=0x5560e8d80800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.997+0000 7efee9660740 1 -- 172.21.15.59:0/2119605583 shutdown_connections 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 1 -- 172.21.15.59:0/2119605583 wait complete. 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 1 -- start start 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 1 Processor -- start 2024-05-08T05:34:04.015 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 0x5560e8d8bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8d8c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d8ca70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5560e8d91cc0 con 0x5560e8d84830 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5560e8ca6370 con 0x5560e8d83f10 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5560e8d91e40 con 0x5560e8d8f0f0 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.998+0000 7efee9660740 10 client.0 _ll_register_callbacks cb 0x5560e89faa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8d8c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8d8c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34010/0 (socket says 172.21.15.59:34010) 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedf7fe640 1 -- 172.21.15.59:0/386265711 learned_addr learned my addr 172.21.15.59:0/386265711 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efee4eeb640 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d8ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedffff640 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 0x5560e8d8bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.016 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedf7fe640 1 -- 172.21.15.59:0/386265711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 msgr2=0x5560e8d8ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedf7fe640 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d8ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedf7fe640 1 -- 172.21.15.59:0/386265711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 msgr2=0x5560e8d8bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedf7fe640 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 0x5560e8d8bff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedf7fe640 1 -- 172.21.15.59:0/386265711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e8ac7620 con 0x5560e8d84830 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efedf7fe640 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8d8c530 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efed4005d90 tx=0x7efed4005770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19797f00891c536d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efebf7fe640 1 -- 172.21.15.59:0/386265711 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed404e020 con 0x5560e8d84830 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efebf7fe640 1 -- 172.21.15.59:0/386265711 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efed4004250 con 0x5560e8d84830 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:03.999+0000 7efee9660740 1 -- 172.21.15.59:0/386265711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed40317a0 con 0x5560e8d84830 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efebf7fe640 1 -- 172.21.15.59:0/386265711 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efed4059050 con 0x5560e8d84830 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efee9660740 10 client.16050 fetch_fsmap learned FSMap version 34 2024-05-08T05:34:04.017 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efee9660740 1 -- 172.21.15.59:0/386265711 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560e8d9daf0 con 0x5560e8d84830 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efebf7fe640 1 -- 172.21.15.59:0/386265711 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed40317a0 con 0x5560e8d84830 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efebf7fe640 1 -- 172.21.15.59:0/386265711 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efed4046570 con 0x5560e8d84830 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efee9660740 10 client.16050 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efee9660740 10 client.16050 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efee9660740 20 client.16050 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efee9660740 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5560e8d9ee40 0x5560e8da1230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efee9660740 4 client.16050 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efee9660740 1 -- 172.21.15.59:0/386265711 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5560e8cf71e0 con 0x5560e8d9ee40 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.000+0000 7efedffff640 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5560e8d9ee40 0x5560e8da1230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.001+0000 7efedffff640 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5560e8d9ee40 0x5560e8da1230 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aa55891701fc432 server_cookie=5029f44fa426a64e in_seq=0 out_seq=0 2024-05-08T05:34:04.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.001+0000 7efebf7fe640 10 client.16050 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.001+0000 7efebf7fe640 1 -- 172.21.15.59:0/386265711 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5560e8cf71e0 con 0x5560e8d9ee40 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.001+0000 7efebf7fe640 10 client.16050 handle_command_reply: tid=0 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.001+0000 7efee9660740 1 client.16050 shutdown 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 -- 172.21.15.59:0/386265711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 msgr2=0x5560e8d8c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8d8c530 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7efed4005d90 tx=0x7efed4005770 comp rx=0 tx=0).stop 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 -- 172.21.15.59:0/386265711 shutdown_connections 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5560e8d9ee40 0x5560e8da1230 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d8ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8d8c530 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efebf7fe640 0 client.16050 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 --2- 172.21.15.59:0/386265711 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 0x5560e8d8bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 -- 172.21.15.59:0/386265711 >> 172.21.15.59:0/386265711 conn(0x5560e8d80420 msgr2=0x5560e8d861a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:04.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 -- 172.21.15.59:0/386265711 shutdown_connections 2024-05-08T05:34:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 1 -- 172.21.15.59:0/386265711 wait complete. 2024-05-08T05:34:04.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.002+0000 7efee9660740 20 client.16050 trim_cache size 0 max 16384 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.020+0000 7efee9660740 1 Processor -- start 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.020+0000 7efee9660740 1 -- start start 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.020+0000 7efee9660740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 0x5560e8da6860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.020+0000 7efee9660740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8da6da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.020+0000 7efee9660740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d89250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.020+0000 7efee9660740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b1e50 con 0x5560e8d84830 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.020+0000 7efee9660740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b39f0 con 0x5560e8d83f10 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.020+0000 7efee9660740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b2520 con 0x5560e8d8f0f0 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8da6da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedf7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8da6da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34026/0 (socket says 172.21.15.59:34026) 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedf7fe640 1 -- 172.21.15.59:0/2700338865 learned_addr learned my addr 172.21.15.59:0/2700338865 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efee4eeb640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d89250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedffff640 1 --2- 172.21.15.59:0/2700338865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 0x5560e8da6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedf7fe640 1 -- 172.21.15.59:0/2700338865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 msgr2=0x5560e8d89250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedf7fe640 1 --2- 172.21.15.59:0/2700338865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d89250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedf7fe640 1 -- 172.21.15.59:0/2700338865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 msgr2=0x5560e8da6860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedf7fe640 1 --2- 172.21.15.59:0/2700338865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 0x5560e8da6860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedf7fe640 1 -- 172.21.15.59:0/2700338865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5560e8d8a540 con 0x5560e8d84830 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedffff640 1 --2- 172.21.15.59:0/2700338865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 0x5560e8da6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efedf7fe640 1 --2- 172.21.15.59:0/2700338865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8da6da0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efed4046990 tx=0x7efed4005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8ac03c567aaf8cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efec5ffb640 1 -- 172.21.15.59:0/2700338865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed404e070 con 0x5560e8d84830 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efec5ffb640 1 -- 172.21.15.59:0/2700338865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed4005850 con 0x5560e8d84830 2024-05-08T05:34:04.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.021+0000 7efec5ffb640 1 -- 172.21.15.59:0/2700338865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed4035040 con 0x5560e8d84830 2024-05-08T05:34:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.022+0000 7efee9660740 1 -- 172.21.15.59:0/2700338865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 msgr2=0x5560e8da6da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:04.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.022+0000 7efee9660740 1 --2- 172.21.15.59:0/2700338865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8da6da0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7efed4046990 tx=0x7efed4005fd0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.022+0000 7efee9660740 1 -- 172.21.15.59:0/2700338865 shutdown_connections 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.022+0000 7efee9660740 1 --2- 172.21.15.59:0/2700338865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d8f0f0 0x5560e8d89250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.022+0000 7efee9660740 1 --2- 172.21.15.59:0/2700338865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d84830 0x5560e8da6da0 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.022+0000 7efee9660740 1 --2- 172.21.15.59:0/2700338865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d83f10 0x5560e8da6860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.022+0000 7efee9660740 1 -- 172.21.15.59:0/2700338865 >> 172.21.15.59:0/2700338865 conn(0x5560e8d80420 msgr2=0x5560e8d7f580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.022+0000 7efee9660740 1 -- 172.21.15.59:0/2700338865 shutdown_connections 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.022+0000 7efee9660740 1 -- 172.21.15.59:0/2700338865 wait complete. 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee9660740 1 -- start start 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee9660740 1 Processor -- start 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee9660740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d92d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee9660740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d84830 0x5560e8d93290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee9660740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d8f0f0 0x5560e8d937d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee9660740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b32c0 con 0x5560e8d83f10 2024-05-08T05:34:04.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee9660740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7efee01b2bf0 con 0x5560e8d8f0f0 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee9660740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7efee01afe10 con 0x5560e8d84830 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee9660740 10 client.0 _ll_register_callbacks cb 0x7efed4040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efedffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d92d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efedffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d92d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34042/0 (socket says 172.21.15.59:34042) 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efedffff640 1 -- 172.21.15.59:0/1219861152 learned_addr learned my addr 172.21.15.59:0/1219861152 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efedf7fe640 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d84830 0x5560e8d93290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.023+0000 7efee4eeb640 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d8f0f0 0x5560e8d937d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efedffff640 1 -- 172.21.15.59:0/1219861152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d84830 msgr2=0x5560e8d93290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efedffff640 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d84830 0x5560e8d93290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efedffff640 1 -- 172.21.15.59:0/1219861152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d8f0f0 msgr2=0x5560e8d937d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efedffff640 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d8f0f0 0x5560e8d937d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efedffff640 1 -- 172.21.15.59:0/1219861152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5560e8e08510 con 0x5560e8d83f10 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efedf7fe640 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d84830 0x5560e8d93290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:04.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efee4eeb640 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d8f0f0 0x5560e8d937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efedffff640 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d92d50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efed00a69f0 tx=0x7efed0073f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586297547ff15d2f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efebf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efebf7fe640 1 -- 172.21.15.59:0/1219861152 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed001c040 con 0x5560e8d83f10 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efebf7fe640 1 -- 172.21.15.59:0/1219861152 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7efed00a5d70 con 0x5560e8d83f10 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efee9660740 1 -- 172.21.15.59:0/1219861152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed4005850 con 0x5560e8d83f10 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efebf7fe640 1 -- 172.21.15.59:0/1219861152 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=34) ==== 24+0+0 (secure 0 0 0) 0x7efed00a9e80 con 0x5560e8d83f10 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efee9660740 10 client.16056 fetch_fsmap learned FSMap version 34 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.024+0000 7efee9660740 1 -- 172.21.15.59:0/1219861152 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=34,monmap=4+}) -- 0x5560e8e09880 con 0x5560e8d83f10 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efebf7fe640 1 -- 172.21.15.59:0/1219861152 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed4005850 con 0x5560e8d83f10 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efebf7fe640 1 -- 172.21.15.59:0/1219861152 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 34) ==== 8370+0+0 (secure 0 0 0) 0x7efed00be780 con 0x5560e8d83f10 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efee9660740 10 client.16056 fetch_fsmap finished waiting for FSMap version 34 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efee9660740 10 client.16056 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efee9660740 20 client.16056 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:04.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efee9660740 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5560e8e0a030 0x5560e8e0c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efee9660740 4 client.16056 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:34:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efee9660740 1 -- 172.21.15.59:0/1219861152 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5560e8cf1050 con 0x5560e8e0a030 2024-05-08T05:34:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efedf7fe640 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5560e8e0a030 0x5560e8e0c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efebf7fe640 10 client.16056 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:04.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.025+0000 7efedf7fe640 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5560e8e0a030 0x5560e8e0c450 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2d2c6802c843954 server_cookie=568bda8c15442fd7 in_seq=0 out_seq=0 2024-05-08T05:34:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efebf7fe640 1 -- 172.21.15.59:0/1219861152 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5560e8cf1050 con 0x5560e8e0a030 2024-05-08T05:34:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efebf7fe640 10 client.16056 handle_command_reply: tid=0 2024-05-08T05:34:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 client.16056 shutdown 2024-05-08T05:34:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 -- 172.21.15.59:0/1219861152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 msgr2=0x5560e8d92d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d92d50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efed00a69f0 tx=0x7efed0073f50 comp rx=0 tx=0).stop 2024-05-08T05:34:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 -- 172.21.15.59:0/1219861152 shutdown_connections 2024-05-08T05:34:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5560e8e0a030 0x5560e8e0c450 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5560e8d8f0f0 0x5560e8d937d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5560e8d84830 0x5560e8d93290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 --2- 172.21.15.59:0/1219861152 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5560e8d83f10 0x5560e8d92d50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 -- 172.21.15.59:0/1219861152 >> 172.21.15.59:0/1219861152 conn(0x5560e8d80420 msgr2=0x5560e8d8bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efebf7fe640 0 client.16056 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 -- 172.21.15.59:0/1219861152 shutdown_connections 2024-05-08T05:34:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.039+0000 7efee9660740 1 -- 172.21.15.59:0/1219861152 wait complete. 2024-05-08T05:34:04.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee9660740 20 client.16056 trim_cache size 0 max 16384 2024-05-08T05:34:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:34:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:34:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:34:04.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 -- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efec80b2a90 msgr2=0x7efec80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efec80b2a90 0x7efec80b4f50 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efed8005c40 tx=0x7efed8009500 comp rx=0 tx=0).stop 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 -- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee0102990 msgr2=0x7efee0074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee0102990 0x7efee0074cb0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7efed000e9f0 tx=0x7efed000edf0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 -- 172.21.15.59:0/404741799 shutdown_connections 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7efec80b2a90 0x7efec80b4f50 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7efee01089a0 0x7efee0075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7efee0103350 0x7efee00751f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 --2- 172.21.15.59:0/404741799 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7efee0102990 0x7efee0074cb0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.040+0000 7efee694e640 1 -- 172.21.15.59:0/404741799 >> 172.21.15.59:0/404741799 conn(0x7efee00fe460 msgr2=0x7efee00fed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.041+0000 7efee694e640 1 -- 172.21.15.59:0/404741799 shutdown_connections 2024-05-08T05:34:04.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:04.041+0000 7efee694e640 1 -- 172.21.15.59:0/404741799 wait complete. 2024-05-08T05:34:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:05 smithi059 ceph-mon[30304]: pgmap v138: 129 pgs: 129 active+clean; 37 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 403 KiB/s wr, 71 op/s 2024-05-08T05:34:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:05 smithi143 ceph-mon[34337]: pgmap v138: 129 pgs: 129 active+clean; 37 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 403 KiB/s wr, 71 op/s 2024-05-08T05:34:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:05 smithi096 ceph-mon[34253]: pgmap v138: 129 pgs: 129 active+clean; 37 MiB data, 787 MiB used, 1.0 TiB / 1.0 TiB avail; 248 KiB/s rd, 403 KiB/s wr, 71 op/s 2024-05-08T05:34:07.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:07 smithi059 ceph-mon[30304]: pgmap v139: 129 pgs: 129 active+clean; 36 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-05-08T05:34:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:07 smithi143 ceph-mon[34337]: pgmap v139: 129 pgs: 129 active+clean; 36 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-05-08T05:34:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:07 smithi096 ceph-mon[34253]: pgmap v139: 129 pgs: 129 active+clean; 36 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 266 op/s 2024-05-08T05:34:07.574 INFO:tasks.workunit.client.0.smithi059.stdout:30000 trunc from 0x7bb2f to 0xcdf83 2024-05-08T05:34:08.107 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xffcdd 2024-05-08T05:34:09.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:09 smithi059 ceph-mon[30304]: pgmap v140: 129 pgs: 129 active+clean; 36 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-05-08T05:34:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:09 smithi143 ceph-mon[34337]: pgmap v140: 129 pgs: 129 active+clean; 36 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-05-08T05:34:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:09 smithi096 ceph-mon[34253]: pgmap v140: 129 pgs: 129 active+clean; 36 MiB data, 878 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-05-08T05:34:10.645 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xffd97 2024-05-08T05:34:11.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:11 smithi059 ceph-mon[30304]: pgmap v141: 129 pgs: 129 active+clean; 39 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 416 op/s 2024-05-08T05:34:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:11 smithi143 ceph-mon[34337]: pgmap v141: 129 pgs: 129 active+clean; 39 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 416 op/s 2024-05-08T05:34:11.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:11 smithi096 ceph-mon[34253]: pgmap v141: 129 pgs: 129 active+clean; 39 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.7 MiB/s wr, 416 op/s 2024-05-08T05:34:13.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:13 smithi059 ceph-mon[30304]: pgmap v142: 129 pgs: 129 active+clean; 39 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2024-05-08T05:34:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:13 smithi143 ceph-mon[34337]: pgmap v142: 129 pgs: 129 active+clean; 39 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2024-05-08T05:34:13.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:13 smithi096 ceph-mon[34253]: pgmap v142: 129 pgs: 129 active+clean; 39 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2024-05-08T05:34:15.330 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:15 smithi096 ceph-mon[34253]: pgmap v143: 129 pgs: 129 active+clean; 39 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2024-05-08T05:34:15.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:15 smithi059 ceph-mon[30304]: pgmap v143: 129 pgs: 129 active+clean; 39 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2024-05-08T05:34:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:15 smithi143 ceph-mon[34337]: pgmap v143: 129 pgs: 129 active+clean; 39 MiB data, 913 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.3 MiB/s wr, 351 op/s 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 -- 172.21.15.59:0/1639573962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b0104560 msgr2=0x7f66b010ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/1639573962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b0104560 0x7f66b010ca40 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6698009900 tx=0x7f669802a510 comp rx=0 tx=0).stop 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 -- 172.21.15.59:0/1639573962 shutdown_connections 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/1639573962 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b010cf80 0x7f66b010f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/1639573962 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b0104560 0x7f66b010ca40 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/1639573962 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0103c40 0x7f66b0104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 -- 172.21.15.59:0/1639573962 >> 172.21.15.59:0/1639573962 conn(0x7f66b0078a30 msgr2=0x7f66b0078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 -- 172.21.15.59:0/1639573962 shutdown_connections 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 -- 172.21.15.59:0/1639573962 wait complete. 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 Processor -- start 2024-05-08T05:34:17.151 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.149+0000 7f66b5dc1640 1 -- start start 2024-05-08T05:34:17.153 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66b5dc1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b0103c40 0x7f66b0101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66b5dc1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0104560 0x7f66b0101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66b5dc1640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b010cf80 0x7f66b0102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66b5dc1640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0112c30 con 0x7f66b0103c40 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66b5dc1640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0112ab0 con 0x7f66b0104560 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66b5dc1640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f66b0112db0 con 0x7f66b010cf80 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0104560 0x7f66b0101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b0103c40 0x7f66b0101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0104560 0x7f66b0101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:52738/0 (socket says 172.21.15.59:52738) 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66affff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b010cf80 0x7f66b0102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66a6ffd640 1 -- 172.21.15.59:0/3190124181 learned_addr learned my addr 172.21.15.59:0/3190124181 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66affff640 1 -- 172.21.15.59:0/3190124181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0104560 msgr2=0x7f66b0101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66affff640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0104560 0x7f66b0101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66affff640 1 -- 172.21.15.59:0/3190124181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b0103c40 msgr2=0x7f66b0101890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:17.154 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66affff640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b0103c40 0x7f66b0101890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.150+0000 7f66affff640 1 -- 172.21.15.59:0/3190124181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66b01a7690 con 0x7f66b010cf80 2024-05-08T05:34:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.151+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0104560 0x7f66b0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.151+0000 7f66affff640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b010cf80 0x7f66b0102310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66a800d6b0 tx=0x7f66a800db70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.151+0000 7f66ad7fa640 1 -- 172.21.15.59:0/3190124181 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8017070 con 0x7f66b010cf80 2024-05-08T05:34:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.151+0000 7f66ad7fa640 1 -- 172.21.15.59:0/3190124181 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a8009cd0 con 0x7f66b010cf80 2024-05-08T05:34:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.151+0000 7f66b5dc1640 1 -- 172.21.15.59:0/3190124181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66b01a7980 con 0x7f66b010cf80 2024-05-08T05:34:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.151+0000 7f66ad7fa640 1 -- 172.21.15.59:0/3190124181 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a8003cc0 con 0x7f66b010cf80 2024-05-08T05:34:17.155 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.151+0000 7f66b5dc1640 1 -- 172.21.15.59:0/3190124181 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66b01a7e50 con 0x7f66b010cf80 2024-05-08T05:34:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.154+0000 7f66b8bba740 1 Processor -- start 2024-05-08T05:34:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66ad7fa640 1 -- 172.21.15.59:0/3190124181 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f66a8009820 con 0x7f66b010cf80 2024-05-08T05:34:17.156 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66b8bba740 1 -- start start 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac416f10 0x559aac4172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac421bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac4244e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66b8bba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559aac37c5d0 con 0x559aac417830 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66b8bba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559aac335890 con 0x559aac416f10 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66b8bba740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559aac378590 con 0x559aac4220f0 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac421bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac416f10 0x559aac4172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac421bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44732/0 (socket says 172.21.15.59:44732) 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66af7fe640 1 -- 172.21.15.59:0/1012140286 learned_addr learned my addr 172.21.15.59:0/1012140286 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:17.158 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.155+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/1012140286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac4244e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66af7fe640 1 -- 172.21.15.59:0/1012140286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 msgr2=0x559aac4244e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66af7fe640 1 --2- 172.21.15.59:0/1012140286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac4244e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66af7fe640 1 -- 172.21.15.59:0/1012140286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac416f10 msgr2=0x559aac4172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66af7fe640 1 --2- 172.21.15.59:0/1012140286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac416f10 0x559aac4172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66af7fe640 1 -- 172.21.15.59:0/1012140286 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aac08e210 con 0x559aac417830 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/1012140286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac4244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66ad7fa640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f669c0b2a10 0x7f669c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66af7fe640 1 --2- 172.21.15.59:0/1012140286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac421bb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f66a00114f0 tx=0x7f66a00119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c120d0ed9e8def8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f669c0b2a10 0x7f669c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66ad7fa640 1 -- 172.21.15.59:0/3190124181 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f66a80e2130 con 0x7f66b010cf80 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66a5ffb640 1 -- 172.21.15.59:0/1012140286 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a001e070 con 0x559aac417830 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66a5ffb640 1 -- 172.21.15.59:0/1012140286 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a0003c30 con 0x559aac417830 2024-05-08T05:34:17.159 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.156+0000 7f66a5ffb640 1 -- 172.21.15.59:0/1012140286 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a000fa00 con 0x559aac417830 2024-05-08T05:34:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66b8bba740 1 -- 172.21.15.59:0/1012140286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 msgr2=0x559aac421bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:17.164 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66b8bba740 1 --2- 172.21.15.59:0/1012140286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac421bb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f66a00114f0 tx=0x7f66a00119b0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f669c0b2a10 0x7f669c0b4ed0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6698031440 tx=0x7f6698005d50 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66b8bba740 1 -- 172.21.15.59:0/1012140286 shutdown_connections 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66b8bba740 1 --2- 172.21.15.59:0/1012140286 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac4244e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66b8bba740 1 --2- 172.21.15.59:0/1012140286 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac421bb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66b8bba740 1 --2- 172.21.15.59:0/1012140286 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac416f10 0x559aac4172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66b8bba740 1 -- 172.21.15.59:0/1012140286 >> 172.21.15.59:0/1012140286 conn(0x559aac413420 msgr2=0x559aac413800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66b8bba740 1 -- 172.21.15.59:0/1012140286 shutdown_connections 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.157+0000 7f66b8bba740 1 -- 172.21.15.59:0/1012140286 wait complete. 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.158+0000 7f66b8bba740 1 -- start start 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.158+0000 7f66b8bba740 1 Processor -- start 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.158+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac416f10 0x559aac41f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.158+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41faf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.165 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66b8bba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x559aac424d20 con 0x559aac416f10 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66b8bba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x559aac339370 con 0x559aac417830 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:52764/0 (socket says 172.21.15.59:52764) 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66a6ffd640 1 -- 172.21.15.59:0/759242549 learned_addr learned my addr 172.21.15.59:0/759242549 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66affff640 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41faf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66b8bba740 1 -- 172.21.15.59:0/759242549 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x559aac424ea0 con 0x559aac4220f0 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66b8bba740 10 client.0 _ll_register_callbacks cb 0x559aac08da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66af7fe640 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac416f10 0x559aac41f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66af7fe640 1 -- 172.21.15.59:0/759242549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 msgr2=0x559aac41faf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66af7fe640 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41faf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66af7fe640 1 -- 172.21.15.59:0/759242549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 msgr2=0x559aac41f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66af7fe640 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.166 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66af7fe640 1 -- 172.21.15.59:0/759242549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aac15a620 con 0x559aac416f10 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66affff640 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f66af7fe640 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac416f10 0x559aac41f070 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f66a00049d0 tx=0x7f66a0004e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83d745e74501bc8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.159+0000 7f668effd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f668effd640 1 -- 172.21.15.59:0/759242549 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a001e050 con 0x559aac416f10 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f668effd640 1 -- 172.21.15.59:0/759242549 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f66a0003a30 con 0x559aac416f10 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f66b8bba740 1 -- 172.21.15.59:0/759242549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a0003c30 con 0x559aac416f10 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f668effd640 1 -- 172.21.15.59:0/759242549 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f66a0032050 con 0x559aac416f10 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f66b8bba740 10 client.16068 fetch_fsmap learned FSMap version 35 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f66b8bba740 1 -- 172.21.15.59:0/759242549 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559aac430bd0 con 0x559aac416f10 2024-05-08T05:34:17.167 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f668effd640 1 -- 172.21.15.59:0/759242549 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a0003c30 con 0x559aac416f10 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f668effd640 1 -- 172.21.15.59:0/759242549 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f66a0005040 con 0x559aac416f10 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f66b8bba740 10 client.16068 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f66b8bba740 10 client.16068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f66b8bba740 20 client.16068 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f66b8bba740 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x559aac432030 0x559aac434420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.160+0000 7f66b8bba740 4 client.16068 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.161+0000 7f66b8bba740 1 -- 172.21.15.59:0/759242549 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559aac38a1e0 con 0x559aac432030 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.161+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x559aac432030 0x559aac434420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.161+0000 7f668effd640 10 client.16068 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.161+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x559aac432030 0x559aac434420 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=45a2f7be85e36698 server_cookie=104abf1975659c3c in_seq=0 out_seq=0 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f668effd640 1 -- 172.21.15.59:0/759242549 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x559aac38a1e0 con 0x559aac432030 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f668effd640 10 client.16068 handle_command_reply: tid=0 2024-05-08T05:34:17.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 client.16068 shutdown 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 -- 172.21.15.59:0/759242549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac416f10 msgr2=0x559aac41f070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac416f10 0x559aac41f070 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f66a00049d0 tx=0x7f66a0004e90 comp rx=0 tx=0).stop 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 -- 172.21.15.59:0/759242549 shutdown_connections 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x559aac432030 0x559aac434420 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41faf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 --2- 172.21.15.59:0/759242549 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac416f10 0x559aac41f070 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 -- 172.21.15.59:0/759242549 >> 172.21.15.59:0/759242549 conn(0x559aac413420 msgr2=0x559aac4191a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f668effd640 0 client.16068 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 -- 172.21.15.59:0/759242549 shutdown_connections 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.162+0000 7f66b8bba740 1 -- 172.21.15.59:0/759242549 wait complete. 2024-05-08T05:34:17.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.163+0000 7f66b8bba740 20 client.16068 trim_cache size 0 max 16384 2024-05-08T05:34:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.180+0000 7f66b8bba740 1 Processor -- start 2024-05-08T05:34:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.180+0000 7f66b8bba740 1 -- start start 2024-05-08T05:34:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.180+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac41f940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.180+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.180+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac42dd70 0x559aac439b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.180+0000 7f66b8bba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01b1e70 con 0x559aac417830 2024-05-08T05:34:17.183 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.180+0000 7f66b8bba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01b39e0 con 0x559aac42dd70 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.180+0000 7f66b8bba740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01b2540 con 0x559aac4220f0 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac41f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac41f940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44754/0 (socket says 172.21.15.59:44754) 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66af7fe640 1 -- 172.21.15.59:0/3592286068 learned_addr learned my addr 172.21.15.59:0/3592286068 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66affff640 1 --2- 172.21.15.59:0/3592286068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac42dd70 0x559aac439b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66af7fe640 1 -- 172.21.15.59:0/3592286068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 msgr2=0x559aac41a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66af7fe640 1 --2- 172.21.15.59:0/3592286068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66af7fe640 1 -- 172.21.15.59:0/3592286068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac42dd70 msgr2=0x559aac439b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66af7fe640 1 --2- 172.21.15.59:0/3592286068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac42dd70 0x559aac439b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66af7fe640 1 -- 172.21.15.59:0/3592286068 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559aac417660 con 0x559aac417830 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/3592286068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66af7fe640 1 --2- 172.21.15.59:0/3592286068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac41f940 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x559aac420180 tx=0x7f66a0005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381576edb210e7a0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:17.184 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.181+0000 7f66a57fa640 1 -- 172.21.15.59:0/3592286068 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a001e040 con 0x559aac417830 2024-05-08T05:34:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.182+0000 7f66a57fa640 1 -- 172.21.15.59:0/3592286068 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a001f040 con 0x559aac417830 2024-05-08T05:34:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.182+0000 7f66a57fa640 1 -- 172.21.15.59:0/3592286068 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66a0003a30 con 0x559aac417830 2024-05-08T05:34:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.182+0000 7f66b8bba740 1 -- 172.21.15.59:0/3592286068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 msgr2=0x559aac41f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.182+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3592286068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac41f940 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x559aac420180 tx=0x7f66a0005cf0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.182+0000 7f66b8bba740 1 -- 172.21.15.59:0/3592286068 shutdown_connections 2024-05-08T05:34:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.182+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3592286068 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac42dd70 0x559aac439b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.182+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3592286068 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac41a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.182+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3592286068 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac417830 0x559aac41f940 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.182+0000 7f66b8bba740 1 -- 172.21.15.59:0/3592286068 >> 172.21.15.59:0/3592286068 conn(0x559aac413420 msgr2=0x559aac40e3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 -- 172.21.15.59:0/3592286068 shutdown_connections 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 -- 172.21.15.59:0/3592286068 wait complete. 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 -- start start 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 Processor -- start 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac42a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac42dd70 0x559aac42a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01b32e0 con 0x559aac42dd70 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01b2c10 con 0x559aac417830 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f66b01afe30 con 0x559aac4220f0 2024-05-08T05:34:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.183+0000 7f66b8bba740 10 client.0 _ll_register_callbacks cb 0x7f66a001c070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66affff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac42dd70 0x559aac42a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac42a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66affff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac42dd70 0x559aac42a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44756/0 (socket says 172.21.15.59:44756) 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66af7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41cca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:52778/0 (socket says 172.21.15.59:52778) 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66af7fe640 1 -- 172.21.15.59:0/3780870829 learned_addr learned my addr 172.21.15.59:0/3780870829 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66a6ffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac42a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:42840/0 (socket says 172.21.15.59:42840) 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66a6ffd640 1 -- 172.21.15.59:0/3780870829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 msgr2=0x559aac41cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66a6ffd640 1 -- 172.21.15.59:0/3780870829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac42dd70 msgr2=0x559aac42a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac42dd70 0x559aac42a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66a6ffd640 1 -- 172.21.15.59:0/3780870829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559aac41b410 con 0x559aac4220f0 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66af7fe640 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66affff640 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac42dd70 0x559aac42a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66a6ffd640 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac42a0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x559aac434ad0 tx=0x7f669802a510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2b2a7a6eb6ee9f04 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f668effd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f668effd640 1 -- 172.21.15.59:0/3780870829 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6698046070 con 0x559aac4220f0 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f668effd640 1 -- 172.21.15.59:0/3780870829 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f66980553a0 con 0x559aac4220f0 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.184+0000 7f66b8bba740 1 -- 172.21.15.59:0/3780870829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66a001f040 con 0x559aac4220f0 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f668effd640 1 -- 172.21.15.59:0/3780870829 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=35) ==== 24+0+0 (secure 0 0 0) 0x7f6698066050 con 0x559aac4220f0 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f66b8bba740 10 client.25630 fetch_fsmap learned FSMap version 35 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f66b8bba740 1 -- 172.21.15.59:0/3780870829 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=35,monmap=4+}) -- 0x559aac49bb50 con 0x559aac4220f0 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f668effd640 1 -- 172.21.15.59:0/3780870829 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66a001f040 con 0x559aac4220f0 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f668effd640 1 -- 172.21.15.59:0/3780870829 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 35) ==== 8366+0+0 (secure 0 0 0) 0x7f669803d650 con 0x559aac4220f0 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f66b8bba740 10 client.25630 fetch_fsmap finished waiting for FSMap version 35 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f66b8bba740 10 client.25630 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f66b8bba740 20 client.25630 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x559aac49c2b0 0x559aac49e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f66b8bba740 4 client.25630 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:34:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f66b8bba740 1 -- 172.21.15.59:0/3780870829 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559aac384050 con 0x559aac49c2b0 2024-05-08T05:34:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.185+0000 7f66af7fe640 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x559aac49c2b0 0x559aac49e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.186+0000 7f668effd640 10 client.25630 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:17.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.186+0000 7f66af7fe640 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x559aac49c2b0 0x559aac49e6d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d1ee23938b0e2af6 server_cookie=efa6e533ad8b6358 in_seq=0 out_seq=0 2024-05-08T05:34:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.201+0000 7f668effd640 1 -- 172.21.15.59:0/3780870829 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559aac384050 con 0x559aac49c2b0 2024-05-08T05:34:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.201+0000 7f668effd640 10 client.25630 handle_command_reply: tid=0 2024-05-08T05:34:17.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.201+0000 7f66b8bba740 1 client.25630 shutdown 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.201+0000 7f66b8bba740 1 -- 172.21.15.59:0/3780870829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 msgr2=0x559aac42a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.201+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac42a0f0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x559aac434ad0 tx=0x7f669802a510 comp rx=0 tx=0).stop 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b8bba740 1 -- 172.21.15.59:0/3780870829 shutdown_connections 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x559aac49c2b0 0x559aac49e6d0 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f668effd640 0 client.25630 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x559aac42dd70 0x559aac42a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x559aac4220f0 0x559aac42a0f0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b8bba740 1 --2- 172.21.15.59:0/3780870829 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x559aac417830 0x559aac41cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b8bba740 1 -- 172.21.15.59:0/3780870829 >> 172.21.15.59:0/3780870829 conn(0x559aac413420 msgr2=0x559aac41a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b8bba740 1 -- 172.21.15.59:0/3780870829 shutdown_connections 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b8bba740 1 -- 172.21.15.59:0/3780870829 wait complete. 2024-05-08T05:34:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b8bba740 20 client.25630 trim_cache size 0 max 16384 2024-05-08T05:34:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:34:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:34:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:34:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:34:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b5dc1640 1 -- 172.21.15.59:0/3190124181 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f669c0b2a10 msgr2=0x7f669c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f669c0b2a10 0x7f669c0b4ed0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f6698031440 tx=0x7f6698005d50 comp rx=0 tx=0).stop 2024-05-08T05:34:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b5dc1640 1 -- 172.21.15.59:0/3190124181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b010cf80 msgr2=0x7f66b0102310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.202+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b010cf80 0x7f66b0102310 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f66a800d6b0 tx=0x7f66a800db70 comp rx=0 tx=0).stop 2024-05-08T05:34:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.203+0000 7f66b5dc1640 1 -- 172.21.15.59:0/3190124181 shutdown_connections 2024-05-08T05:34:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.203+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f669c0b2a10 0x7f669c0b4ed0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.203+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f66b010cf80 0x7f66b0102310 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.203+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f66b0104560 0x7f66b0101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.203+0000 7f66b5dc1640 1 --2- 172.21.15.59:0/3190124181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f66b0103c40 0x7f66b0101890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.203+0000 7f66b5dc1640 1 -- 172.21.15.59:0/3190124181 >> 172.21.15.59:0/3190124181 conn(0x7f66b0078a30 msgr2=0x7f66b00fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.203+0000 7f66b5dc1640 1 -- 172.21.15.59:0/3190124181 shutdown_connections 2024-05-08T05:34:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:17.203+0000 7f66b5dc1640 1 -- 172.21.15.59:0/3190124181 wait complete. 2024-05-08T05:34:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:17 smithi059 ceph-mon[30304]: pgmap v144: 129 pgs: 129 active+clean; 42 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 503 op/s 2024-05-08T05:34:17.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:17 smithi143 ceph-mon[34337]: pgmap v144: 129 pgs: 129 active+clean; 42 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 503 op/s 2024-05-08T05:34:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:17 smithi096 ceph-mon[34253]: pgmap v144: 129 pgs: 129 active+clean; 42 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 3.3 MiB/s wr, 503 op/s 2024-05-08T05:34:18.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:18 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:34:18.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:18 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:34:18.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:18 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:34:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:19 smithi059 ceph-mon[30304]: pgmap v145: 129 pgs: 129 active+clean; 42 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 308 op/s 2024-05-08T05:34:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:19 smithi143 ceph-mon[34337]: pgmap v145: 129 pgs: 129 active+clean; 42 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 308 op/s 2024-05-08T05:34:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:19 smithi096 ceph-mon[34253]: pgmap v145: 129 pgs: 129 active+clean; 42 MiB data, 920 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 2.1 MiB/s wr, 308 op/s 2024-05-08T05:34:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:21 smithi059 ceph-mon[30304]: pgmap v146: 129 pgs: 129 active+clean; 40 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 452 op/s 2024-05-08T05:34:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:21 smithi143 ceph-mon[34337]: pgmap v146: 129 pgs: 129 active+clean; 40 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 452 op/s 2024-05-08T05:34:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:21 smithi096 ceph-mon[34253]: pgmap v146: 129 pgs: 129 active+clean; 40 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 MiB/s rd, 3.0 MiB/s wr, 452 op/s 2024-05-08T05:34:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:23 smithi059 ceph-mon[30304]: pgmap v147: 129 pgs: 129 active+clean; 40 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-05-08T05:34:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:23 smithi143 ceph-mon[34337]: pgmap v147: 129 pgs: 129 active+clean; 40 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-05-08T05:34:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:23 smithi096 ceph-mon[34253]: pgmap v147: 129 pgs: 129 active+clean; 40 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-05-08T05:34:23.653 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0xffdb8 2024-05-08T05:34:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:25 smithi059 ceph-mon[30304]: pgmap v148: 129 pgs: 129 active+clean; 40 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-05-08T05:34:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:25 smithi143 ceph-mon[34337]: pgmap v148: 129 pgs: 129 active+clean; 40 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-05-08T05:34:25.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:25 smithi096 ceph-mon[34253]: pgmap v148: 129 pgs: 129 active+clean; 40 MiB data, 925 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 MiB/s rd, 1.9 MiB/s wr, 296 op/s 2024-05-08T05:34:27.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:27 smithi059 ceph-mon[30304]: pgmap v149: 129 pgs: 129 active+clean; 41 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 434 op/s 2024-05-08T05:34:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:27 smithi143 ceph-mon[34337]: pgmap v149: 129 pgs: 129 active+clean; 41 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 434 op/s 2024-05-08T05:34:27.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:27 smithi096 ceph-mon[34253]: pgmap v149: 129 pgs: 129 active+clean; 41 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 2.3 MiB/s rd, 2.8 MiB/s wr, 434 op/s 2024-05-08T05:34:28.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:28 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:34:28.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:28 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:34:28.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:28 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:34:29.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:29 smithi143 ceph-mon[34337]: pgmap v150: 129 pgs: 129 active+clean; 41 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 282 op/s 2024-05-08T05:34:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:29 smithi096 ceph-mon[34253]: pgmap v150: 129 pgs: 129 active+clean; 41 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 282 op/s 2024-05-08T05:34:29.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:29 smithi059 ceph-mon[30304]: pgmap v150: 129 pgs: 129 active+clean; 41 MiB data, 969 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 1.8 MiB/s wr, 282 op/s 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.304+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/1332392135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28103350 msgr2=0x7fdb2810f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.304+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/1332392135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28103350 0x7fdb2810f860 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fdb20009cd0 tx=0x7fdb2002a8e0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.304+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/1332392135 shutdown_connections 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.304+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/1332392135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28103350 0x7fdb2810f860 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.304+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/1332392135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb28102990 0x7fdb28102e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.304+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/1332392135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb281089a0 0x7fdb28108d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.304+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/1332392135 >> 172.21.15.59:0/1332392135 conn(0x7fdb28079e40 msgr2=0x7fdb2807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.304+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/1332392135 shutdown_connections 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.304+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/1332392135 wait complete. 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2ed2b640 1 Processor -- start 2024-05-08T05:34:30.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2ed2b640 1 -- start start 2024-05-08T05:34:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2ed2b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28102990 0x7fdb281990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2ed2b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28103350 0x7fdb281995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2ed2b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb281089a0 0x7fdb2819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2ed2b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28112c60 con 0x7fdb28103350 2024-05-08T05:34:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2ed2b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28112ae0 con 0x7fdb281089a0 2024-05-08T05:34:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2ed2b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdb28112de0 con 0x7fdb28102990 2024-05-08T05:34:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2cac7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28102990 0x7fdb281990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.307 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28103350 0x7fdb281995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2cac7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28102990 0x7fdb281990a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:47580/0 (socket says 172.21.15.59:47580) 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2cac7640 1 -- 172.21.15.59:0/2459179450 learned_addr learned my addr 172.21.15.59:0/2459179450 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb281089a0 0x7fdb2819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2cac7640 1 -- 172.21.15.59:0/2459179450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb281089a0 msgr2=0x7fdb2819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2cac7640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb281089a0 0x7fdb2819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2cac7640 1 -- 172.21.15.59:0/2459179450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28103350 msgr2=0x7fdb281995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2cac7640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28103350 0x7fdb281995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.305+0000 7fdb2cac7640 1 -- 172.21.15.59:0/2459179450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb2819e090 con 0x7fdb28102990 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.306+0000 7fdb2cac7640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28102990 0x7fdb281990a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e9f0 tx=0x7fdb1c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.306+0000 7fdb25ffb640 1 -- 172.21.15.59:0/2459179450 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c00cd80 con 0x7fdb28102990 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.306+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/2459179450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb2819e380 con 0x7fdb28102990 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.306+0000 7fdb25ffb640 1 -- 172.21.15.59:0/2459179450 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb1c0108e0 con 0x7fdb28102990 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.306+0000 7fdb25ffb640 1 -- 172.21.15.59:0/2459179450 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1c018980 con 0x7fdb28102990 2024-05-08T05:34:30.308 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.306+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/2459179450 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb28074be0 con 0x7fdb28102990 2024-05-08T05:34:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb31a47740 1 Processor -- start 2024-05-08T05:34:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb31a47740 1 -- start start 2024-05-08T05:34:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fb92f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc3bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc64e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb31a47740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1f1e5d0 con 0x55e6c1fc40f0 2024-05-08T05:34:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb31a47740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1ed7890 con 0x55e6c1fb9830 2024-05-08T05:34:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb31a47740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1f1a590 con 0x55e6c1fb8f10 2024-05-08T05:34:30.310 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc64e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb2d2c8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fb92f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb2d2c8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fb92f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:47588/0 (socket says 172.21.15.59:47588) 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc64e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46588/0 (socket says 172.21.15.59:46588) 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/1241903283 learned_addr learned my addr 172.21.15.59:0/1241903283 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.309+0000 7fdb25ffb640 1 -- 172.21.15.59:0/2459179450 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdb1c010430 con 0x7fdb28102990 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/1241903283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb9830 msgr2=0x55e6c1fc3bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/1241903283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc3bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/1241903283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 msgr2=0x55e6c1fc64e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/1241903283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc64e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.311 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/1241903283 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c1c30210 con 0x55e6c1fb8f10 2024-05-08T05:34:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/1241903283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fb92f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb20035000 tx=0x7fdb20009a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=25ce795515c63ba6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb25ffb640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdb180b2a10 0x7fdb180b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb12ffd640 1 -- 172.21.15.59:0/1241903283 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2004d070 con 0x55e6c1fb8f10 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb12ffd640 1 -- 172.21.15.59:0/1241903283 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb20036040 con 0x55e6c1fb8f10 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.310+0000 7fdb12ffd640 1 -- 172.21.15.59:0/1241903283 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2003c980 con 0x55e6c1fb8f10 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb25ffb640 1 -- 172.21.15.59:0/2459179450 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fdb1c0e2c20 con 0x7fdb28102990 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb2cac7640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdb180b2a10 0x7fdb180b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb31a47740 1 -- 172.21.15.59:0/1241903283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb8f10 msgr2=0x55e6c1fb92f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb2cac7640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdb180b2a10 0x7fdb180b4ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55e6c1fba4b0 tx=0x7fdb1c018420 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1241903283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fb92f0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdb20035000 tx=0x7fdb20009a20 comp rx=0 tx=0).stop 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb31a47740 1 -- 172.21.15.59:0/1241903283 shutdown_connections 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1241903283 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc64e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1241903283 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc3bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1241903283 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fb92f0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.311+0000 7fdb31a47740 1 -- 172.21.15.59:0/1241903283 >> 172.21.15.59:0/1241903283 conn(0x55e6c1fb5420 msgr2=0x55e6c1fb5800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.312+0000 7fdb31a47740 1 -- 172.21.15.59:0/1241903283 shutdown_connections 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.312+0000 7fdb31a47740 1 -- 172.21.15.59:0/1241903283 wait complete. 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.312+0000 7fdb31a47740 1 -- start start 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.312+0000 7fdb31a47740 1 Processor -- start 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.312+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fc1010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc1550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb31a47740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1fc6cc0 con 0x55e6c1fb8f10 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb31a47740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1edb370 con 0x55e6c1fc40f0 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb31a47740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55e6c1fc6e40 con 0x55e6c1fb9830 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb2d2c8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fc1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb2d2c8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fc1010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46600/0 (socket says 172.21.15.59:46600) 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/1664458503 learned_addr learned my addr 172.21.15.59:0/1664458503 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb27fff640 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc1550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb31a47740 10 client.0 _ll_register_callbacks cb 0x55e6c1c2fa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb27fff640 1 -- 172.21.15.59:0/1664458503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fc40f0 msgr2=0x55e6c1fc1a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb27fff640 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc1a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb27fff640 1 -- 172.21.15.59:0/1664458503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 msgr2=0x55e6c1fc1010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb27fff640 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fc1010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb27fff640 1 -- 172.21.15.59:0/1664458503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c1cfc620 con 0x55e6c1fb9830 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fc1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb27fff640 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc1550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55e6c1fb65b0 tx=0x7fdb14012b60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e01ec2fc86af4a84 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.313+0000 7fdb07fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb07fff640 1 -- 172.21.15.59:0/1664458503 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1400f890 con 0x55e6c1fb9830 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb07fff640 1 -- 172.21.15.59:0/1664458503 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fdb1400fa30 con 0x55e6c1fb9830 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb31a47740 1 -- 172.21.15.59:0/1664458503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb20036040 con 0x55e6c1fb9830 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb07fff640 1 -- 172.21.15.59:0/1664458503 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdb1400fc30 con 0x55e6c1fb9830 2024-05-08T05:34:30.318 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb31a47740 10 client.25642 fetch_fsmap learned FSMap version 36 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb31a47740 1 -- 172.21.15.59:0/1664458503 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e6c1fd2bb0 con 0x55e6c1fb9830 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb07fff640 1 -- 172.21.15.59:0/1664458503 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb20036040 con 0x55e6c1fb9830 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb07fff640 1 -- 172.21.15.59:0/1664458503 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fdb1401c420 con 0x55e6c1fb9830 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb31a47740 10 client.25642 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb31a47740 10 client.25642 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb31a47740 20 client.25642 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e6c1fd40c0 0x55e6c1fd6510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb31a47740 4 client.25642 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.314+0000 7fdb31a47740 1 -- 172.21.15.59:0/1664458503 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e6c1f2c1e0 con 0x55e6c1fd40c0 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.315+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e6c1fd40c0 0x55e6c1fd6510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.315+0000 7fdb07fff640 10 client.25642 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:30.319 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.315+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e6c1fd40c0 0x55e6c1fd6510 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=37a118f68458171d server_cookie=4432e52ec1fe577e in_seq=0 out_seq=0 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb07fff640 1 -- 172.21.15.59:0/1664458503 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55e6c1f2c1e0 con 0x55e6c1fd40c0 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb07fff640 10 client.25642 handle_command_reply: tid=0 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 client.25642 shutdown 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 -- 172.21.15.59:0/1664458503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 msgr2=0x55e6c1fc1550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc1550 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x55e6c1fb65b0 tx=0x7fdb14012b60 comp rx=0 tx=0).stop 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 -- 172.21.15.59:0/1664458503 shutdown_connections 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e6c1fd40c0 0x55e6c1fd6510 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc1a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc1550 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb07fff640 0 client.25642 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 --2- 172.21.15.59:0/1664458503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fc1010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 -- 172.21.15.59:0/1664458503 >> 172.21.15.59:0/1664458503 conn(0x55e6c1fb5420 msgr2=0x55e6c1fbb150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 -- 172.21.15.59:0/1664458503 shutdown_connections 2024-05-08T05:34:30.321 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 1 -- 172.21.15.59:0/1664458503 wait complete. 2024-05-08T05:34:30.322 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.316+0000 7fdb31a47740 20 client.25642 trim_cache size 0 max 16384 2024-05-08T05:34:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.334+0000 7fdb31a47740 1 Processor -- start 2024-05-08T05:34:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb31a47740 1 -- start start 2024-05-08T05:34:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fdbe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb9830 0x55e6c1fdc360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.336 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fbe270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb31a47740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281ada20 con 0x55e6c1fb8f10 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb31a47740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281af560 con 0x55e6c1fb9830 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb31a47740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281ae0f0 con 0x55e6c1fc40f0 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb2d2c8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fdbe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb2d2c8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fdbe20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46604/0 (socket says 172.21.15.59:46604) 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/4092987333 learned_addr learned my addr 172.21.15.59:0/4092987333 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/4092987333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fbe270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb27fff640 1 --2- 172.21.15.59:0/4092987333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb9830 0x55e6c1fdc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/4092987333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fc40f0 msgr2=0x55e6c1fbe270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/4092987333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fbe270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/4092987333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb9830 msgr2=0x55e6c1fdc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/4092987333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb9830 0x55e6c1fdc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.335+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/4092987333 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c1fd6b90 con 0x55e6c1fb8f10 2024-05-08T05:34:30.337 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.336+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/4092987333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fdbe20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb20005ef0 tx=0x7fdb2002a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be0c062516bcb49 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.336+0000 7fdb127fc640 1 -- 172.21.15.59:0/4092987333 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb2004d040 con 0x55e6c1fb8f10 2024-05-08T05:34:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.336+0000 7fdb127fc640 1 -- 172.21.15.59:0/4092987333 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2005f070 con 0x55e6c1fb8f10 2024-05-08T05:34:30.338 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.336+0000 7fdb127fc640 1 -- 172.21.15.59:0/4092987333 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb20030bc0 con 0x55e6c1fb8f10 2024-05-08T05:34:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 -- 172.21.15.59:0/4092987333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 msgr2=0x55e6c1fdbe20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 --2- 172.21.15.59:0/4092987333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fdbe20 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fdb20005ef0 tx=0x7fdb2002a8e0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 -- 172.21.15.59:0/4092987333 shutdown_connections 2024-05-08T05:34:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 --2- 172.21.15.59:0/4092987333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fbe270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 --2- 172.21.15.59:0/4092987333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb9830 0x55e6c1fdc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 --2- 172.21.15.59:0/4092987333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fdbe20 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 -- 172.21.15.59:0/4092987333 >> 172.21.15.59:0/4092987333 conn(0x55e6c1fb5420 msgr2=0x55e6c1fb4580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:30.341 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 -- 172.21.15.59:0/4092987333 shutdown_connections 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 -- 172.21.15.59:0/4092987333 wait complete. 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 -- start start 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.337+0000 7fdb31a47740 1 Processor -- start 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fcc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb31a47740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb31a47740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281aee90 con 0x55e6c1fc40f0 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb31a47740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281ae7c0 con 0x55e6c1fb8f10 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb31a47740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdb281ab9e0 con 0x55e6c1fb9830 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb31a47740 10 client.0 _ll_register_callbacks cb 0x7fdb14009850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:30.342 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb2d2c8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb2d2c8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc8ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46610/0 (socket says 172.21.15.59:46610) 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb2d2c8640 1 -- 172.21.15.59:0/2802407561 learned_addr learned my addr 172.21.15.59:0/2802407561 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb27fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:47612/0 (socket says 172.21.15.59:47612) 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fcc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb27fff640 1 -- 172.21.15.59:0/2802407561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb8f10 msgr2=0x55e6c1fcc610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb27fff640 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fcc610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb27fff640 1 -- 172.21.15.59:0/2802407561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 msgr2=0x55e6c1fc8ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb27fff640 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc8ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.338+0000 7fdb27fff640 1 -- 172.21.15.59:0/2802407561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c1fba2e0 con 0x55e6c1fb9830 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb27fff640 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc8990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55e6c1fc6520 tx=0x7fdb1401cc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6850418738bcb944 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb07fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb07fff640 1 -- 172.21.15.59:0/2802407561 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb1401c630 con 0x55e6c1fb9830 2024-05-08T05:34:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb07fff640 1 -- 172.21.15.59:0/2802407561 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fdb1400f790 con 0x55e6c1fb9830 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb31a47740 1 -- 172.21.15.59:0/2802407561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb2005f070 con 0x55e6c1fb9830 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb07fff640 1 -- 172.21.15.59:0/2802407561 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fdb1400f990 con 0x55e6c1fb9830 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb31a47740 10 client.25648 fetch_fsmap learned FSMap version 36 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb31a47740 1 -- 172.21.15.59:0/2802407561 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55e6c203f850 con 0x55e6c1fb9830 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb07fff640 1 -- 172.21.15.59:0/2802407561 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb2005f070 con 0x55e6c1fb9830 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb07fff640 1 -- 172.21.15.59:0/2802407561 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fdb14003920 con 0x55e6c1fb9830 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb31a47740 10 client.25648 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb31a47740 10 client.25648 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb31a47740 20 client.25648 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.339+0000 7fdb31a47740 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e6c2040050 0x55e6c2042470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.340+0000 7fdb31a47740 4 client.25648 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.340+0000 7fdb31a47740 1 -- 172.21.15.59:0/2802407561 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e6c1f26050 con 0x55e6c2040050 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.340+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e6c2040050 0x55e6c2042470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.340+0000 7fdb2d2c8640 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e6c2040050 0x55e6c2042470 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2a1183ce1f8331ff server_cookie=ba5464d78c02391c in_seq=0 out_seq=0 2024-05-08T05:34:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.340+0000 7fdb07fff640 10 client.25648 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.354+0000 7fdb07fff640 1 -- 172.21.15.59:0/2802407561 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e6c1f26050 con 0x55e6c2040050 2024-05-08T05:34:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.354+0000 7fdb07fff640 10 client.25648 handle_command_reply: tid=0 2024-05-08T05:34:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.354+0000 7fdb31a47740 1 client.25648 shutdown 2024-05-08T05:34:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 -- 172.21.15.59:0/2802407561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 msgr2=0x55e6c1fc8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:30.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc8990 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x55e6c1fc6520 tx=0x7fdb1401cc50 comp rx=0 tx=0).stop 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 -- 172.21.15.59:0/2802407561 shutdown_connections 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55e6c2040050 0x55e6c2042470 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55e6c1fc40f0 0x55e6c1fc8ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55e6c1fb9830 0x55e6c1fc8990 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb07fff640 0 client.25648 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 --2- 172.21.15.59:0/2802407561 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55e6c1fb8f10 0x55e6c1fcc610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 -- 172.21.15.59:0/2802407561 >> 172.21.15.59:0/2802407561 conn(0x55e6c1fb5420 msgr2=0x55e6c1fba890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 -- 172.21.15.59:0/2802407561 shutdown_connections 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 1 -- 172.21.15.59:0/2802407561 wait complete. 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.355+0000 7fdb31a47740 20 client.25648 trim_cache size 0 max 16384 2024-05-08T05:34:30.356 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:34:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:34:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:34:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:34:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/2459179450 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdb180b2a10 msgr2=0x7fdb180b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdb180b2a10 0x7fdb180b4ed0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55e6c1fba4b0 tx=0x7fdb1c018420 comp rx=0 tx=0).stop 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/2459179450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28102990 msgr2=0x7fdb281990a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28102990 0x7fdb281990a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fdb1c00e9f0 tx=0x7fdb1c00eeb0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/2459179450 shutdown_connections 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdb180b2a10 0x7fdb180b4ed0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdb281089a0 0x7fdb2819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdb28103350 0x7fdb281995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 --2- 172.21.15.59:0/2459179450 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdb28102990 0x7fdb281990a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/2459179450 >> 172.21.15.59:0/2459179450 conn(0x7fdb28079e40 msgr2=0x7fdb281001c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/2459179450 shutdown_connections 2024-05-08T05:34:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:30.356+0000 7fdb2ed2b640 1 -- 172.21.15.59:0/2459179450 wait complete. 2024-05-08T05:34:31.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:31 smithi143 ceph-mon[34337]: pgmap v151: 129 pgs: 129 active+clean; 43 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 414 op/s 2024-05-08T05:34:31.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:31 smithi096 ceph-mon[34253]: pgmap v151: 129 pgs: 129 active+clean; 43 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 414 op/s 2024-05-08T05:34:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:31 smithi059 ceph-mon[30304]: pgmap v151: 129 pgs: 129 active+clean; 43 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 2.6 MiB/s wr, 414 op/s 2024-05-08T05:34:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:33 smithi143 ceph-mon[34337]: pgmap v152: 129 pgs: 129 active+clean; 43 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T05:34:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:33 smithi096 ceph-mon[34253]: pgmap v152: 129 pgs: 129 active+clean; 43 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T05:34:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:33 smithi059 ceph-mon[30304]: pgmap v152: 129 pgs: 129 active+clean; 43 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T05:34:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:35 smithi143 ceph-mon[34337]: pgmap v153: 129 pgs: 129 active+clean; 43 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T05:34:35.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:35 smithi096 ceph-mon[34253]: pgmap v153: 129 pgs: 129 active+clean; 43 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T05:34:35.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:35 smithi059 ceph-mon[30304]: pgmap v153: 129 pgs: 129 active+clean; 43 MiB data, 1004 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 1.6 MiB/s wr, 270 op/s 2024-05-08T05:34:37.096 INFO:tasks.workunit.client.0.smithi059.stdout:50000 copy from 0x898b8 to 0x92015, (0x875d bytes) at 0xb142c 2024-05-08T05:34:37.096 INFO:tasks.workunit.client.0.smithi059.stdout:All 50000 operations completed A-OK! 2024-05-08T05:34:37.098 INFO:tasks.workunit.client.0.smithi059.stderr:+ ./fsx -z 10MB -N 50000 -p 10000 -l 10485760 2024-05-08T05:34:37.105 INFO:tasks.workunit.client.0.smithi059.stdout:Seed set to 1 2024-05-08T05:34:37.233 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-05-08T05:34:37.233 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-05-08T05:34:37.235 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-08T05:34:37.235 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-08T05:34:37.235 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support clone range, disabling! 2024-05-08T05:34:37.237 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-08T05:34:37.239 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support exchange range, disabling! 2024-05-08T05:34:37.239 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero length punch hole 2024-05-08T05:34:37.240 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero length punch hole 2024-05-08T05:34:37.240 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero length punch hole 2024-05-08T05:34:37.240 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero size read 2024-05-08T05:34:37.240 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero length punch hole 2024-05-08T05:34:37.240 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero size read 2024-05-08T05:34:37.240 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero length punch hole 2024-05-08T05:34:37.240 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0x4310e9 2024-05-08T05:34:37.244 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0x954a45 2024-05-08T05:34:37.305 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0x9bc582 2024-05-08T05:34:37.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:37 smithi059 ceph-mon[30304]: pgmap v154: 129 pgs: 129 active+clean; 37 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 399 op/s 2024-05-08T05:34:37.338 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0x9eecbc 2024-05-08T05:34:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:37 smithi143 ceph-mon[34337]: pgmap v154: 129 pgs: 129 active+clean; 37 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 399 op/s 2024-05-08T05:34:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:37 smithi096 ceph-mon[34253]: pgmap v154: 129 pgs: 129 active+clean; 37 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 2.0 MiB/s rd, 2.4 MiB/s wr, 399 op/s 2024-05-08T05:34:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:39 smithi143 ceph-mon[34337]: pgmap v155: 129 pgs: 129 active+clean; 37 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-05-08T05:34:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:39 smithi096 ceph-mon[34253]: pgmap v155: 129 pgs: 129 active+clean; 37 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-05-08T05:34:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:39 smithi059 ceph-mon[30304]: pgmap v155: 129 pgs: 129 active+clean; 37 MiB data, 1009 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.6 MiB/s wr, 260 op/s 2024-05-08T05:34:40.226 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0x9fdd84 2024-05-08T05:34:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:41 smithi143 ceph-mon[34337]: pgmap v156: 129 pgs: 129 active+clean; 45 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 440 op/s 2024-05-08T05:34:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:41 smithi096 ceph-mon[34253]: pgmap v156: 129 pgs: 129 active+clean; 45 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 440 op/s 2024-05-08T05:34:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:41 smithi059 ceph-mon[30304]: pgmap v156: 129 pgs: 129 active+clean; 45 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 3.2 MiB/s rd, 2.6 MiB/s wr, 440 op/s 2024-05-08T05:34:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.472+0000 7f4352a3f640 1 -- 172.21.15.59:0/3955738329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c1024c0 msgr2=0x7f434c10e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.472+0000 7f4352a3f640 1 --2- 172.21.15.59:0/3955738329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c1024c0 0x7f434c10e6f0 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4344009ca0 tx=0x7f434402a8c0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.472+0000 7f4352a3f640 1 -- 172.21.15.59:0/3955738329 shutdown_connections 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.472+0000 7f4352a3f640 1 --2- 172.21.15.59:0/3955738329 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c1024c0 0x7f434c10e6f0 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.472+0000 7f4352a3f640 1 --2- 172.21.15.59:0/3955738329 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434c101b20 0x7f434c101f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.472+0000 7f4352a3f640 1 --2- 172.21.15.59:0/3955738329 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f434c107950 0x7f434c107d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.472+0000 7f4352a3f640 1 -- 172.21.15.59:0/3955738329 >> 172.21.15.59:0/3955738329 conn(0x7f434c078790 msgr2=0x7f434c078ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.472+0000 7f4352a3f640 1 -- 172.21.15.59:0/3955738329 shutdown_connections 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.472+0000 7f4352a3f640 1 -- 172.21.15.59:0/3955738329 wait complete. 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.473+0000 7f4352a3f640 1 Processor -- start 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.473+0000 7f4352a3f640 1 -- start start 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.473+0000 7f4352a3f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434c101b20 0x7f434c198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.473+0000 7f4352a3f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f434c1024c0 0x7f434c198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.473+0000 7f4352a3f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c107950 0x7f434c19c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.473+0000 7f4352a3f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f434c111b80 con 0x7f434c107950 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.473+0000 7f4352a3f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f434c111a00 con 0x7f434c1024c0 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.473+0000 7f4352a3f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f434c111d00 con 0x7f434c101b20 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.473+0000 7f4350fdc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c107950 0x7f434c19c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4350fdc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c107950 0x7f434c19c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46880/0 (socket says 172.21.15.59:46880) 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4350fdc640 1 -- 172.21.15.59:0/2821901093 learned_addr learned my addr 172.21.15.59:0/2821901093 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f434b7fe640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f434c1024c0 0x7f434c198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f434bfff640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434c101b20 0x7f434c198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4350fdc640 1 -- 172.21.15.59:0/2821901093 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434c101b20 msgr2=0x7f434c198020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4350fdc640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434c101b20 0x7f434c198020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4350fdc640 1 -- 172.21.15.59:0/2821901093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f434c1024c0 msgr2=0x7f434c198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4350fdc640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f434c1024c0 0x7f434c198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4350fdc640 1 -- 172.21.15.59:0/2821901093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f434c19d010 con 0x7f434c107950 2024-05-08T05:34:43.477 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4350fdc640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c107950 0x7f434c19c8e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4344005cc0 tx=0x7f4344031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f43497fa640 1 -- 172.21.15.59:0/2821901093 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4344003b70 con 0x7f434c107950 2024-05-08T05:34:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f43497fa640 1 -- 172.21.15.59:0/2821901093 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43440318f0 con 0x7f434c107950 2024-05-08T05:34:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4352a3f640 1 -- 172.21.15.59:0/2821901093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f434c19d2a0 con 0x7f434c107950 2024-05-08T05:34:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f43497fa640 1 -- 172.21.15.59:0/2821901093 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434402f950 con 0x7f434c107950 2024-05-08T05:34:43.478 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.474+0000 7f4352a3f640 1 -- 172.21.15.59:0/2821901093 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f434c06abb0 con 0x7f434c107950 2024-05-08T05:34:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f43497fa640 1 -- 172.21.15.59:0/2821901093 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f434402fb50 con 0x7f434c107950 2024-05-08T05:34:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f4355830740 1 Processor -- start 2024-05-08T05:34:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f4355830740 1 -- start start 2024-05-08T05:34:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f4355830740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c7162f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f4355830740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c716830 0x561c0c720bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f4355830740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 0x561c0c7234e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f4355830740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c0c67b5d0 con 0x561c0c7210f0 2024-05-08T05:34:43.481 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f4355830740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c0c634890 con 0x561c0c716830 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f4355830740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561c0c677590 con 0x561c0c715f10 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f434b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c7162f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f434b7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c7162f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:50476/0 (socket says 172.21.15.59:50476) 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f434bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c716830 0x561c0c720bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f434b7fe640 1 -- 172.21.15.59:0/4000531631 learned_addr learned my addr 172.21.15.59:0/4000531631 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.478+0000 7f434b7fe640 1 --2- 172.21.15.59:0/4000531631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 0x561c0c7234e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f434b7fe640 1 -- 172.21.15.59:0/4000531631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c716830 msgr2=0x561c0c720bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f434b7fe640 1 --2- 172.21.15.59:0/4000531631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c716830 0x561c0c720bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f434b7fe640 1 -- 172.21.15.59:0/4000531631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 msgr2=0x561c0c7234e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f434b7fe640 1 --2- 172.21.15.59:0/4000531631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 0x561c0c7234e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f434b7fe640 1 -- 172.21.15.59:0/4000531631 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0c38d210 con 0x561c0c715f10 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f43497fa640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43340b4c20 0x7f43340b70e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.482 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f434b7fe640 1 --2- 172.21.15.59:0/4000531631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c7162f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4340012750 tx=0x7f43400096e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a6d328cdc6edac36 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f434bfff640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43340b4c20 0x7f43340b70e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f43327fc640 1 -- 172.21.15.59:0/4000531631 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340012cd0 con 0x561c0c715f10 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f43327fc640 1 -- 172.21.15.59:0/4000531631 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43400107d0 con 0x561c0c715f10 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f43497fa640 1 -- 172.21.15.59:0/2821901093 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f4344103c60 con 0x7f434c107950 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.479+0000 7f43327fc640 1 -- 172.21.15.59:0/4000531631 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43400044e0 con 0x561c0c715f10 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f4355830740 1 -- 172.21.15.59:0/4000531631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 msgr2=0x561c0c7162f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f434bfff640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43340b4c20 0x7f43340b70e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x561c0c7174b0 tx=0x7f433c005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f4355830740 1 --2- 172.21.15.59:0/4000531631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c7162f0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4340012750 tx=0x7f43400096e0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f4355830740 1 -- 172.21.15.59:0/4000531631 shutdown_connections 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f4355830740 1 --2- 172.21.15.59:0/4000531631 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 0x561c0c7234e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f4355830740 1 --2- 172.21.15.59:0/4000531631 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c716830 0x561c0c720bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f4355830740 1 --2- 172.21.15.59:0/4000531631 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c7162f0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f4355830740 1 -- 172.21.15.59:0/4000531631 >> 172.21.15.59:0/4000531631 conn(0x561c0c712420 msgr2=0x561c0c712800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f4355830740 1 -- 172.21.15.59:0/4000531631 shutdown_connections 2024-05-08T05:34:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.480+0000 7f4355830740 1 -- 172.21.15.59:0/4000531631 wait complete. 2024-05-08T05:34:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.481+0000 7f4355830740 1 -- start start 2024-05-08T05:34:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.481+0000 7f4355830740 1 Processor -- start 2024-05-08T05:34:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.481+0000 7f4355830740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c71e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.481+0000 7f4355830740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c71e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.481+0000 7f4355830740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c7210f0 0x561c0c71eb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f4355830740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561c0c723d20 con 0x561c0c716830 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f4355830740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561c0c638370 con 0x561c0c7210f0 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f4355830740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561c0c723ea0 con 0x561c0c715f10 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434b7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c71e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434b7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c71e5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46898/0 (socket says 172.21.15.59:46898) 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f4350fdc640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c7210f0 0x561c0c71eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434b7fe640 1 -- 172.21.15.59:0/2036532199 learned_addr learned my addr 172.21.15.59:0/2036532199 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434bfff640 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c71e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f4355830740 10 client.0 _ll_register_callbacks cb 0x561c0c38ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434b7fe640 1 -- 172.21.15.59:0/2036532199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 msgr2=0x561c0c71e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434b7fe640 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c71e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434b7fe640 1 -- 172.21.15.59:0/2036532199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c7210f0 msgr2=0x561c0c71eb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434b7fe640 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c7210f0 0x561c0c71eb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434b7fe640 1 -- 172.21.15.59:0/2036532199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0c459620 con 0x561c0c716830 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f4350fdc640 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c7210f0 0x561c0c71eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f434b7fe640 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c71e5f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4340005c50 tx=0x7f4340005c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8799bf935f7bf020 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f432b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f432b7fe640 1 -- 172.21.15.59:0/2036532199 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4340012030 con 0x561c0c716830 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f4355830740 1 -- 172.21.15.59:0/2036532199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43400107d0 con 0x561c0c716830 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.482+0000 7f432b7fe640 1 -- 172.21.15.59:0/2036532199 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f43400121d0 con 0x561c0c716830 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f432b7fe640 1 -- 172.21.15.59:0/2036532199 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4340030050 con 0x561c0c716830 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f4355830740 10 client.16104 fetch_fsmap learned FSMap version 36 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f4355830740 1 -- 172.21.15.59:0/2036532199 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561c0c734030 con 0x561c0c716830 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f432b7fe640 1 -- 172.21.15.59:0/2036532199 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43400107d0 con 0x561c0c716830 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f432b7fe640 1 -- 172.21.15.59:0/2036532199 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4340003a90 con 0x561c0c716830 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f4355830740 10 client.16104 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:34:43.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f4355830740 10 client.16104 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f4355830740 20 client.16104 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f4355830740 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561c0c7358b0 0x561c0c737d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f4355830740 4 client.16104 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.483+0000 7f4355830740 1 -- 172.21.15.59:0/2036532199 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c0c6891e0 con 0x561c0c7358b0 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.484+0000 7f434bfff640 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561c0c7358b0 0x561c0c737d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.484+0000 7f432b7fe640 10 client.16104 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.484+0000 7f434bfff640 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561c0c7358b0 0x561c0c737d30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=801c9fdeae1760c5 server_cookie=db0023f3e073b1a9 in_seq=0 out_seq=0 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f432b7fe640 1 -- 172.21.15.59:0/2036532199 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561c0c6891e0 con 0x561c0c7358b0 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f432b7fe640 10 client.16104 handle_command_reply: tid=0 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 client.16104 shutdown 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 -- 172.21.15.59:0/2036532199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 msgr2=0x561c0c71e5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c71e5f0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4340005c50 tx=0x7f4340005c80 comp rx=0 tx=0).stop 2024-05-08T05:34:43.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 -- 172.21.15.59:0/2036532199 shutdown_connections 2024-05-08T05:34:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561c0c7358b0 0x561c0c737d30 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c7210f0 0x561c0c71eb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c71e5f0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 --2- 172.21.15.59:0/2036532199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c715f10 0x561c0c71e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 -- 172.21.15.59:0/2036532199 >> 172.21.15.59:0/2036532199 conn(0x561c0c712420 msgr2=0x561c0c718150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f432b7fe640 0 client.16104 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.485+0000 7f4355830740 1 -- 172.21.15.59:0/2036532199 shutdown_connections 2024-05-08T05:34:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.486+0000 7f4355830740 1 -- 172.21.15.59:0/2036532199 wait complete. 2024-05-08T05:34:43.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.486+0000 7f4355830740 20 client.16104 trim_cache size 0 max 16384 2024-05-08T05:34:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:43 smithi143 ceph-mon[34337]: pgmap v157: 129 pgs: 129 active+clean; 45 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2024-05-08T05:34:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:43 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:34:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.508+0000 7f4355830740 1 Processor -- start 2024-05-08T05:34:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.508+0000 7f4355830740 1 -- start start 2024-05-08T05:34:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.508+0000 7f4355830740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c711400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.508+0000 7f4355830740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c73d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.510 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f4355830740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c7210f0 0x561c0c73db80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f4355830740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1ada70 con 0x561c0c716830 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f4355830740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1af5b0 con 0x561c0c715f10 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f4355830740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1ae140 con 0x561c0c7210f0 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434b7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c73d640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434b7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c73d640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46902/0 (socket says 172.21.15.59:46902) 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434b7fe640 1 -- 172.21.15.59:0/111400054 learned_addr learned my addr 172.21.15.59:0/111400054 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434bfff640 1 --2- 172.21.15.59:0/111400054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c711400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f4350fdc640 1 --2- 172.21.15.59:0/111400054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c7210f0 0x561c0c73db80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434b7fe640 1 -- 172.21.15.59:0/111400054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c7210f0 msgr2=0x561c0c73db80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434b7fe640 1 --2- 172.21.15.59:0/111400054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c7210f0 0x561c0c73db80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434b7fe640 1 -- 172.21.15.59:0/111400054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 msgr2=0x561c0c711400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434b7fe640 1 --2- 172.21.15.59:0/111400054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c711400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434b7fe640 1 -- 172.21.15.59:0/111400054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c0c716660 con 0x561c0c716830 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.509+0000 7f434b7fe640 1 --2- 172.21.15.59:0/111400054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c73d640 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f43400095b0 tx=0x7f4340012650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7bbb70c0b286c7 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:43.511 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.510+0000 7f4331ffb640 1 -- 172.21.15.59:0/111400054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43400096b0 con 0x561c0c716830 2024-05-08T05:34:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.510+0000 7f4331ffb640 1 -- 172.21.15.59:0/111400054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4340009a10 con 0x561c0c716830 2024-05-08T05:34:43.512 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.510+0000 7f4331ffb640 1 -- 172.21.15.59:0/111400054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434002ac40 con 0x561c0c716830 2024-05-08T05:34:43.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.510+0000 7f4355830740 1 -- 172.21.15.59:0/111400054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 msgr2=0x561c0c73d640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:43.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.510+0000 7f4355830740 1 --2- 172.21.15.59:0/111400054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c73d640 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f43400095b0 tx=0x7f4340012650 comp rx=0 tx=0).stop 2024-05-08T05:34:43.516 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.511+0000 7f4355830740 1 -- 172.21.15.59:0/111400054 shutdown_connections 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.511+0000 7f4355830740 1 --2- 172.21.15.59:0/111400054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c7210f0 0x561c0c73db80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.511+0000 7f4355830740 1 --2- 172.21.15.59:0/111400054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c716830 0x561c0c73d640 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.511+0000 7f4355830740 1 --2- 172.21.15.59:0/111400054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c711400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.511+0000 7f4355830740 1 -- 172.21.15.59:0/111400054 >> 172.21.15.59:0/111400054 conn(0x561c0c712420 msgr2=0x561c0c710cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.511+0000 7f4355830740 1 -- 172.21.15.59:0/111400054 shutdown_connections 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.511+0000 7f4355830740 1 -- 172.21.15.59:0/111400054 wait complete. 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.511+0000 7f4355830740 1 -- start start 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.511+0000 7f4355830740 1 Processor -- start 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f4355830740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c72d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f4355830740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c716830 0x561c0c728ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f4355830740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 0x561c0c7290e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f4355830740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1aeee0 con 0x561c0c7210f0 2024-05-08T05:34:43.517 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f4355830740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1ae810 con 0x561c0c715f10 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f4355830740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f434c1aba30 con 0x561c0c716830 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f4355830740 10 client.0 _ll_register_callbacks cb 0x7f4340009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c72d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434bfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c72d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:36502/0 (socket says 172.21.15.59:36502) 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434bfff640 1 -- 172.21.15.59:0/4165278147 learned_addr learned my addr 172.21.15.59:0/4165278147 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434b7fe640 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c716830 0x561c0c728ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f4350fdc640 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 0x561c0c7290e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434b7fe640 1 -- 172.21.15.59:0/4165278147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 msgr2=0x561c0c72d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434b7fe640 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c72d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434b7fe640 1 -- 172.21.15.59:0/4165278147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 msgr2=0x561c0c7290e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434b7fe640 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 0x561c0c7290e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434b7fe640 1 -- 172.21.15.59:0/4165278147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c0c729620 con 0x561c0c716830 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.512+0000 7f434bfff640 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c72d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:34:43.518 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f434b7fe640 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c716830 0x561c0c728ba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4340005d80 tx=0x7f43400109d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ecf03007661d98d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f432b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f432b7fe640 1 -- 172.21.15.59:0/4165278147 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43400096b0 con 0x561c0c716830 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f4355830740 1 -- 172.21.15.59:0/4165278147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4340009a10 con 0x561c0c716830 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f432b7fe640 1 -- 172.21.15.59:0/4165278147 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f4340010dc0 con 0x561c0c716830 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f432b7fe640 1 -- 172.21.15.59:0/4165278147 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f4340045050 con 0x561c0c716830 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f4355830740 10 client.25666 fetch_fsmap learned FSMap version 36 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f4355830740 1 -- 172.21.15.59:0/4165278147 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561c0c79fec0 con 0x561c0c716830 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f432b7fe640 1 -- 172.21.15.59:0/4165278147 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4340009a10 con 0x561c0c716830 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f432b7fe640 1 -- 172.21.15.59:0/4165278147 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f4340003a60 con 0x561c0c716830 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.513+0000 7f4355830740 10 client.25666 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.514+0000 7f4355830740 10 client.25666 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.514+0000 7f4355830740 20 client.25666 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:43.519 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.514+0000 7f4355830740 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561c0c7a0520 0x561c0c7a2910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.514+0000 7f4355830740 4 client.25666 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:34:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.514+0000 7f4355830740 1 -- 172.21.15.59:0/4165278147 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561c0c683050 con 0x561c0c7a0520 2024-05-08T05:34:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.514+0000 7f434bfff640 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561c0c7a0520 0x561c0c7a2910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.516+0000 7f432b7fe640 10 client.25666 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:43.520 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.516+0000 7f434bfff640 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561c0c7a0520 0x561c0c7a2910 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f64f7331db0ec5f server_cookie=f7c326f47b5c3074 in_seq=0 out_seq=0 2024-05-08T05:34:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:43 smithi096 ceph-mon[34253]: pgmap v157: 129 pgs: 129 active+clean; 45 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2024-05-08T05:34:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:43 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:34:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f432b7fe640 1 -- 172.21.15.59:0/4165278147 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561c0c683050 con 0x561c0c7a0520 2024-05-08T05:34:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f432b7fe640 10 client.25666 handle_command_reply: tid=0 2024-05-08T05:34:43.533 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 client.25666 shutdown 2024-05-08T05:34:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 -- 172.21.15.59:0/4165278147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c716830 msgr2=0x561c0c728ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c716830 0x561c0c728ba0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4340005d80 tx=0x7f43400109d0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 -- 172.21.15.59:0/4165278147 shutdown_connections 2024-05-08T05:34:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561c0c7a0520 0x561c0c7a2910 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561c0c7210f0 0x561c0c7290e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561c0c716830 0x561c0c728ba0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.534 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f432b7fe640 0 client.25666 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 --2- 172.21.15.59:0/4165278147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561c0c715f10 0x561c0c72d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 -- 172.21.15.59:0/4165278147 >> 172.21.15.59:0/4165278147 conn(0x561c0c712420 msgr2=0x561c0c711580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 -- 172.21.15.59:0/4165278147 shutdown_connections 2024-05-08T05:34:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.533+0000 7f4355830740 1 -- 172.21.15.59:0/4165278147 wait complete. 2024-05-08T05:34:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.534+0000 7f4355830740 20 client.25666 trim_cache size 0 max 16384 2024-05-08T05:34:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:34:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:34:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:34:43.535 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:34:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.534+0000 7f4352a3f640 1 -- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43340b4c20 msgr2=0x7f43340b70e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:43.536 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.535+0000 7f4352a3f640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43340b4c20 0x7f43340b70e0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x561c0c7174b0 tx=0x7f433c005f90 comp rx=0 tx=0).stop 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.535+0000 7f4352a3f640 1 -- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c107950 msgr2=0x7f434c19c8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.535+0000 7f4352a3f640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c107950 0x7f434c19c8e0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4344005cc0 tx=0x7f4344031700 comp rx=0 tx=0).stop 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.535+0000 7f4352a3f640 1 -- 172.21.15.59:0/2821901093 shutdown_connections 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.535+0000 7f4352a3f640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43340b4c20 0x7f43340b70e0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.535+0000 7f4352a3f640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f434c107950 0x7f434c19c8e0 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.535+0000 7f4352a3f640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f434c1024c0 0x7f434c198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.535+0000 7f4352a3f640 1 --2- 172.21.15.59:0/2821901093 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f434c101b20 0x7f434c198020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.535+0000 7f4352a3f640 1 -- 172.21.15.59:0/2821901093 >> 172.21.15.59:0/2821901093 conn(0x7f434c078790 msgr2=0x7f434c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.536+0000 7f4352a3f640 1 -- 172.21.15.59:0/2821901093 shutdown_connections 2024-05-08T05:34:43.537 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:43.536+0000 7f4352a3f640 1 -- 172.21.15.59:0/2821901093 wait complete. 2024-05-08T05:34:43.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:43 smithi059 ceph-mon[30304]: pgmap v157: 129 pgs: 129 active+clean; 45 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2024-05-08T05:34:43.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:43 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:34:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:44 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:34:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:44 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:34:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:44 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:34:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:44 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:34:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:44 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:34:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:44 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:34:44.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:44 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:34:44.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:44 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:34:44.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:44 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:34:45.394 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:45 smithi143 ceph-mon[34337]: pgmap v158: 129 pgs: 129 active+clean; 45 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2024-05-08T05:34:45.430 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:45 smithi096 ceph-mon[34253]: pgmap v158: 129 pgs: 129 active+clean; 45 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2024-05-08T05:34:45.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:45 smithi059 ceph-mon[30304]: pgmap v158: 129 pgs: 129 active+clean; 45 MiB data, 982 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.8 MiB/s wr, 308 op/s 2024-05-08T05:34:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:47 smithi143 ceph-mon[34337]: pgmap v159: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 548 op/s 2024-05-08T05:34:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:47 smithi096 ceph-mon[34253]: pgmap v159: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 548 op/s 2024-05-08T05:34:47.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:47 smithi059 ceph-mon[30304]: pgmap v159: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 3.1 MiB/s wr, 548 op/s 2024-05-08T05:34:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:49 smithi143 ceph-mon[34337]: pgmap v160: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 420 op/s 2024-05-08T05:34:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:49 smithi096 ceph-mon[34253]: pgmap v160: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 420 op/s 2024-05-08T05:34:49.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:49 smithi059 ceph-mon[30304]: pgmap v160: 129 pgs: 129 active+clean; 47 MiB data, 1.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.3 MiB/s wr, 420 op/s 2024-05-08T05:34:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:51 smithi143 ceph-mon[34337]: pgmap v161: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 638 op/s 2024-05-08T05:34:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:51 smithi096 ceph-mon[34253]: pgmap v161: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 638 op/s 2024-05-08T05:34:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:51 smithi059 ceph-mon[30304]: pgmap v161: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.1 MiB/s rd, 3.5 MiB/s wr, 638 op/s 2024-05-08T05:34:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:53 smithi143 ceph-mon[34337]: pgmap v162: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-05-08T05:34:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:53 smithi096 ceph-mon[34253]: pgmap v162: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-05-08T05:34:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:53 smithi059 ceph-mon[30304]: pgmap v162: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-05-08T05:34:55.387 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:55 smithi059 ceph-mon[30304]: pgmap v163: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-05-08T05:34:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:55 smithi143 ceph-mon[34337]: pgmap v163: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-05-08T05:34:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:55 smithi096 ceph-mon[34253]: pgmap v163: 129 pgs: 129 active+clean; 49 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.5 MiB/s wr, 458 op/s 2024-05-08T05:34:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.653+0000 7fbb1f42a640 1 -- 172.21.15.59:0/3864955245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18101900 msgr2=0x7fbb18101d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.653+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/3864955245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18101900 0x7fbb18101d80 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fbb08009890 tx=0x7fbb0802a380 comp rx=0 tx=0).stop 2024-05-08T05:34:56.659 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.653+0000 7fbb1f42a640 1 -- 172.21.15.59:0/3864955245 shutdown_connections 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.653+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/3864955245 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb181022c0 0x7fbb1810e780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.653+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/3864955245 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb18101900 0x7fbb18101d80 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.653+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/3864955245 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb181078c0 0x7fbb18107ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.653+0000 7fbb1f42a640 1 -- 172.21.15.59:0/3864955245 >> 172.21.15.59:0/3864955245 conn(0x7fbb18078a00 msgr2=0x7fbb18078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.653+0000 7fbb1f42a640 1 -- 172.21.15.59:0/3864955245 shutdown_connections 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.654+0000 7fbb1f42a640 1 -- 172.21.15.59:0/3864955245 wait complete. 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1f42a640 1 Processor -- start 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1f42a640 1 -- start start 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1f42a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb18101900 0x7fbb181980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1f42a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb181022c0 0x7fbb181985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1f42a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb181078c0 0x7fbb1819c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1f42a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18111b80 con 0x7fbb181078c0 2024-05-08T05:34:56.660 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1f42a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18111a00 con 0x7fbb18101900 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1f42a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbb18111d00 con 0x7fbb181022c0 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1d1c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb18101900 0x7fbb181980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.655+0000 7fbb1d9c7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb181078c0 0x7fbb1819c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1c9c5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb181022c0 0x7fbb181985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1d1c6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb18101900 0x7fbb181980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:59746/0 (socket says 172.21.15.59:59746) 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1d1c6640 1 -- 172.21.15.59:0/2362972291 learned_addr learned my addr 172.21.15.59:0/2362972291 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1c9c5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb181022c0 0x7fbb181985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:49390/0 (socket says 172.21.15.59:49390) 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1d9c7640 1 -- 172.21.15.59:0/2362972291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb181022c0 msgr2=0x7fbb181985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1d9c7640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb181022c0 0x7fbb181985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1d9c7640 1 -- 172.21.15.59:0/2362972291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb18101900 msgr2=0x7fbb181980a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1d9c7640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb18101900 0x7fbb181980a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1d9c7640 1 -- 172.21.15.59:0/2362972291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb1819d040 con 0x7fbb181078c0 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1d9c7640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb181078c0 0x7fbb1819c910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000da10 tx=0x7fbb1000ded0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:56.661 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb0e7fc640 1 -- 172.21.15.59:0/2362972291 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1000bd70 con 0x7fbb181078c0 2024-05-08T05:34:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb0e7fc640 1 -- 172.21.15.59:0/2362972291 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb10003e20 con 0x7fbb181078c0 2024-05-08T05:34:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1f42a640 1 -- 172.21.15.59:0/2362972291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb1819d330 con 0x7fbb181078c0 2024-05-08T05:34:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb0e7fc640 1 -- 172.21.15.59:0/2362972291 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100106c0 con 0x7fbb181078c0 2024-05-08T05:34:56.662 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.656+0000 7fbb1f42a640 1 -- 172.21.15.59:0/2362972291 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb1806ac10 con 0x7fbb181078c0 2024-05-08T05:34:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.659+0000 7fbb22222740 1 Processor -- start 2024-05-08T05:34:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.660+0000 7fbb0e7fc640 1 -- 172.21.15.59:0/2362972291 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbb10004270 con 0x7fbb181078c0 2024-05-08T05:34:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb22222740 1 -- start start 2024-05-08T05:34:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb22222740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa5f10 0x561e13aa62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb22222740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13ab0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb22222740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 0x561e13ab34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.663 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb22222740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561e13a0b5d0 con 0x561e13aa6830 2024-05-08T05:34:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb22222740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561e139c4890 con 0x561e13aa5f10 2024-05-08T05:34:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb22222740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561e13a07590 con 0x561e13ab10f0 2024-05-08T05:34:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb1c9c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa5f10 0x561e13aa62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb1c9c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa5f10 0x561e13aa62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:59750/0 (socket says 172.21.15.59:59750) 2024-05-08T05:34:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb1c9c5640 1 -- 172.21.15.59:0/1988703212 learned_addr learned my addr 172.21.15.59:0/1988703212 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/1988703212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 0x561e13ab34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.661+0000 7fbb1d1c6640 1 --2- 172.21.15.59:0/1988703212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13ab0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.662+0000 7fbb1c9c5640 1 -- 172.21.15.59:0/1988703212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 msgr2=0x561e13ab34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:56.664 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.662+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/1988703212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 0x561e13ab34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.662+0000 7fbb1c9c5640 1 -- 172.21.15.59:0/1988703212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 msgr2=0x561e13ab0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.662+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/1988703212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13ab0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.662+0000 7fbb1c9c5640 1 -- 172.21.15.59:0/1988703212 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e1371d210 con 0x561e13aa5f10 2024-05-08T05:34:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.662+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/1988703212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa5f10 0x561e13aa62f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb08005b60 tx=0x7fbb08005a90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6d3dc28d2e6a1c0e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.662+0000 7fbb0e7fc640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbb040b2a10 0x7fbb040b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.665 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.663+0000 7fbb0e7fc640 1 -- 172.21.15.59:0/2362972291 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fbb100e2cf0 con 0x7fbb181078c0 2024-05-08T05:34:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.663+0000 7fbb1d1c6640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbb040b2a10 0x7fbb040b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.664+0000 7fbb037fe640 1 -- 172.21.15.59:0/1988703212 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08037040 con 0x561e13aa5f10 2024-05-08T05:34:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.664+0000 7fbb037fe640 1 -- 172.21.15.59:0/1988703212 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb08030450 con 0x561e13aa5f10 2024-05-08T05:34:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb22222740 1 -- 172.21.15.59:0/1988703212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa5f10 msgr2=0x561e13aa62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb1d1c6640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbb040b2a10 0x7fbb040b4ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x561e13aa74b0 tx=0x7fbb14009260 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:56.668 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb22222740 1 --2- 172.21.15.59:0/1988703212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa5f10 0x561e13aa62f0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fbb08005b60 tx=0x7fbb08005a90 comp rx=0 tx=0).stop 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb22222740 1 -- 172.21.15.59:0/1988703212 shutdown_connections 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb22222740 1 --2- 172.21.15.59:0/1988703212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 0x561e13ab34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb22222740 1 --2- 172.21.15.59:0/1988703212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13ab0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb22222740 1 --2- 172.21.15.59:0/1988703212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa5f10 0x561e13aa62f0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb22222740 1 -- 172.21.15.59:0/1988703212 >> 172.21.15.59:0/1988703212 conn(0x561e13aa2420 msgr2=0x561e13aa2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb22222740 1 -- 172.21.15.59:0/1988703212 shutdown_connections 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.665+0000 7fbb22222740 1 -- 172.21.15.59:0/1988703212 wait complete. 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.666+0000 7fbb22222740 1 -- start start 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.666+0000 7fbb22222740 1 Processor -- start 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.666+0000 7fbb22222740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13aadf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.666+0000 7fbb22222740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13aae4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.666+0000 7fbb22222740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13aaea10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.666+0000 7fbb22222740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x561e13ab3cc0 con 0x561e13aa6830 2024-05-08T05:34:56.669 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.666+0000 7fbb22222740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x561e139c8370 con 0x561e13ab10f0 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.666+0000 7fbb22222740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x561e13ab3e40 con 0x561e13aa5f10 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.666+0000 7fbb22222740 10 client.0 _ll_register_callbacks cb 0x561e1371ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d1c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13aadf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d1c6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13aadf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:49408/0 (socket says 172.21.15.59:49408) 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d1c6640 1 -- 172.21.15.59:0/1974563199 learned_addr learned my addr 172.21.15.59:0/1974563199 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d9c7640 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13aaea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13aae4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d1c6640 1 -- 172.21.15.59:0/1974563199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 msgr2=0x561e13aaea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d1c6640 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13aaea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d1c6640 1 -- 172.21.15.59:0/1974563199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 msgr2=0x561e13aae4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d1c6640 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13aae4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d1c6640 1 -- 172.21.15.59:0/1974563199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e137e9620 con 0x561e13aa5f10 2024-05-08T05:34:56.670 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb1d1c6640 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13aadf90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb14005c20 tx=0x7fbb14004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f4ff21f56070ab server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:34:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.667+0000 7fbb00ff9640 1 -- 172.21.15.59:0/1974563199 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb1402d060 con 0x561e13aa5f10 2024-05-08T05:34:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb22222740 1 -- 172.21.15.59:0/1974563199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb08030450 con 0x561e13aa5f10 2024-05-08T05:34:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb00ff9640 1 -- 172.21.15.59:0/1974563199 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fbb14018030 con 0x561e13aa5f10 2024-05-08T05:34:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb00ff9640 1 -- 172.21.15.59:0/1974563199 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb14034050 con 0x561e13aa5f10 2024-05-08T05:34:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb22222740 10 client.25678 fetch_fsmap learned FSMap version 36 2024-05-08T05:34:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb22222740 1 -- 172.21.15.59:0/1974563199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561e13abfb90 con 0x561e13aa5f10 2024-05-08T05:34:56.671 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb00ff9640 1 -- 172.21.15.59:0/1974563199 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb08030450 con 0x561e13aa5f10 2024-05-08T05:34:56.672 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb00ff9640 1 -- 172.21.15.59:0/1974563199 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbb1400f3c0 con 0x561e13aa5f10 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb22222740 10 client.25678 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb22222740 10 client.25678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb22222740 20 client.25678 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb22222740 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561e13ac1050 0x561e13ac3440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb22222740 4 client.25678 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.668+0000 7fbb22222740 1 -- 172.21.15.59:0/1974563199 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561e13a191e0 con 0x561e13ac1050 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.669+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561e13ac1050 0x561e13ac3440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.669+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561e13ac1050 0x561e13ac3440 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98a48d9a54096647 server_cookie=3a87e6b9765a5d75 in_seq=0 out_seq=0 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.669+0000 7fbb00ff9640 10 client.25678 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb00ff9640 1 -- 172.21.15.59:0/1974563199 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x561e13a191e0 con 0x561e13ac1050 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb00ff9640 10 client.25678 handle_command_reply: tid=0 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 client.25678 shutdown 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 -- 172.21.15.59:0/1974563199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 msgr2=0x561e13aadf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13aadf90 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fbb14005c20 tx=0x7fbb14004730 comp rx=0 tx=0).stop 2024-05-08T05:34:56.673 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 -- 172.21.15.59:0/1974563199 shutdown_connections 2024-05-08T05:34:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561e13ac1050 0x561e13ac3440 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13aaea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13aae4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 --2- 172.21.15.59:0/1974563199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13aadf90 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb00ff9640 0 client.25678 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 -- 172.21.15.59:0/1974563199 >> 172.21.15.59:0/1974563199 conn(0x561e13aa2420 msgr2=0x561e13aa81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 -- 172.21.15.59:0/1974563199 shutdown_connections 2024-05-08T05:34:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.670+0000 7fbb22222740 1 -- 172.21.15.59:0/1974563199 wait complete. 2024-05-08T05:34:56.674 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.671+0000 7fbb22222740 20 client.25678 trim_cache size 0 max 16384 2024-05-08T05:34:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.691+0000 7fbb22222740 1 Processor -- start 2024-05-08T05:34:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb22222740 1 -- start start 2024-05-08T05:34:56.693 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb22222740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13ab0290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb22222740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13aae830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb22222740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13ac8d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb22222740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b1f10 con 0x561e13aa6830 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb22222740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b3a50 con 0x561e13ab10f0 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb22222740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b25e0 con 0x561e13aa5f10 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1d9c7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13ac8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1d9c7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13ac8d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:59776/0 (socket says 172.21.15.59:59776) 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1d9c7640 1 -- 172.21.15.59:0/4160010604 learned_addr learned my addr 172.21.15.59:0/4160010604 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1d1c6640 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13ab0290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13aae830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.694 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1d9c7640 1 -- 172.21.15.59:0/4160010604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 msgr2=0x561e13ab0290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1d9c7640 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13ab0290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1d9c7640 1 -- 172.21.15.59:0/4160010604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 msgr2=0x561e13aae830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1d9c7640 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13aae830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1d9c7640 1 -- 172.21.15.59:0/4160010604 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561e13ac37b0 con 0x561e13ab10f0 2024-05-08T05:34:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.692+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13aae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:34:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.693+0000 7fbb1d9c7640 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13ac8d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbb100045c0 tx=0x7fbb10004610 comp rx=0 tx=0).ready entity=mon.2 client_cookie=74f8f1edb3c47907 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.693+0000 7fbb02ffd640 1 -- 172.21.15.59:0/4160010604 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100b2410 con 0x561e13ab10f0 2024-05-08T05:34:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.693+0000 7fbb02ffd640 1 -- 172.21.15.59:0/4160010604 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb10091070 con 0x561e13ab10f0 2024-05-08T05:34:56.695 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.693+0000 7fbb02ffd640 1 -- 172.21.15.59:0/4160010604 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb100b2630 con 0x561e13ab10f0 2024-05-08T05:34:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 -- 172.21.15.59:0/4160010604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 msgr2=0x561e13ac8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:56.698 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13ac8d30 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbb100045c0 tx=0x7fbb10004610 comp rx=0 tx=0).stop 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 -- 172.21.15.59:0/4160010604 shutdown_connections 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13ab10f0 0x561e13ac8d30 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa6830 0x561e13aae830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 --2- 172.21.15.59:0/4160010604 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13aa5f10 0x561e13ab0290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 -- 172.21.15.59:0/4160010604 >> 172.21.15.59:0/4160010604 conn(0x561e13aa2420 msgr2=0x561e13aa1580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 -- 172.21.15.59:0/4160010604 shutdown_connections 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 -- 172.21.15.59:0/4160010604 wait complete. 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 -- start start 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.694+0000 7fbb22222740 1 Processor -- start 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb22222740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa5f10 0x561e13ab90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb22222740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa6830 0x561e13ab9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb22222740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 0x561e13ab60b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.699 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb22222740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b3380 con 0x561e13aa5f10 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb22222740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181b2cb0 con 0x561e13aa6830 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb22222740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbb181afed0 con 0x561e13ab10f0 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb22222740 10 client.0 _ll_register_callbacks cb 0x7fbb14009a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1d1c6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa5f10 0x561e13ab90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1c9c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa6830 0x561e13ab9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1c9c5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa6830 0x561e13ab9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:59782/0 (socket says 172.21.15.59:59782) 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1d1c6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa5f10 0x561e13ab90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:41566/0 (socket says 172.21.15.59:41566) 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1c9c5640 1 -- 172.21.15.59:0/936506772 learned_addr learned my addr 172.21.15.59:0/936506772 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1d9c7640 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 0x561e13ab60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1c9c5640 1 -- 172.21.15.59:0/936506772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 msgr2=0x561e13ab60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 0x561e13ab60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1c9c5640 1 -- 172.21.15.59:0/936506772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa5f10 msgr2=0x561e13ab90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa5f10 0x561e13ab90e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.700 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1c9c5640 1 -- 172.21.15.59:0/936506772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561e13ab65f0 con 0x561e13aa6830 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1d9c7640 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 0x561e13ab60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb1c9c5640 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa6830 0x561e13ab9620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb08000c00 tx=0x7fbb0802a380 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1c475b0730358554 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb00ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb00ff9640 1 -- 172.21.15.59:0/936506772 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb08037040 con 0x561e13aa6830 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb00ff9640 1 -- 172.21.15.59:0/936506772 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fbb08003bf0 con 0x561e13aa6830 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.695+0000 7fbb22222740 1 -- 172.21.15.59:0/936506772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb10091070 con 0x561e13aa6830 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb00ff9640 1 -- 172.21.15.59:0/936506772 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbb08003df0 con 0x561e13aa6830 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb22222740 10 client.25730 fetch_fsmap learned FSMap version 36 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb22222740 1 -- 172.21.15.59:0/936506772 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x561e13b2c0c0 con 0x561e13aa6830 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb00ff9640 1 -- 172.21.15.59:0/936506772 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb10091070 con 0x561e13aa6830 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb00ff9640 1 -- 172.21.15.59:0/936506772 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbb0804d170 con 0x561e13aa6830 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb22222740 10 client.25730 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb22222740 10 client.25730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:34:56.701 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb22222740 20 client.25730 populate_metadata read hostname 'smithi059' 2024-05-08T05:34:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb22222740 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561e13b2c290 0x561e13b2e680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:34:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb22222740 4 client.25730 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:34:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb22222740 1 -- 172.21.15.59:0/936506772 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x561e13a13050 con 0x561e13b2c290 2024-05-08T05:34:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.696+0000 7fbb1d1c6640 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561e13b2c290 0x561e13b2e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:34:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.697+0000 7fbb00ff9640 10 client.25730 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:56.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.697+0000 7fbb1d1c6640 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561e13b2c290 0x561e13b2e680 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e36fc687f0ea7518 server_cookie=680b89c9fad9e55c in_seq=0 out_seq=0 2024-05-08T05:34:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.716+0000 7fbb00ff9640 1 -- 172.21.15.59:0/936506772 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x561e13a13050 con 0x561e13b2c290 2024-05-08T05:34:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.716+0000 7fbb00ff9640 10 client.25730 handle_command_reply: tid=0 2024-05-08T05:34:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.716+0000 7fbb22222740 1 client.25730 shutdown 2024-05-08T05:34:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.716+0000 7fbb22222740 1 -- 172.21.15.59:0/936506772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa6830 msgr2=0x561e13ab9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.716+0000 7fbb22222740 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa6830 0x561e13ab9620 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fbb08000c00 tx=0x7fbb0802a380 comp rx=0 tx=0).stop 2024-05-08T05:34:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb22222740 1 -- 172.21.15.59:0/936506772 shutdown_connections 2024-05-08T05:34:56.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb22222740 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x561e13b2c290 0x561e13b2e680 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb22222740 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x561e13ab10f0 0x561e13ab60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb22222740 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x561e13aa6830 0x561e13ab9620 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb22222740 1 --2- 172.21.15.59:0/936506772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x561e13aa5f10 0x561e13ab90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb22222740 1 -- 172.21.15.59:0/936506772 >> 172.21.15.59:0/936506772 conn(0x561e13aa2420 msgr2=0x561e13abbd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb00ff9640 0 client.25730 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:34:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb22222740 1 -- 172.21.15.59:0/936506772 shutdown_connections 2024-05-08T05:34:56.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb22222740 1 -- 172.21.15.59:0/936506772 wait complete. 2024-05-08T05:34:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.717+0000 7fbb22222740 20 client.25730 trim_cache size 0 max 16384 2024-05-08T05:34:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:34:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:34:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:34:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:34:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.719+0000 7fbb1f42a640 1 -- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbb040b2a10 msgr2=0x7fbb040b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:56.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.719+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbb040b2a10 0x7fbb040b4ed0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x561e13aa74b0 tx=0x7fbb14009260 comp rx=0 tx=0).stop 2024-05-08T05:34:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.719+0000 7fbb1f42a640 1 -- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb181078c0 msgr2=0x7fbb1819c910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:34:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.719+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb181078c0 0x7fbb1819c910 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fbb1000da10 tx=0x7fbb1000ded0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.720+0000 7fbb1f42a640 1 -- 172.21.15.59:0/2362972291 shutdown_connections 2024-05-08T05:34:56.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.720+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbb040b2a10 0x7fbb040b4ed0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.720+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbb181078c0 0x7fbb1819c910 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.720+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbb181022c0 0x7fbb181985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.720+0000 7fbb1f42a640 1 --2- 172.21.15.59:0/2362972291 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbb18101900 0x7fbb181980a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:34:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.720+0000 7fbb1f42a640 1 -- 172.21.15.59:0/2362972291 >> 172.21.15.59:0/2362972291 conn(0x7fbb18078a00 msgr2=0x7fbb181044c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:34:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.720+0000 7fbb1f42a640 1 -- 172.21.15.59:0/2362972291 shutdown_connections 2024-05-08T05:34:56.721 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:34:56.720+0000 7fbb1f42a640 1 -- 172.21.15.59:0/2362972291 wait complete. 2024-05-08T05:34:57.027 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0x9ff773 2024-05-08T05:34:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:57 smithi143 ceph-mon[34337]: pgmap v164: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.7 MiB/s wr, 666 op/s 2024-05-08T05:34:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:57 smithi096 ceph-mon[34253]: pgmap v164: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.7 MiB/s wr, 666 op/s 2024-05-08T05:34:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:57 smithi059 ceph-mon[30304]: pgmap v164: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 3.7 MiB/s wr, 666 op/s 2024-05-08T05:34:57.686 INFO:tasks.workunit.client.0.smithi059.stdout:20000 punch from 0x34ddcb to 0x35324f, (0x5484 bytes) 2024-05-08T05:34:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:34:59 smithi143 ceph-mon[34337]: pgmap v165: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.5 MiB/s wr, 426 op/s 2024-05-08T05:34:59.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:34:59 smithi096 ceph-mon[34253]: pgmap v165: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.5 MiB/s wr, 426 op/s 2024-05-08T05:34:59.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:34:59 smithi059 ceph-mon[30304]: pgmap v165: 129 pgs: 129 active+clean; 47 MiB data, 1.1 GiB used, 1.0 TiB / 1.0 TiB avail; 6.8 MiB/s rd, 2.5 MiB/s wr, 426 op/s 2024-05-08T05:35:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:01 smithi143 ceph-mon[34337]: pgmap v166: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 629 op/s 2024-05-08T05:35:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:01 smithi096 ceph-mon[34253]: pgmap v166: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 629 op/s 2024-05-08T05:35:01.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:01 smithi059 ceph-mon[30304]: pgmap v166: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 629 op/s 2024-05-08T05:35:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:03 smithi143 ceph-mon[34337]: pgmap v167: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-05-08T05:35:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:03 smithi096 ceph-mon[34253]: pgmap v167: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-05-08T05:35:03.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:03 smithi059 ceph-mon[30304]: pgmap v167: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-05-08T05:35:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:05 smithi143 ceph-mon[34337]: pgmap v168: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-05-08T05:35:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:05 smithi096 ceph-mon[34253]: pgmap v168: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-05-08T05:35:05.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:05 smithi059 ceph-mon[30304]: pgmap v168: 129 pgs: 129 active+clean; 44 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 2.3 MiB/s wr, 411 op/s 2024-05-08T05:35:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:07 smithi143 ceph-mon[34337]: pgmap v169: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 629 op/s 2024-05-08T05:35:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:07 smithi096 ceph-mon[34253]: pgmap v169: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 629 op/s 2024-05-08T05:35:07.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:07 smithi059 ceph-mon[30304]: pgmap v169: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 3.6 MiB/s wr, 629 op/s 2024-05-08T05:35:09.199 INFO:tasks.workunit.client.0.smithi059.stdout:30000 trunc from 0x7a5444 to 0x143a5 2024-05-08T05:35:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:09 smithi143 ceph-mon[34337]: pgmap v170: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-05-08T05:35:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:09 smithi096 ceph-mon[34253]: pgmap v170: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-05-08T05:35:09.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:09 smithi059 ceph-mon[30304]: pgmap v170: 129 pgs: 129 active+clean; 51 MiB data, 1.2 GiB used, 1.0 TiB / 1.0 TiB avail; 4.9 MiB/s rd, 2.4 MiB/s wr, 421 op/s 2024-05-08T05:35:09.834 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.833+0000 7fc0b1b32640 1 -- 172.21.15.59:0/145076060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac10d010 msgr2=0x7fc0ac10f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.833+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/145076060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac10d010 0x7fc0ac10f440 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009ca0 tx=0x7fc0a402a950 comp rx=0 tx=0).stop 2024-05-08T05:35:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.834+0000 7fc0b1b32640 1 -- 172.21.15.59:0/145076060 shutdown_connections 2024-05-08T05:35:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.834+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/145076060 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac10d010 0x7fc0ac10f440 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.834+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/145076060 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac104560 0x7fc0ac10ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.834+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/145076060 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ac103c40 0x7fc0ac104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.834+0000 7fc0b1b32640 1 -- 172.21.15.59:0/145076060 >> 172.21.15.59:0/145076060 conn(0x7fc0ac0789d0 msgr2=0x7fc0ac078de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.834+0000 7fc0b1b32640 1 -- 172.21.15.59:0/145076060 shutdown_connections 2024-05-08T05:35:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.834+0000 7fc0b1b32640 1 -- 172.21.15.59:0/145076060 wait complete. 2024-05-08T05:35:09.835 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.834+0000 7fc0b1b32640 1 Processor -- start 2024-05-08T05:35:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0b1b32640 1 -- start start 2024-05-08T05:35:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0b1b32640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac103c40 0x7fc0ac101890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0b1b32640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac104560 0x7fc0ac101dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0b1b32640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ac10d010 0x7fc0ac102310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0b1b32640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac112c50 con 0x7fc0ac104560 2024-05-08T05:35:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0b1b32640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac112ad0 con 0x7fc0ac103c40 2024-05-08T05:35:09.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0b1b32640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac112dd0 con 0x7fc0ac10d010 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0aaffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac104560 0x7fc0ac101dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac103c40 0x7fc0ac101890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0aaffd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac104560 0x7fc0ac101dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56238/0 (socket says 172.21.15.59:56238) 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac103c40 0x7fc0ac101890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:43978/0 (socket says 172.21.15.59:43978) 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0aaffd640 1 -- 172.21.15.59:0/2654755333 learned_addr learned my addr 172.21.15.59:0/2654755333 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0abfff640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ac10d010 0x7fc0ac102310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0ab7fe640 1 -- 172.21.15.59:0/2654755333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ac10d010 msgr2=0x7fc0ac102310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0ab7fe640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ac10d010 0x7fc0ac102310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0ab7fe640 1 -- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac104560 msgr2=0x7fc0ac101dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0ab7fe640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac104560 0x7fc0ac101dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0ab7fe640 1 -- 172.21.15.59:0/2654755333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0ac1a75b0 con 0x7fc0ac103c40 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.835+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac104560 0x7fc0ac101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:35:09.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.836+0000 7fc0ab7fe640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac103c40 0x7fc0ac101890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00e9f0 tx=0x7fc09c00eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.836+0000 7fc0a8ff9640 1 -- 172.21.15.59:0/2654755333 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c00cd80 con 0x7fc0ac103c40 2024-05-08T05:35:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.836+0000 7fc0b1b32640 1 -- 172.21.15.59:0/2654755333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0ac1a78a0 con 0x7fc0ac103c40 2024-05-08T05:35:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.836+0000 7fc0b1b32640 1 -- 172.21.15.59:0/2654755333 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0ac1a7e40 con 0x7fc0ac103c40 2024-05-08T05:35:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.836+0000 7fc0a8ff9640 1 -- 172.21.15.59:0/2654755333 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc09c0035f0 con 0x7fc0ac103c40 2024-05-08T05:35:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.836+0000 7fc0a8ff9640 1 -- 172.21.15.59:0/2654755333 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc09c010690 con 0x7fc0ac103c40 2024-05-08T05:35:09.839 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.839+0000 7fc0b4850740 1 Processor -- start 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.839+0000 7fc0b4850740 1 -- start start 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.839+0000 7fc0a8ff9640 1 -- 172.21.15.59:0/2654755333 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc09c0037a0 con 0x7fc0ac103c40 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.839+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c64b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c65e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c65e900 0x55ed2c660cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0b4850740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed2c5b05d0 con 0x55ed2c65e900 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0b4850740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ed2c569890 con 0x55ed2c64ae50 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0b4850740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ed2c5ac590 con 0x55ed2c64b770 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c64b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c64b230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:43980/0 (socket says 172.21.15.59:43980) 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0abfff640 1 -- 172.21.15.59:0/1125005426 learned_addr learned my addr 172.21.15.59:0/1125005426 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0ab7fe640 1 --2- 172.21.15.59:0/1125005426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c65e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.841 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1125005426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c65e900 0x55ed2c660cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0abfff640 1 -- 172.21.15.59:0/1125005426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 msgr2=0x55ed2c65e3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1125005426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c65e3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0abfff640 1 -- 172.21.15.59:0/1125005426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c65e900 msgr2=0x55ed2c660cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1125005426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c65e900 0x55ed2c660cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.840+0000 7fc0abfff640 1 -- 172.21.15.59:0/1125005426 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed2c2c2210 con 0x55ed2c64ae50 2024-05-08T05:35:09.842 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.841+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1125005426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c64b230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009ca0 tx=0x7fc0a40093e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d671902c1d4184 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.841+0000 7fc0a8ff9640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc0940b2a90 0x7fc0940b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.841+0000 7fc091ffb640 1 -- 172.21.15.59:0/1125005426 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4048070 con 0x55ed2c64ae50 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.841+0000 7fc091ffb640 1 -- 172.21.15.59:0/1125005426 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a4003e20 con 0x55ed2c64ae50 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.841+0000 7fc091ffb640 1 -- 172.21.15.59:0/1125005426 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4034040 con 0x55ed2c64ae50 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.841+0000 7fc0ab7fe640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc0940b2a90 0x7fc0940b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.841+0000 7fc0a8ff9640 1 -- 172.21.15.59:0/2654755333 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc09c0e2c00 con 0x7fc0ac103c40 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 -- 172.21.15.59:0/1125005426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 msgr2=0x55ed2c64b230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1125005426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c64b230 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc0a4009ca0 tx=0x7fc0a40093e0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0ab7fe640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc0940b2a90 0x7fc0940b4f50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c6a0 tx=0x7fc09c0e66f0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:09.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 -- 172.21.15.59:0/1125005426 shutdown_connections 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1125005426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c65e900 0x55ed2c660cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1125005426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c65e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1125005426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c64b230 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 -- 172.21.15.59:0/1125005426 >> 172.21.15.59:0/1125005426 conn(0x55ed2c647420 msgr2=0x55ed2c647800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 -- 172.21.15.59:0/1125005426 shutdown_connections 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 -- 172.21.15.59:0/1125005426 wait complete. 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.842+0000 7fc0b4850740 1 -- start start 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0b4850740 1 Processor -- start 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c65b870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64b770 0x55ed2c65bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c65e900 0x55ed2c65c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0b4850740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55ed2c661470 con 0x55ed2c64b770 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0b4850740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55ed2c56d370 con 0x55ed2c64ae50 2024-05-08T05:35:09.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0b4850740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55ed2c6615f0 con 0x55ed2c65e900 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0b4850740 10 client.0 _ll_register_callbacks cb 0x55ed2c2c1a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c65b870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c65b870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:43986/0 (socket says 172.21.15.59:43986) 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0abfff640 1 -- 172.21.15.59:0/1729617907 learned_addr learned my addr 172.21.15.59:0/1729617907 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c65e900 0x55ed2c65c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64b770 0x55ed2c65bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0abfff640 1 -- 172.21.15.59:0/1729617907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c65e900 msgr2=0x55ed2c65c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c65e900 0x55ed2c65c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0abfff640 1 -- 172.21.15.59:0/1729617907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64b770 msgr2=0x55ed2c65bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64b770 0x55ed2c65bdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.843+0000 7fc0abfff640 1 -- 172.21.15.59:0/1729617907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed2c38e620 con 0x55ed2c64ae50 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c65b870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0a402f4a0 tx=0x7fc0a402fc40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e2164383beff888e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc08affd640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc08affd640 1 -- 172.21.15.59:0/1729617907 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4048040 con 0x55ed2c64ae50 2024-05-08T05:35:09.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc08affd640 1 -- 172.21.15.59:0/1729617907 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc0a4004000 con 0x55ed2c64ae50 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc0b4850740 1 -- 172.21.15.59:0/1729617907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a4003e20 con 0x55ed2c64ae50 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc08affd640 1 -- 172.21.15.59:0/1729617907 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc0a4058050 con 0x55ed2c64ae50 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc0b4850740 10 client.25742 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc0b4850740 1 -- 172.21.15.59:0/1729617907 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ed2c66d3d0 con 0x55ed2c64ae50 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc08affd640 1 -- 172.21.15.59:0/1729617907 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a4003e20 con 0x55ed2c64ae50 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc08affd640 1 -- 172.21.15.59:0/1729617907 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc0a4030740 con 0x55ed2c64ae50 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.844+0000 7fc0b4850740 10 client.25742 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.845+0000 7fc0b4850740 10 client.25742 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.845+0000 7fc0b4850740 20 client.25742 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.845+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ed2c66e720 0x55ed2c670b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.845+0000 7fc0b4850740 4 client.25742 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.845+0000 7fc0b4850740 1 -- 172.21.15.59:0/1729617907 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed2c5be1e0 con 0x55ed2c66e720 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.845+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ed2c66e720 0x55ed2c670b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.845+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ed2c66e720 0x55ed2c670b10 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4382893a4ff5a49c server_cookie=722aa8add4a789a6 in_seq=0 out_seq=0 2024-05-08T05:35:09.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.845+0000 7fc08affd640 10 client.25742 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.846+0000 7fc08affd640 1 -- 172.21.15.59:0/1729617907 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55ed2c5be1e0 con 0x55ed2c66e720 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.846+0000 7fc08affd640 10 client.25742 handle_command_reply: tid=0 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.846+0000 7fc0b4850740 1 client.25742 shutdown 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 -- 172.21.15.59:0/1729617907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 msgr2=0x55ed2c65b870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c65b870 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc0a402f4a0 tx=0x7fc0a402fc40 comp rx=0 tx=0).stop 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 -- 172.21.15.59:0/1729617907 shutdown_connections 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ed2c66e720 0x55ed2c670b10 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c65e900 0x55ed2c65c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64b770 0x55ed2c65bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc08affd640 0 client.25742 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:09.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1729617907 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c64ae50 0x55ed2c65b870 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 -- 172.21.15.59:0/1729617907 >> 172.21.15.59:0/1729617907 conn(0x55ed2c647420 msgr2=0x55ed2c64d130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 -- 172.21.15.59:0/1729617907 shutdown_connections 2024-05-08T05:35:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 1 -- 172.21.15.59:0/1729617907 wait complete. 2024-05-08T05:35:09.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.847+0000 7fc0b4850740 20 client.25742 trim_cache size 0 max 16384 2024-05-08T05:35:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0b4850740 1 Processor -- start 2024-05-08T05:35:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0b4850740 1 -- start start 2024-05-08T05:35:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c65d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c658b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.869 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 0x55ed2c659070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0b4850740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1ada00 con 0x55ed2c64ae50 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0b4850740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1af540 con 0x55ed2c65e900 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0b4850740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1ae0d0 con 0x55ed2c64b770 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0abfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c65d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0abfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c65d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56254/0 (socket says 172.21.15.59:56254) 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0abfff640 1 -- 172.21.15.59:0/1344793035 learned_addr learned my addr 172.21.15.59:0/1344793035 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c658b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.868+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 0x55ed2c659070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0aaffd640 1 -- 172.21.15.59:0/1344793035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 msgr2=0x55ed2c659070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 0x55ed2c659070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0aaffd640 1 -- 172.21.15.59:0/1344793035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 msgr2=0x55ed2c65d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c65d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0aaffd640 1 -- 172.21.15.59:0/1344793035 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed2c659e00 con 0x55ed2c64b770 2024-05-08T05:35:09.870 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c65d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:35:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c658b30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55ed2c6616f0 tx=0x7fc0a0005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fd087a5ffb16556f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0917fa640 1 -- 172.21.15.59:0/1344793035 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0004310 con 0x55ed2c64b770 2024-05-08T05:35:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0917fa640 1 -- 172.21.15.59:0/1344793035 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a001ee50 con 0x55ed2c64b770 2024-05-08T05:35:09.871 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.869+0000 7fc0917fa640 1 -- 172.21.15.59:0/1344793035 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a0005620 con 0x55ed2c64b770 2024-05-08T05:35:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 -- 172.21.15.59:0/1344793035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 msgr2=0x55ed2c658b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c658b30 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55ed2c6616f0 tx=0x7fc0a0005f90 comp rx=0 tx=0).stop 2024-05-08T05:35:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 -- 172.21.15.59:0/1344793035 shutdown_connections 2024-05-08T05:35:09.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 0x55ed2c659070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c658b30 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1344793035 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c65d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 -- 172.21.15.59:0/1344793035 >> 172.21.15.59:0/1344793035 conn(0x55ed2c647420 msgr2=0x55ed2c65d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 -- 172.21.15.59:0/1344793035 shutdown_connections 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 -- 172.21.15.59:0/1344793035 wait complete. 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 -- start start 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.870+0000 7fc0b4850740 1 Processor -- start 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c674190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c672530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0b4850740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 0x55ed2c672bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0b4850740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1aee70 con 0x55ed2c64ae50 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0b4850740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1ae7a0 con 0x55ed2c65e900 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0b4850740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc0ac1ab9c0 con 0x55ed2c64b770 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0b4850740 10 client.0 _ll_register_callbacks cb 0x7fc0a4046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:09.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c674190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0aaffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c672530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c674190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56266/0 (socket says 172.21.15.59:56266) 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 -- 172.21.15.59:0/1972113887 learned_addr learned my addr 172.21.15.59:0/1972113887 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 0x55ed2c672bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 -- 172.21.15.59:0/1972113887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 msgr2=0x55ed2c672530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c672530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 -- 172.21.15.59:0/1972113887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 msgr2=0x55ed2c672bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 0x55ed2c672bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 -- 172.21.15.59:0/1972113887 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed2c6d8c50 con 0x55ed2c64ae50 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c672530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc0abfff640 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c674190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc0a402ae70 tx=0x7fc0a402f4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7fa3f10849bcd121 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.871+0000 7fc08affd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc08affd640 1 -- 172.21.15.59:0/1972113887 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc0a4048070 con 0x55ed2c64ae50 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc08affd640 1 -- 172.21.15.59:0/1972113887 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc0a403a430 con 0x55ed2c64ae50 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc0b4850740 1 -- 172.21.15.59:0/1972113887 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc0a001ee50 con 0x55ed2c64ae50 2024-05-08T05:35:09.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc08affd640 1 -- 172.21.15.59:0/1972113887 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc0a4034040 con 0x55ed2c64ae50 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc0b4850740 10 client.16146 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc0b4850740 1 -- 172.21.15.59:0/1972113887 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55ed2c6d9dd0 con 0x55ed2c64ae50 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc08affd640 1 -- 172.21.15.59:0/1972113887 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc0a001ee50 con 0x55ed2c64ae50 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc08affd640 1 -- 172.21.15.59:0/1972113887 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc0a403ab10 con 0x55ed2c64ae50 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc0b4850740 10 client.16146 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc0b4850740 10 client.16146 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc0b4850740 20 client.16146 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.872+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ed2c6da200 0x55ed2c6dc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.873+0000 7fc0b4850740 4 client.16146 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.873+0000 7fc0b4850740 1 -- 172.21.15.59:0/1972113887 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed2c5b8050 con 0x55ed2c6da200 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.873+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ed2c6da200 0x55ed2c6dc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.873+0000 7fc08affd640 10 client.16146 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:09.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.873+0000 7fc0aaffd640 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ed2c6da200 0x55ed2c6dc620 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c48f95d84b34c31 server_cookie=e0db56b9bccbb6e7 in_seq=0 out_seq=0 2024-05-08T05:35:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.888+0000 7fc08affd640 1 -- 172.21.15.59:0/1972113887 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed2c5b8050 con 0x55ed2c6da200 2024-05-08T05:35:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.888+0000 7fc08affd640 10 client.16146 handle_command_reply: tid=0 2024-05-08T05:35:09.888 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.888+0000 7fc0b4850740 1 client.16146 shutdown 2024-05-08T05:35:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.889+0000 7fc0b4850740 1 -- 172.21.15.59:0/1972113887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 msgr2=0x55ed2c674190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:09.889 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.889+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c674190 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fc0a402ae70 tx=0x7fc0a402f4d0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc0b4850740 1 -- 172.21.15.59:0/1972113887 shutdown_connections 2024-05-08T05:35:09.890 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55ed2c6da200 0x55ed2c6dc620 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55ed2c65e900 0x55ed2c672bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc08affd640 0 client.16146 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55ed2c64b770 0x55ed2c672530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc0b4850740 1 --2- 172.21.15.59:0/1972113887 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55ed2c64ae50 0x55ed2c674190 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc0b4850740 1 -- 172.21.15.59:0/1972113887 >> 172.21.15.59:0/1972113887 conn(0x55ed2c647420 msgr2=0x55ed2c65d930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc0b4850740 1 -- 172.21.15.59:0/1972113887 shutdown_connections 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc0b4850740 1 -- 172.21.15.59:0/1972113887 wait complete. 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.890+0000 7fc0b4850740 20 client.16146 trim_cache size 0 max 16384 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:35:09.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:35:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:35:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:35:09.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 -- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc0940b2a90 msgr2=0x7fc0940b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc0940b2a90 0x7fc0940b4f50 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00c6a0 tx=0x7fc09c0e66f0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 -- 172.21.15.59:0/2654755333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac103c40 msgr2=0x7fc0ac101890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac103c40 0x7fc0ac101890 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fc09c00e9f0 tx=0x7fc09c00eeb0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 -- 172.21.15.59:0/2654755333 shutdown_connections 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc0940b2a90 0x7fc0940b4f50 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc0ac10d010 0x7fc0ac102310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc0ac104560 0x7fc0ac101dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 --2- 172.21.15.59:0/2654755333 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc0ac103c40 0x7fc0ac101890 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 -- 172.21.15.59:0/2654755333 >> 172.21.15.59:0/2654755333 conn(0x7fc0ac0789d0 msgr2=0x7fc0ac0fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 -- 172.21.15.59:0/2654755333 shutdown_connections 2024-05-08T05:35:09.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:09.891+0000 7fc0b1b32640 1 -- 172.21.15.59:0/2654755333 wait complete. 2024-05-08T05:35:11.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:11 smithi143 ceph-mon[34337]: pgmap v171: 129 pgs: 129 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.6 MiB/s wr, 613 op/s 2024-05-08T05:35:11.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:11 smithi096 ceph-mon[34253]: pgmap v171: 129 pgs: 129 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.6 MiB/s wr, 613 op/s 2024-05-08T05:35:11.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:11 smithi059 ceph-mon[30304]: pgmap v171: 129 pgs: 129 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.8 MiB/s rd, 3.6 MiB/s wr, 613 op/s 2024-05-08T05:35:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:13 smithi143 ceph-mon[34337]: pgmap v172: 129 pgs: 129 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 409 op/s 2024-05-08T05:35:13.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:13 smithi096 ceph-mon[34253]: pgmap v172: 129 pgs: 129 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 409 op/s 2024-05-08T05:35:13.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:13 smithi059 ceph-mon[30304]: pgmap v172: 129 pgs: 129 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 409 op/s 2024-05-08T05:35:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:15 smithi143 ceph-mon[34337]: pgmap v173: 129 pgs: 129 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 409 op/s 2024-05-08T05:35:15.525 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0x9ffb01 2024-05-08T05:35:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:15 smithi096 ceph-mon[34253]: pgmap v173: 129 pgs: 129 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 409 op/s 2024-05-08T05:35:15.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:15 smithi059 ceph-mon[30304]: pgmap v173: 129 pgs: 129 active+clean; 58 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 5.6 MiB/s rd, 2.4 MiB/s wr, 409 op/s 2024-05-08T05:35:17.142 INFO:tasks.quiescer.fs.[cephfs]:Total duration (51.4) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-08T05:35:17.142 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.7 2024-05-08T05:35:17.143 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:35:17.460 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:35:17.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:17 smithi143 ceph-mon[34337]: pgmap v174: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 594 op/s 2024-05-08T05:35:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:17 smithi096 ceph-mon[34253]: pgmap v174: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 594 op/s 2024-05-08T05:35:17.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:17 smithi059 ceph-mon[30304]: pgmap v174: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 3.6 MiB/s wr, 594 op/s 2024-05-08T05:35:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:19 smithi143 ceph-mon[34337]: pgmap v175: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 376 op/s 2024-05-08T05:35:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:19 smithi096 ceph-mon[34253]: pgmap v175: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 376 op/s 2024-05-08T05:35:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:19 smithi059 ceph-mon[30304]: pgmap v175: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 MiB/s rd, 2.4 MiB/s wr, 376 op/s 2024-05-08T05:35:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:21 smithi059 ceph-mon[30304]: pgmap v176: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 548 op/s 2024-05-08T05:35:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:21 smithi143 ceph-mon[34337]: pgmap v176: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 548 op/s 2024-05-08T05:35:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:21 smithi096 ceph-mon[34253]: pgmap v176: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 3.4 MiB/s wr, 548 op/s 2024-05-08T05:35:22.074 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.074+0000 7f435c277700 1 -- 172.21.15.59:0/300460968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43541023a0 msgr2=0x7f4354102820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.074+0000 7f435c277700 1 --2- 172.21.15.59:0/300460968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43541023a0 0x7f4354102820 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4348009990 tx=0x7f4348009ca0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.074+0000 7f435c277700 1 -- 172.21.15.59:0/300460968 shutdown_connections 2024-05-08T05:35:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.074+0000 7f435c277700 1 --2- 172.21.15.59:0/300460968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4354102d60 0x7f435410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.074+0000 7f435c277700 1 --2- 172.21.15.59:0/300460968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f4354108410 0x7f43541087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.074+0000 7f435c277700 1 --2- 172.21.15.59:0/300460968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f43541023a0 0x7f4354102820 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.075 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.074+0000 7f435c277700 1 -- 172.21.15.59:0/300460968 >> 172.21.15.59:0/300460968 conn(0x7f4354079350 msgr2=0x7f4354079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.076+0000 7f435c277700 1 -- 172.21.15.59:0/300460968 shutdown_connections 2024-05-08T05:35:22.077 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.076+0000 7f435c277700 1 -- 172.21.15.59:0/300460968 wait complete. 2024-05-08T05:35:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.078+0000 7f435c277700 1 Processor -- start 2024-05-08T05:35:22.078 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.078+0000 7f435c277700 1 -- start start 2024-05-08T05:35:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435c277700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43541023a0 0x7f43541977f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435c277700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4354102d60 0x7f4354197d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435c277700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4354108410 0x7f4354198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435c277700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f43541114f0 con 0x7f4354108410 2024-05-08T05:35:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435c277700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4354111370 con 0x7f43541023a0 2024-05-08T05:35:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435c277700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f4354111670 con 0x7f4354102d60 2024-05-08T05:35:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435a814700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4354108410 0x7f4354198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:22.080 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435a013700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43541023a0 0x7f43541977f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435a013700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43541023a0 0x7f43541977f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:39592/0 (socket says 172.21.15.59:39592) 2024-05-08T05:35:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435a013700 1 -- 172.21.15.59:0/2312069047 learned_addr learned my addr 172.21.15.59:0/2312069047 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f4359812700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4354102d60 0x7f4354197d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435a814700 1 -- 172.21.15.59:0/2312069047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4354102d60 msgr2=0x7f4354197d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.079+0000 7f435a814700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4354102d60 0x7f4354197d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.080+0000 7f435a814700 1 -- 172.21.15.59:0/2312069047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43541023a0 msgr2=0x7f43541977f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.080+0000 7f435a814700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43541023a0 0x7f43541977f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.080+0000 7f435a814700 1 -- 172.21.15.59:0/2312069047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43541ac460 con 0x7f4354108410 2024-05-08T05:35:22.081 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.080+0000 7f435a814700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4354108410 0x7f4354198270 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f434c009f60 tx=0x7f434c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.080+0000 7f43477fe700 1 -- 172.21.15.59:0/2312069047 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c0092e0 con 0x7f4354108410 2024-05-08T05:35:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.080+0000 7f43477fe700 1 -- 172.21.15.59:0/2312069047 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f434c004bb0 con 0x7f4354108410 2024-05-08T05:35:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.080+0000 7f43477fe700 1 -- 172.21.15.59:0/2312069047 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f434c004280 con 0x7f4354108410 2024-05-08T05:35:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.080+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43541ac6f0 con 0x7f4354108410 2024-05-08T05:35:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.080+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43541acc30 con 0x7f4354108410 2024-05-08T05:35:22.082 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.082+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4354103ac0 con 0x7f4354108410 2024-05-08T05:35:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.084+0000 7f43477fe700 1 -- 172.21.15.59:0/2312069047 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f434c01b030 con 0x7f4354108410 2024-05-08T05:35:22.086 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.085+0000 7f43477fe700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43400b2aa0 0x7f43400b4f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:22.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.085+0000 7f43477fe700 1 -- 172.21.15.59:0/2312069047 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f434c0e2d30 con 0x7f4354108410 2024-05-08T05:35:22.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.085+0000 7f435a013700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43400b2aa0 0x7f43400b4f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:22.087 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.086+0000 7f435a013700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43400b2aa0 0x7f43400b4f60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f435000b3c0 tx=0x7f435000f040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:22.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.090+0000 7f43477fe700 1 -- 172.21.15.59:0/2312069047 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f434c022020 con 0x7f4354108410 2024-05-08T05:35:22.107 INFO:tasks.workunit.client.0.smithi059.stdout:40000 mapwrite 0x1becad thru 0x1c6a92 (0x7de6 bytes) 2024-05-08T05:35:22.543 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.542+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4354198cd0 con 0x7f4354108410 2024-05-08T05:35:22.544 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.543+0000 7f43477fe700 1 -- 172.21.15.59:0/2312069047 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11929 (secure 0 0 0) 0x7f434c010040 con 0x7f4354108410 2024-05-08T05:35:22.544 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:35:22.544 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:34:26.942121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24443,24473,24421,24433,24419]},"id":1}]} 2024-05-08T05:35:22.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43400b2aa0 msgr2=0x7f43400b4f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:22.547 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43400b2aa0 0x7f43400b4f60 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f435000b3c0 tx=0x7f435000f040 comp rx=0 tx=0).stop 2024-05-08T05:35:22.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4354108410 msgr2=0x7f4354198270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:22.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4354108410 0x7f4354198270 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f434c009f60 tx=0x7f434c009f90 comp rx=0 tx=0).stop 2024-05-08T05:35:22.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 shutdown_connections 2024-05-08T05:35:22.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f43400b2aa0 0x7f43400b4f60 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f4354108410 0x7f4354198270 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.548 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f43541023a0 0x7f43541977f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 --2- 172.21.15.59:0/2312069047 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f4354102d60 0x7f4354197d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.547+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 >> 172.21.15.59:0/2312069047 conn(0x7f4354079350 msgr2=0x7f4354100ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.548+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 shutdown_connections 2024-05-08T05:35:22.549 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:22.548+0000 7f435c277700 1 -- 172.21.15.59:0/2312069047 wait complete. 2024-05-08T05:35:22.550 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 36 2024-05-08T05:35:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.017+0000 7fc350a1d640 1 -- 172.21.15.59:0/3182968966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c10d000 msgr2=0x7fc34c10f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:23.018 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.018+0000 7fc350a1d640 1 --2- 172.21.15.59:0/3182968966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c10d000 0x7fc34c10f430 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fc344009ca0 tx=0x7fc34402a950 comp rx=0 tx=0).stop 2024-05-08T05:35:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.018+0000 7fc350a1d640 1 -- 172.21.15.59:0/3182968966 shutdown_connections 2024-05-08T05:35:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.018+0000 7fc350a1d640 1 --2- 172.21.15.59:0/3182968966 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c10d000 0x7fc34c10f430 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.018+0000 7fc350a1d640 1 --2- 172.21.15.59:0/3182968966 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc34c104550 0x7fc34c10ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.018+0000 7fc350a1d640 1 --2- 172.21.15.59:0/3182968966 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c103c30 0x7fc34c104010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.018+0000 7fc350a1d640 1 -- 172.21.15.59:0/3182968966 >> 172.21.15.59:0/3182968966 conn(0x7fc34c078a00 msgr2=0x7fc34c078e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.018+0000 7fc350a1d640 1 -- 172.21.15.59:0/3182968966 shutdown_connections 2024-05-08T05:35:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.018+0000 7fc350a1d640 1 -- 172.21.15.59:0/3182968966 wait complete. 2024-05-08T05:35:23.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.019+0000 7fc350a1d640 1 Processor -- start 2024-05-08T05:35:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.019+0000 7fc350a1d640 1 -- start start 2024-05-08T05:35:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc350a1d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c103c30 0x7fc34c196f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc350a1d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c104550 0x7fc34c197440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc350a1d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc34c10d000 0x7fc34c19b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc350a1d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c110a80 con 0x7fc34c104550 2024-05-08T05:35:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc350a1d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c110900 con 0x7fc34c103c30 2024-05-08T05:35:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc350a1d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c110c00 con 0x7fc34c10d000 2024-05-08T05:35:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc349d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c104550 0x7fc34c197440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc349d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c104550 0x7fc34c197440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54024/0 (socket says 172.21.15.59:54024) 2024-05-08T05:35:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc349d9b640 1 -- 172.21.15.59:0/1245564399 learned_addr learned my addr 172.21.15.59:0/1245564399 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:23.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.020+0000 7fc34ad9d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc34c10d000 0x7fc34c19b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.021+0000 7fc34ad9d640 1 -- 172.21.15.59:0/1245564399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c103c30 msgr2=0x7fc34c196f00 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-05-08T05:35:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.021+0000 7fc34ad9d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c103c30 0x7fc34c196f00 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.022 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.021+0000 7fc34ad9d640 1 -- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c104550 msgr2=0x7fc34c197440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.021+0000 7fc34ad9d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c104550 0x7fc34c197440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.021+0000 7fc34ad9d640 1 -- 172.21.15.59:0/1245564399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc34c19bea0 con 0x7fc34c10d000 2024-05-08T05:35:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.021+0000 7fc349d9b640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c104550 0x7fc34c197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:35:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.021+0000 7fc34ad9d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc34c10d000 0x7fc34c19b770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc344005e30 tx=0x7fc344003c10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.021+0000 7fc33b7fe640 1 -- 172.21.15.59:0/1245564399 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344034040 con 0x7fc34c10d000 2024-05-08T05:35:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.021+0000 7fc33b7fe640 1 -- 172.21.15.59:0/1245564399 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc34402fd20 con 0x7fc34c10d000 2024-05-08T05:35:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.022+0000 7fc33b7fe640 1 -- 172.21.15.59:0/1245564399 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc344004540 con 0x7fc34c10d000 2024-05-08T05:35:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.022+0000 7fc350a1d640 1 -- 172.21.15.59:0/1245564399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc34c19c130 con 0x7fc34c10d000 2024-05-08T05:35:23.023 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.022+0000 7fc350a1d640 1 -- 172.21.15.59:0/1245564399 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc34c19c5a0 con 0x7fc34c10d000 2024-05-08T05:35:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.025+0000 7fc33b7fe640 1 -- 172.21.15.59:0/1245564399 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc344037020 con 0x7fc34c10d000 2024-05-08T05:35:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.025+0000 7fc353810740 1 Processor -- start 2024-05-08T05:35:23.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.025+0000 7fc353810740 1 -- start start 2024-05-08T05:35:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.025+0000 7fc353810740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61cc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.025+0000 7fc353810740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 0x55c2f61d6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.025+0000 7fc353810740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d94e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.025+0000 7fc353810740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c2f61315d0 con 0x55c2f61cbf10 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.025+0000 7fc353810740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c2f60ea890 con 0x55c2f61cc830 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.025+0000 7fc353810740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c2f612d590 con 0x55c2f61d70f0 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61cc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61cc2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54038/0 (socket says 172.21.15.59:54038) 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 -- 172.21.15.59:0/873383171 learned_addr learned my addr 172.21.15.59:0/873383171 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 --2- 172.21.15.59:0/873383171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 -- 172.21.15.59:0/873383171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 msgr2=0x55c2f61d94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 --2- 172.21.15.59:0/873383171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d94e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 -- 172.21.15.59:0/873383171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 msgr2=0x55c2f61d6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 --2- 172.21.15.59:0/873383171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 0x55c2f61d6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 -- 172.21.15.59:0/873383171 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f5e43210 con 0x55c2f61cbf10 2024-05-08T05:35:23.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.026+0000 7fc349d9b640 1 --2- 172.21.15.59:0/873383171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61cc2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55c2f61c95b0 tx=0x7fc3400095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3abbae47846fb6b6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.027+0000 7fc338ff9640 1 -- 172.21.15.59:0/873383171 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340013d20 con 0x55c2f61cbf10 2024-05-08T05:35:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.027+0000 7fc338ff9640 1 -- 172.21.15.59:0/873383171 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc340013170 con 0x55c2f61cbf10 2024-05-08T05:35:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.027+0000 7fc338ff9640 1 -- 172.21.15.59:0/873383171 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340010c90 con 0x55c2f61cbf10 2024-05-08T05:35:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.027+0000 7fc33b7fe640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc3340b6ea0 0x7fc3340b9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.027+0000 7fc34a59c640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc3340b6ea0 0x7fc3340b9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.029 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.027+0000 7fc33b7fe640 1 -- 172.21.15.59:0/1245564399 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc3441035a0 con 0x7fc34c10d000 2024-05-08T05:35:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.027+0000 7fc353810740 1 -- 172.21.15.59:0/873383171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 msgr2=0x55c2f61cc2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc353810740 1 --2- 172.21.15.59:0/873383171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61cc2f0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x55c2f61c95b0 tx=0x7fc3400095b0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc353810740 1 -- 172.21.15.59:0/873383171 shutdown_connections 2024-05-08T05:35:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc353810740 1 --2- 172.21.15.59:0/873383171 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc353810740 1 --2- 172.21.15.59:0/873383171 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 0x55c2f61d6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc353810740 1 --2- 172.21.15.59:0/873383171 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61cc2f0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc353810740 1 -- 172.21.15.59:0/873383171 >> 172.21.15.59:0/873383171 conn(0x55c2f61c8420 msgr2=0x55c2f61c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc34a59c640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc3340b6ea0 0x7fc3340b9360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0058c0 tx=0x7fc33c005850 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:23.030 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc353810740 1 -- 172.21.15.59:0/873383171 shutdown_connections 2024-05-08T05:35:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc353810740 1 -- 172.21.15.59:0/873383171 wait complete. 2024-05-08T05:35:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.028+0000 7fc353810740 1 -- start start 2024-05-08T05:35:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc353810740 1 Processor -- start 2024-05-08T05:35:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc353810740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cbf10 0x55c2f61d40b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc353810740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cc830 0x55c2f61d45f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc353810740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d4b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc353810740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55c2f61d9d70 con 0x55c2f61cc830 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc353810740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55c2f60ee370 con 0x55c2f61cbf10 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc353810740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55c2f61d9ef0 con 0x55c2f61d70f0 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc34ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d4b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc34ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d4b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:44368/0 (socket says 172.21.15.59:44368) 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc34ad9d640 1 -- 172.21.15.59:0/1695530397 learned_addr learned my addr 172.21.15.59:0/1695530397 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.029+0000 7fc353810740 10 client.0 _ll_register_callbacks cb 0x55c2f5e42a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.030+0000 7fc34ad9d640 1 -- 172.21.15.59:0/1695530397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cbf10 msgr2=0x55c2f61d40b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.030+0000 7fc34ad9d640 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cbf10 0x55c2f61d40b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.030+0000 7fc34ad9d640 1 -- 172.21.15.59:0/1695530397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cc830 msgr2=0x55c2f61d45f0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.030+0000 7fc349d9b640 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cc830 0x55c2f61d45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.030+0000 7fc34ad9d640 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cc830 0x55c2f61d45f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.030+0000 7fc34ad9d640 1 -- 172.21.15.59:0/1695530397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f5f0f620 con 0x55c2f61d70f0 2024-05-08T05:35:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.030+0000 7fc349d9b640 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cc830 0x55c2f61d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:35:23.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.032+0000 7fc34ad9d640 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d4b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3440c66a0 tx=0x7fc3440c9ec0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1adc2d2c6ec33191 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.032+0000 7fc331ffb640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.032+0000 7fc331ffb640 1 -- 172.21.15.59:0/1695530397 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34402c040 con 0x55c2f61d70f0 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.032+0000 7fc331ffb640 1 -- 172.21.15.59:0/1695530397 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc3440c68b0 con 0x55c2f61d70f0 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.032+0000 7fc353810740 1 -- 172.21.15.59:0/1695530397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc340013170 con 0x55c2f61d70f0 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.032+0000 7fc331ffb640 1 -- 172.21.15.59:0/1695530397 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc3440ca620 con 0x55c2f61d70f0 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.032+0000 7fc353810740 10 client.25720 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.032+0000 7fc353810740 1 -- 172.21.15.59:0/1695530397 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c2f61d9e20 con 0x55c2f61d70f0 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.032+0000 7fc331ffb640 1 -- 172.21.15.59:0/1695530397 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc340013170 con 0x55c2f61d70f0 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.033+0000 7fc331ffb640 1 -- 172.21.15.59:0/1695530397 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc3440d2ff0 con 0x55c2f61d70f0 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.033+0000 7fc353810740 10 client.25720 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.033+0000 7fc353810740 10 client.25720 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:35:23.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.033+0000 7fc353810740 20 client.25720 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.033+0000 7fc353810740 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55c2f61e5960 0x55c2f61e7de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.033+0000 7fc353810740 4 client.25720 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:35:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.033+0000 7fc353810740 1 -- 172.21.15.59:0/1695530397 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2f613f1e0 con 0x55c2f61e5960 2024-05-08T05:35:23.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.033+0000 7fc34a59c640 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55c2f61e5960 0x55c2f61e7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.034+0000 7fc34a59c640 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55c2f61e5960 0x55c2f61e7de0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9f4d263606dd5e34 server_cookie=c4a5062b49df23a0 in_seq=0 out_seq=0 2024-05-08T05:35:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.034+0000 7fc331ffb640 10 client.25720 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.034+0000 7fc331ffb640 1 -- 172.21.15.59:0/1695530397 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55c2f613f1e0 con 0x55c2f61e5960 2024-05-08T05:35:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.034+0000 7fc331ffb640 10 client.25720 handle_command_reply: tid=0 2024-05-08T05:35:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 client.25720 shutdown 2024-05-08T05:35:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 -- 172.21.15.59:0/1695530397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 msgr2=0x55c2f61d4b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:23.036 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d4b30 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fc3440c66a0 tx=0x7fc3440c9ec0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 -- 172.21.15.59:0/1695530397 shutdown_connections 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55c2f61e5960 0x55c2f61e7de0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61d70f0 0x55c2f61d4b30 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cc830 0x55c2f61d45f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 --2- 172.21.15.59:0/1695530397 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cbf10 0x55c2f61d40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 -- 172.21.15.59:0/1695530397 >> 172.21.15.59:0/1695530397 conn(0x55c2f61c8420 msgr2=0x55c2f61ce1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc331ffb640 0 client.25720 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 -- 172.21.15.59:0/1695530397 shutdown_connections 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 1 -- 172.21.15.59:0/1695530397 wait complete. 2024-05-08T05:35:23.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.035+0000 7fc353810740 20 client.25720 trim_cache size 0 max 16384 2024-05-08T05:35:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.066+0000 7fc353810740 1 Processor -- start 2024-05-08T05:35:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.066+0000 7fc353810740 1 -- start start 2024-05-08T05:35:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.066+0000 7fc353810740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61d4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.066+0000 7fc353810740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cc830 0x55c2f61f3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.066+0000 7fc353810740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61d70f0 0x55c2f61f39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.067+0000 7fc353810740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1ad970 con 0x55c2f61cbf10 2024-05-08T05:35:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.067+0000 7fc349d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cc830 0x55c2f61f3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.067+0000 7fc349d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cc830 0x55c2f61f3460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:44374/0 (socket says 172.21.15.59:44374) 2024-05-08T05:35:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.067+0000 7fc349d9b640 1 -- 172.21.15.59:0/345857144 learned_addr learned my addr 172.21.15.59:0/345857144 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.067+0000 7fc353810740 1 -- 172.21.15.59:0/345857144 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1af4b0 con 0x55c2f61d70f0 2024-05-08T05:35:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.067+0000 7fc353810740 1 -- 172.21.15.59:0/345857144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1ae040 con 0x55c2f61cc830 2024-05-08T05:35:23.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.067+0000 7fc34a59c640 1 --2- 172.21.15.59:0/345857144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61d4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.068+0000 7fc349d9b640 1 -- 172.21.15.59:0/345857144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61d70f0 msgr2=0x55c2f61f39a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:35:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.068+0000 7fc349d9b640 1 --2- 172.21.15.59:0/345857144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61d70f0 0x55c2f61f39a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.068+0000 7fc349d9b640 1 -- 172.21.15.59:0/345857144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 msgr2=0x55c2f61d4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.068+0000 7fc349d9b640 1 --2- 172.21.15.59:0/345857144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61d4950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.068+0000 7fc349d9b640 1 -- 172.21.15.59:0/345857144 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2f61c3f70 con 0x55c2f61cc830 2024-05-08T05:35:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.068+0000 7fc34a59c640 1 --2- 172.21.15.59:0/345857144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61d4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:35:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.068+0000 7fc349d9b640 1 --2- 172.21.15.59:0/345857144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cc830 0x55c2f61f3460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc340009640 tx=0x7fc340013550 comp rx=0 tx=0).ready entity=mon.1 client_cookie=91129e2e7658f92 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.069+0000 7fc333fff640 1 -- 172.21.15.59:0/345857144 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc340013170 con 0x55c2f61cc830 2024-05-08T05:35:23.069 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.069+0000 7fc333fff640 1 -- 172.21.15.59:0/345857144 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc340013310 con 0x55c2f61cc830 2024-05-08T05:35:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 -- 172.21.15.59:0/345857144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cc830 msgr2=0x55c2f61f3460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 --2- 172.21.15.59:0/345857144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cc830 0x55c2f61f3460 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fc340009640 tx=0x7fc340013550 comp rx=0 tx=0).stop 2024-05-08T05:35:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 -- 172.21.15.59:0/345857144 shutdown_connections 2024-05-08T05:35:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 --2- 172.21.15.59:0/345857144 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61d70f0 0x55c2f61f39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 --2- 172.21.15.59:0/345857144 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cc830 0x55c2f61f3460 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 --2- 172.21.15.59:0/345857144 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61cbf10 0x55c2f61d4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.070 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 -- 172.21.15.59:0/345857144 >> 172.21.15.59:0/345857144 conn(0x55c2f61c8420 msgr2=0x55c2f61c7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 -- 172.21.15.59:0/345857144 shutdown_connections 2024-05-08T05:35:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 -- 172.21.15.59:0/345857144 wait complete. 2024-05-08T05:35:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.070+0000 7fc353810740 1 -- start start 2024-05-08T05:35:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.071+0000 7fc353810740 1 Processor -- start 2024-05-08T05:35:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.071+0000 7fc353810740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cbf10 0x55c2f61f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.071+0000 7fc353810740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 0x55c2f61f0c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.071 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.071+0000 7fc353810740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61d70f0 0x55c2f61f11b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.072 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.071+0000 7fc353810740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1aede0 con 0x55c2f61d70f0 2024-05-08T05:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc353810740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1ae710 con 0x55c2f61cc830 2024-05-08T05:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc353810740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc34c1ab930 con 0x55c2f61cbf10 2024-05-08T05:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc353810740 10 client.0 _ll_register_callbacks cb 0x7fc344005f10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:23.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc349d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 0x55c2f61f0c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc349d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 0x55c2f61f0c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:39642/0 (socket says 172.21.15.59:39642) 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc349d9b640 1 -- 172.21.15.59:0/4086027477 learned_addr learned my addr 172.21.15.59:0/4086027477 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc34ad9d640 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61d70f0 0x55c2f61f11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc34a59c640 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cbf10 0x55c2f61f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc349d9b640 1 -- 172.21.15.59:0/4086027477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cbf10 msgr2=0x55c2f61f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc349d9b640 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cbf10 0x55c2f61f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc349d9b640 1 -- 172.21.15.59:0/4086027477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61d70f0 msgr2=0x55c2f61f11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc349d9b640 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61d70f0 0x55c2f61f11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc349d9b640 1 -- 172.21.15.59:0/4086027477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2f61f1820 con 0x55c2f61cc830 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc34ad9d640 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61d70f0 0x55c2f61f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc349d9b640 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 0x55c2f61f0c70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc340016020 tx=0x7fc34000fe50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5265b2305b95aea3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:23.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.072+0000 7fc331ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.073+0000 7fc331ffb640 1 -- 172.21.15.59:0/4086027477 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc34002b020 con 0x55c2f61cc830 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.073+0000 7fc353810740 1 -- 172.21.15.59:0/4086027477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c2f6139050 con 0x55c2f61cc830 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.073+0000 7fc331ffb640 1 -- 172.21.15.59:0/4086027477 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fc340004f50 con 0x55c2f61cc830 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.073+0000 7fc331ffb640 1 -- 172.21.15.59:0/4086027477 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fc340036050 con 0x55c2f61cc830 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.073+0000 7fc353810740 10 client.25760 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.073+0000 7fc353810740 1 -- 172.21.15.59:0/4086027477 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55c2f6263000 con 0x55c2f61cc830 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.074+0000 7fc331ffb640 1 -- 172.21.15.59:0/4086027477 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c2f6139050 con 0x55c2f61cc830 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.074+0000 7fc331ffb640 1 -- 172.21.15.59:0/4086027477 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fc340010820 con 0x55c2f61cc830 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.074+0000 7fc353810740 10 client.25760 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.074+0000 7fc353810740 10 client.25760 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.074+0000 7fc353810740 20 client.25760 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.074+0000 7fc353810740 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55c2f6263610 0x55c2f6265a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.074+0000 7fc353810740 4 client.25760 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:35:23.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.074+0000 7fc353810740 1 -- 172.21.15.59:0/4086027477 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2f60b45f0 con 0x55c2f6263610 2024-05-08T05:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.075+0000 7fc34a59c640 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55c2f6263610 0x55c2f6265a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.076+0000 7fc34a59c640 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55c2f6263610 0x55c2f6265a00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4710a990919744c3 server_cookie=10f070ce6f3a9a38 in_seq=0 out_seq=0 2024-05-08T05:35:23.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.076+0000 7fc331ffb640 10 client.25760 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc331ffb640 1 -- 172.21.15.59:0/4086027477 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2f60b45f0 con 0x55c2f6263610 2024-05-08T05:35:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc331ffb640 10 client.25760 handle_command_reply: tid=0 2024-05-08T05:35:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc353810740 1 client.25760 shutdown 2024-05-08T05:35:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc353810740 1 -- 172.21.15.59:0/4086027477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 msgr2=0x55c2f61f0c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc353810740 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 0x55c2f61f0c70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc340016020 tx=0x7fc34000fe50 comp rx=0 tx=0).stop 2024-05-08T05:35:23.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc353810740 1 -- 172.21.15.59:0/4086027477 shutdown_connections 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc353810740 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55c2f6263610 0x55c2f6265a00 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc353810740 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55c2f61d70f0 0x55c2f61f11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc353810740 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55c2f61cc830 0x55c2f61f0c70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc353810740 1 --2- 172.21.15.59:0/4086027477 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55c2f61cbf10 0x55c2f61f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.090+0000 7fc353810740 1 -- 172.21.15.59:0/4086027477 >> 172.21.15.59:0/4086027477 conn(0x55c2f61c8420 msgr2=0x55c2f61cd910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.091+0000 7fc331ffb640 0 client.25760 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.091+0000 7fc353810740 1 -- 172.21.15.59:0/4086027477 shutdown_connections 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.091+0000 7fc353810740 1 -- 172.21.15.59:0/4086027477 wait complete. 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.091+0000 7fc353810740 20 client.25760 trim_cache size 0 max 16384 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:35:23.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 -- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc3340b6ea0 msgr2=0x7fc3340b9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc3340b6ea0 0x7fc3340b9360 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc33c0058c0 tx=0x7fc33c005850 comp rx=0 tx=0).stop 2024-05-08T05:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 -- 172.21.15.59:0/1245564399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc34c10d000 msgr2=0x7fc34c19b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc34c10d000 0x7fc34c19b770 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fc344005e30 tx=0x7fc344003c10 comp rx=0 tx=0).stop 2024-05-08T05:35:23.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 -- 172.21.15.59:0/1245564399 shutdown_connections 2024-05-08T05:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc3340b6ea0 0x7fc3340b9360 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc34c10d000 0x7fc34c19b770 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc34c104550 0x7fc34c197440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 --2- 172.21.15.59:0/1245564399 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc34c103c30 0x7fc34c196f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 -- 172.21.15.59:0/1245564399 >> 172.21.15.59:0/1245564399 conn(0x7fc34c078a00 msgr2=0x7fc34c0ffaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 -- 172.21.15.59:0/1245564399 shutdown_connections 2024-05-08T05:35:23.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:23.092+0000 7fc350a1d640 1 -- 172.21.15.59:0/1245564399 wait complete. 2024-05-08T05:35:23.382 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db / --timeout 60 --expiration 85.7 --await-for 42.0' 2024-05-08T05:35:23.382 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db / --timeout 60 --expiration 85.7 --await-for 42.0 2024-05-08T05:35:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:23 smithi143 ceph-mon[34337]: pgmap v177: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.2 MiB/s wr, 356 op/s 2024-05-08T05:35:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:23 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2312069047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:35:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:23 smithi096 ceph-mon[34253]: pgmap v177: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.2 MiB/s wr, 356 op/s 2024-05-08T05:35:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:23 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2312069047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:35:23.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:23 smithi059 ceph-mon[30304]: pgmap v177: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.2 MiB/s wr, 356 op/s 2024-05-08T05:35:23.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:23 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2312069047' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:35:23.717 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:35:25.452 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:25 smithi059 ceph-mon[30304]: pgmap v178: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.2 MiB/s wr, 356 op/s 2024-05-08T05:35:25.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.482+0000 7f373aae3700 1 -- 172.21.15.59:0/3353451992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37341037b0 msgr2=0x7f3734107030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/3353451992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37341037b0 0x7f3734107030 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f37200099c0 tx=0x7f3720009cd0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.482+0000 7f373aae3700 1 -- 172.21.15.59:0/3353451992 shutdown_connections 2024-05-08T05:35:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/3353451992 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3734107760 0x7f373410f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/3353451992 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102e00 0x7f37341031e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/3353451992 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f37341037b0 0x7f3734107030 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.482+0000 7f373aae3700 1 -- 172.21.15.59:0/3353451992 >> 172.21.15.59:0/3353451992 conn(0x7f3734078a40 msgr2=0x7f37340ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.482+0000 7f373aae3700 1 -- 172.21.15.59:0/3353451992 shutdown_connections 2024-05-08T05:35:25.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.482+0000 7f373aae3700 1 -- 172.21.15.59:0/3353451992 wait complete. 2024-05-08T05:35:25.486 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373aae3700 1 Processor -- start 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373aae3700 1 -- start start 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373aae3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102e00 0x7f373419e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373aae3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37341037b0 0x7f373419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373aae3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3734107760 0x7f37341987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373aae3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3734112570 con 0x7f3734107760 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373aae3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37341123f0 con 0x7f3734102e00 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373aae3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f37341126f0 con 0x7f37341037b0 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373887f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102e00 0x7f373419e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373887f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102e00 0x7f373419e650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:39662/0 (socket says 172.21.15.59:39662) 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f373887f700 1 -- 172.21.15.59:0/1318383222 learned_addr learned my addr 172.21.15.59:0/1318383222 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f3739080700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3734107760 0x7f37341987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.487 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.483+0000 7f3733fff700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37341037b0 0x7f373419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.484+0000 7f373887f700 1 -- 172.21.15.59:0/1318383222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37341037b0 msgr2=0x7f373419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.484+0000 7f373887f700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37341037b0 0x7f373419eb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.484+0000 7f373887f700 1 -- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3734107760 msgr2=0x7f37341987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.484+0000 7f373887f700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3734107760 0x7f37341987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.484+0000 7f373887f700 1 -- 172.21.15.59:0/1318383222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3734198fb0 con 0x7f3734102e00 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.484+0000 7f373887f700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102e00 0x7f373419e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f372800e9a0 tx=0x7f372800ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.484+0000 7f3731ffb700 1 -- 172.21.15.59:0/1318383222 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372800cbd0 con 0x7f3734102e00 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.484+0000 7f3731ffb700 1 -- 172.21.15.59:0/1318383222 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372800cd70 con 0x7f3734102e00 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.484+0000 7f373aae3700 1 -- 172.21.15.59:0/1318383222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37341992a0 con 0x7f3734102e00 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.485+0000 7f3731ffb700 1 -- 172.21.15.59:0/1318383222 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3728018680 con 0x7f3734102e00 2024-05-08T05:35:25.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.485+0000 7f373aae3700 1 -- 172.21.15.59:0/1318383222 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3734101a90 con 0x7f3734102e00 2024-05-08T05:35:25.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.488+0000 7f3731ffb700 1 -- 172.21.15.59:0/1318383222 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3728010430 con 0x7f3734102e00 2024-05-08T05:35:25.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3745376b80 1 Processor -- start 2024-05-08T05:35:25.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3745376b80 1 -- start start 2024-05-08T05:35:25.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3745376b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d5410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3745376b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 0x5567bb0d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.492 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3745376b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0d12b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3745376b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5567bb0229f0 con 0x5567bb0d0e50 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3745376b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5567bafc1950 con 0x5567bb0d5030 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3745376b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5567bb01e9b0 con 0x5567bb0d5950 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3739080700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d5410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3739080700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d5410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:39668/0 (socket says 172.21.15.59:39668) 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f373887f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 0x5567bb0d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3739080700 1 -- 172.21.15.59:0/3484223618 learned_addr learned my addr 172.21.15.59:0/3484223618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.489+0000 7f3731ffb700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f37240b2b20 0x7f37240b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3739080700 1 -- 172.21.15.59:0/3484223618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 msgr2=0x5567bb0d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3733fff700 1 --2- 172.21.15.59:0/3484223618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0d12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3739080700 1 --2- 172.21.15.59:0/3484223618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 0x5567bb0d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3739080700 1 -- 172.21.15.59:0/3484223618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 msgr2=0x5567bb0d12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3739080700 1 --2- 172.21.15.59:0/3484223618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0d12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.493 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3739080700 1 -- 172.21.15.59:0/3484223618 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567baf6fa60 con 0x5567bb0d5030 2024-05-08T05:35:25.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3733fff700 1 --2- 172.21.15.59:0/3484223618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:35:25.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3731ffb700 1 -- 172.21.15.59:0/1318383222 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f37280e2ba0 con 0x7f3734102e00 2024-05-08T05:35:25.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3739080700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f37240b2b20 0x7f37240b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3739080700 1 --2- 172.21.15.59:0/3484223618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d5410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f372c0093a0 tx=0x7f372c009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=650ad127be295bce server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:25.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f371effd700 1 -- 172.21.15.59:0/3484223618 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c004620 con 0x5567bb0d5030 2024-05-08T05:35:25.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f371effd700 1 -- 172.21.15.59:0/3484223618 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c0047c0 con 0x5567bb0d5030 2024-05-08T05:35:25.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f371effd700 1 -- 172.21.15.59:0/3484223618 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372c023440 con 0x5567bb0d5030 2024-05-08T05:35:25.494 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.490+0000 7f3739080700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f37240b2b20 0x7f37240b4fe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f372c020000 tx=0x7f372c023000 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:25.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:25 smithi143 ceph-mon[34337]: pgmap v178: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.2 MiB/s wr, 356 op/s 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.491+0000 7f3745376b80 1 -- 172.21.15.59:0/3484223618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 msgr2=0x5567bb0d5410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.491+0000 7f3745376b80 1 --2- 172.21.15.59:0/3484223618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d5410 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f372c0093a0 tx=0x7f372c009920 comp rx=0 tx=0).stop 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.492+0000 7f3745376b80 1 -- 172.21.15.59:0/3484223618 shutdown_connections 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.492+0000 7f3745376b80 1 --2- 172.21.15.59:0/3484223618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0d12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.492+0000 7f3745376b80 1 --2- 172.21.15.59:0/3484223618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d5410 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.492+0000 7f3745376b80 1 --2- 172.21.15.59:0/3484223618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 0x5567bb0d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.492+0000 7f3745376b80 1 -- 172.21.15.59:0/3484223618 >> 172.21.15.59:0/3484223618 conn(0x5567baffc290 msgr2=0x5567bb0cb070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.492+0000 7f3745376b80 1 -- 172.21.15.59:0/3484223618 shutdown_connections 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.492+0000 7f3745376b80 1 -- 172.21.15.59:0/3484223618 wait complete. 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.492+0000 7f3745376b80 1 -- start start 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.492+0000 7f3745376b80 1 Processor -- start 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3745376b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0dc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3745376b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d9b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.500 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3745376b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 0x5567bb0da060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f373887f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0dc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f373887f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0dc710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:54096/0 (socket says 172.21.15.59:54096) 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f373887f700 1 -- 172.21.15.59:0/3030465589 learned_addr learned my addr 172.21.15.59:0/3030465589 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5567bb0dd520 con 0x5567bb0d0e50 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5567baf88dc0 con 0x5567bb0d5030 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3733fff700 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d9b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5567bb0dd6a0 con 0x5567bb0d5950 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3745376b80 10 client.0 _ll_register_callbacks cb 0x5567bafbaab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3739080700 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 0x5567bb0da060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3733fff700 1 -- 172.21.15.59:0/3030465589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 msgr2=0x5567bb0da060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3733fff700 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 0x5567bb0da060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3733fff700 1 -- 172.21.15.59:0/3030465589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 msgr2=0x5567bb0dc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3733fff700 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0dc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.493+0000 7f3733fff700 1 -- 172.21.15.59:0/3030465589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567bafbe820 con 0x5567bb0d5030 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3733fff700 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d9b20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3720009990 tx=0x7f3720005760 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24f10ad206373fe5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:25.501 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3713fff700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3713fff700 1 -- 172.21.15.59:0/3030465589 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3720027060 con 0x5567bb0d5030 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f372c0047c0 con 0x5567bb0d5030 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3713fff700 1 -- 172.21.15.59:0/3030465589 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f37200039c0 con 0x5567bb0d5030 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3713fff700 1 -- 172.21.15.59:0/3030465589 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f372002e050 con 0x5567bb0d5030 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3745376b80 10 client.25772 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5567bb0f04f0 con 0x5567bb0d5030 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3713fff700 1 -- 172.21.15.59:0/3030465589 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f372c0047c0 con 0x5567bb0d5030 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3713fff700 1 -- 172.21.15.59:0/3030465589 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f372000f460 con 0x5567bb0d5030 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3745376b80 10 client.25772 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3745376b80 10 client.25772 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3745376b80 20 client.25772 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3745376b80 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5567bb0f2a80 0x5567bb0f4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3745376b80 4 client.25772 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:35:25.502 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.494+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5567bb09c7e0 con 0x5567bb0f2a80 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.495+0000 7f373887f700 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5567bb0f2a80 0x5567bb0f4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.495+0000 7f3713fff700 10 client.25772 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.495+0000 7f373887f700 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5567bb0f2a80 0x5567bb0f4e70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f28887e5f91ea55 server_cookie=f86056ac1836e2a0 in_seq=0 out_seq=0 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.496+0000 7f3713fff700 1 -- 172.21.15.59:0/3030465589 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5567bb09c7e0 con 0x5567bb0f2a80 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.496+0000 7f3713fff700 10 client.25772 handle_command_reply: tid=0 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.496+0000 7f3745376b80 1 client.25772 shutdown 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.496+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 msgr2=0x5567bb0d9b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.496+0000 7f3745376b80 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d9b20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f3720009990 tx=0x7f3720005760 comp rx=0 tx=0).stop 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 shutdown_connections 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3745376b80 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5567bb0f2a80 0x5567bb0f4e70 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3745376b80 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5950 0x5567bb0da060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3745376b80 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0dc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.503 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3745376b80 1 --2- 172.21.15.59:0/3030465589 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5030 0x5567bb0d9b20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 >> 172.21.15.59:0/3030465589 conn(0x5567baffc290 msgr2=0x5567bb0c9e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3713fff700 0 client.25772 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 shutdown_connections 2024-05-08T05:35:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3745376b80 1 -- 172.21.15.59:0/3030465589 wait complete. 2024-05-08T05:35:25.504 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.497+0000 7f3745376b80 20 client.25772 trim_cache size 0 max 16384 2024-05-08T05:35:25.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.523+0000 7f3745376b80 1 Processor -- start 2024-05-08T05:35:25.524 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.523+0000 7f3745376b80 1 -- start start 2024-05-08T05:35:25.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.523+0000 7f3745376b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0fa6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.523+0000 7f3745376b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5030 0x5567bb0fac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.523+0000 7f3745376b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 0x5567bb0d6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.523+0000 7f3745376b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37341ad1e0 con 0x5567bb0d0e50 2024-05-08T05:35:25.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.523+0000 7f3733fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5030 0x5567bb0fac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f3739080700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 0x5567bb0d6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.525 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f3733fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5030 0x5567bb0fac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:44440/0 (socket says 172.21.15.59:44440) 2024-05-08T05:35:25.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f3733fff700 1 -- 172.21.15.59:0/1621102891 learned_addr learned my addr 172.21.15.59:0/1621102891 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:25.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f3745376b80 1 -- 172.21.15.59:0/1621102891 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37341aed20 con 0x5567bb0d5950 2024-05-08T05:35:25.526 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f3745376b80 1 -- 172.21.15.59:0/1621102891 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f37341ad8b0 con 0x5567bb0d5030 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f373887f700 1 --2- 172.21.15.59:0/1621102891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0fa6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f373887f700 1 -- 172.21.15.59:0/1621102891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5030 msgr2=0x5567bb0fac30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f373887f700 1 --2- 172.21.15.59:0/1621102891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5030 0x5567bb0fac30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f373887f700 1 -- 172.21.15.59:0/1621102891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 msgr2=0x5567bb0d6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f373887f700 1 --2- 172.21.15.59:0/1621102891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 0x5567bb0d6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f373887f700 1 -- 172.21.15.59:0/1621102891 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5567bb0dca80 con 0x5567bb0d0e50 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f3739080700 1 --2- 172.21.15.59:0/1621102891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 0x5567bb0d6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.524+0000 7f373887f700 1 --2- 172.21.15.59:0/1621102891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0fa6f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f37280a5b90 tx=0x7f37280e5030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34550896a3fa79b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.525+0000 7f371e7fc700 1 -- 172.21.15.59:0/1621102891 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37280a9460 con 0x5567bb0d0e50 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.525+0000 7f371e7fc700 1 -- 172.21.15.59:0/1621102891 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37280b22b0 con 0x5567bb0d0e50 2024-05-08T05:35:25.528 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.525+0000 7f371e7fc700 1 -- 172.21.15.59:0/1621102891 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37280a52a0 con 0x5567bb0d0e50 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 -- 172.21.15.59:0/1621102891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 msgr2=0x5567bb0fa6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 --2- 172.21.15.59:0/1621102891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0fa6f0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f37280a5b90 tx=0x7f37280e5030 comp rx=0 tx=0).stop 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 -- 172.21.15.59:0/1621102891 shutdown_connections 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 --2- 172.21.15.59:0/1621102891 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 0x5567bb0d6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 --2- 172.21.15.59:0/1621102891 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d0e50 0x5567bb0fa6f0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 --2- 172.21.15.59:0/1621102891 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d5030 0x5567bb0fac30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 -- 172.21.15.59:0/1621102891 >> 172.21.15.59:0/1621102891 conn(0x5567baffc290 msgr2=0x5567bb0f4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 -- 172.21.15.59:0/1621102891 shutdown_connections 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 -- 172.21.15.59:0/1621102891 wait complete. 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.526+0000 7f3745376b80 1 -- start start 2024-05-08T05:35:25.529 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3745376b80 1 Processor -- start 2024-05-08T05:35:25.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:25 smithi096 ceph-mon[34253]: pgmap v178: 129 pgs: 129 active+clean; 50 MiB data, 1.3 GiB used, 1.0 TiB / 1.0 TiB avail; 6.0 MiB/s rd, 2.2 MiB/s wr, 356 op/s 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3745376b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d0e50 0x5567bb0eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3745376b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d5030 0x5567bb0e9b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3745376b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 0x5567bb0ea170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3745376b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f37341ae650 con 0x5567bb0d5030 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3745376b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37341adf80 con 0x5567bb0d5950 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3745376b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f37341ab160 con 0x5567bb0d0e50 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3745376b80 10 client.0 _ll_register_callbacks cb 0x5567bafbaab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f373887f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d0e50 0x5567bb0eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3733fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d5030 0x5567bb0e9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.532 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f373887f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d0e50 0x5567bb0eb7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:44450/0 (socket says 172.21.15.59:44450) 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f373887f700 1 -- 172.21.15.59:0/1475064260 learned_addr learned my addr 172.21.15.59:0/1475064260 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.527+0000 7f3739080700 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 0x5567bb0ea170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f373887f700 1 -- 172.21.15.59:0/1475064260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 msgr2=0x5567bb0ea170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f373887f700 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 0x5567bb0ea170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f373887f700 1 -- 172.21.15.59:0/1475064260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d5030 msgr2=0x5567bb0e9b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f373887f700 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d5030 0x5567bb0e9b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f373887f700 1 -- 172.21.15.59:0/1475064260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5567bb15d2c0 con 0x5567bb0d0e50 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f373887f700 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d0e50 0x5567bb0eb7b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37280e4010 tx=0x7f37280e63f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2cd2aa4bc20a8d99 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f3713fff700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f3713fff700 1 -- 172.21.15.59:0/1475064260 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f372808e020 con 0x5567bb0d0e50 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f3745376b80 1 -- 172.21.15.59:0/1475064260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5567bb0303d0 con 0x5567bb0d0e50 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f3713fff700 1 -- 172.21.15.59:0/1475064260 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f37280b26f0 con 0x5567bb0d0e50 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f3713fff700 1 -- 172.21.15.59:0/1475064260 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f37280a9280 con 0x5567bb0d0e50 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f3745376b80 10 client.25741 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.528+0000 7f3745376b80 1 -- 172.21.15.59:0/1475064260 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5567bb15e3a0 con 0x5567bb0d0e50 2024-05-08T05:35:25.533 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.529+0000 7f3713fff700 1 -- 172.21.15.59:0/1475064260 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5567bb0303d0 con 0x5567bb0d0e50 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.529+0000 7f3713fff700 1 -- 172.21.15.59:0/1475064260 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f3728010720 con 0x5567bb0d0e50 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.529+0000 7f3745376b80 10 client.25741 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.529+0000 7f3745376b80 10 client.25741 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.529+0000 7f3745376b80 20 client.25741 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.529+0000 7f3745376b80 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5567bb15ea80 0x5567bb160ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.529+0000 7f3745376b80 4 client.25741 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.7, "await_for": 42.0}] 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.529+0000 7f3745376b80 1 -- 172.21.15.59:0/1475064260 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.7, "await_for": 42.0}) -- 0x5567bb02a470 con 0x5567bb15ea80 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.529+0000 7f3733fff700 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5567bb15ea80 0x5567bb160ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.530+0000 7f3733fff700 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5567bb15ea80 0x5567bb160ea0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c90d7a52038c7c4 server_cookie=4877e89c4eb26206 in_seq=0 out_seq=0 2024-05-08T05:35:25.534 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:25.530+0000 7f3713fff700 10 client.25741 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:26.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3713fff700 1 -- 172.21.15.59:0/1475064260 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5567bb02a470 con 0x5567bb15ea80 2024-05-08T05:35:26.483 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3713fff700 10 client.25741 handle_command_reply: tid=0 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 client.25741 shutdown 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 -- 172.21.15.59:0/1475064260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d0e50 msgr2=0x5567bb0eb7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d0e50 0x5567bb0eb7b0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f37280e4010 tx=0x7f37280e63f0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 -- 172.21.15.59:0/1475064260 shutdown_connections 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5567bb15ea80 0x5567bb160ea0 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5567bb0d5950 0x5567bb0ea170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5567bb0d0e50 0x5567bb0eb7b0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 --2- 172.21.15.59:0/1475064260 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5567bb0d5030 0x5567bb0e9b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3713fff700 0 client.25741 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 -- 172.21.15.59:0/1475064260 >> 172.21.15.59:0/1475064260 conn(0x5567baffc290 msgr2=0x5567bb0d3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 -- 172.21.15.59:0/1475064260 shutdown_connections 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 1 -- 172.21.15.59:0/1475064260 wait complete. 2024-05-08T05:35:26.484 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.481+0000 7f3745376b80 20 client.25741 trim_cache size 0 max 16384 2024-05-08T05:35:26.485 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:35:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 36, 2024-05-08T05:35:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:35:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 18, 2024-05-08T05:35:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:35:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "3cff3c00": { 2024-05-08T05:35:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 18, 2024-05-08T05:35:26.485 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 85.7, 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:35:26.486 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:35:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:35:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:26.487 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:26.487 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 -- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f37240b2b20 msgr2=0x7f37240b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f37240b2b20 0x7f37240b4fe0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f372c020000 tx=0x7f372c023000 comp rx=0 tx=0).stop 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 -- 172.21.15.59:0/1318383222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102e00 msgr2=0x7f373419e650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102e00 0x7f373419e650 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f372800e9a0 tx=0x7f372800ecb0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 -- 172.21.15.59:0/1318383222 shutdown_connections 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f37240b2b20 0x7f37240b4fe0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3734107760 0x7f37341987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3734102e00 0x7f373419e650 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 --2- 172.21.15.59:0/1318383222 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f37341037b0 0x7f373419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:26.488 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 -- 172.21.15.59:0/1318383222 >> 172.21.15.59:0/1318383222 conn(0x7f3734078a40 msgr2=0x7f3734105f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:26.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 -- 172.21.15.59:0/1318383222 shutdown_connections 2024-05-08T05:35:26.489 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:26.482+0000 7f373aae3700 1 -- 172.21.15.59:0/1318383222 wait complete. 2024-05-08T05:35:27.259 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:27 smithi059 ceph-mon[30304]: pgmap v179: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.2 MiB/s wr, 528 op/s 2024-05-08T05:35:27.295 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '3cff3c00', quiesce took 10.2 seconds. Will release after: 25.7 2024-05-08T05:35:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:27 smithi143 ceph-mon[34337]: pgmap v179: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.2 MiB/s wr, 528 op/s 2024-05-08T05:35:27.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:27 smithi096 ceph-mon[34253]: pgmap v179: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.7 MiB/s rd, 3.2 MiB/s wr, 528 op/s 2024-05-08T05:35:29.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:29 smithi143 ceph-mon[34337]: pgmap v180: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.0 MiB/s wr, 343 op/s 2024-05-08T05:35:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:29 smithi096 ceph-mon[34253]: pgmap v180: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.0 MiB/s wr, 343 op/s 2024-05-08T05:35:29.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:29 smithi059 ceph-mon[30304]: pgmap v180: 129 pgs: 129 active+clean; 44 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 MiB/s rd, 2.0 MiB/s wr, 343 op/s 2024-05-08T05:35:31.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:31 smithi143 ceph-mon[34337]: pgmap v181: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.1 MiB/s wr, 369 op/s 2024-05-08T05:35:31.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:31 smithi096 ceph-mon[34253]: pgmap v181: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.1 MiB/s wr, 369 op/s 2024-05-08T05:35:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:31 smithi059 ceph-mon[30304]: pgmap v181: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.7 MiB/s rd, 2.1 MiB/s wr, 369 op/s 2024-05-08T05:35:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:33 smithi143 ceph-mon[34337]: pgmap v182: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-05-08T05:35:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:33 smithi096 ceph-mon[34253]: pgmap v182: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-05-08T05:35:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:33 smithi059 ceph-mon[30304]: pgmap v182: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-05-08T05:35:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:35 smithi143 ceph-mon[34337]: pgmap v183: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-05-08T05:35:35.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:35 smithi096 ceph-mon[34253]: pgmap v183: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-05-08T05:35:35.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:35 smithi059 ceph-mon[30304]: pgmap v183: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 197 op/s 2024-05-08T05:35:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.195+0000 7fb13a746640 1 -- 172.21.15.59:0/861421084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 msgr2=0x7fb134107cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.195+0000 7fb13a746640 1 --2- 172.21.15.59:0/861421084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 0x7fb134107cc0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7fb124009930 tx=0x7fb12402a420 comp rx=0 tx=0).stop 2024-05-08T05:35:36.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.196+0000 7fb13a746640 1 -- 172.21.15.59:0/861421084 shutdown_connections 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.196+0000 7fb13a746640 1 --2- 172.21.15.59:0/861421084 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb134102290 0x7fb13410e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.196+0000 7fb13a746640 1 --2- 172.21.15.59:0/861421084 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb1341018d0 0x7fb134101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.196+0000 7fb13a746640 1 --2- 172.21.15.59:0/861421084 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 0x7fb134107cc0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.196+0000 7fb13a746640 1 -- 172.21.15.59:0/861421084 >> 172.21.15.59:0/861421084 conn(0x7fb134077c30 msgr2=0x7fb134078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.196+0000 7fb13a746640 1 -- 172.21.15.59:0/861421084 shutdown_connections 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.196+0000 7fb13a746640 1 -- 172.21.15.59:0/861421084 wait complete. 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.196+0000 7fb13a746640 1 Processor -- start 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.196+0000 7fb13a746640 1 -- start start 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb13a746640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1341018d0 0x7fb134197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb13a746640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb134102290 0x7fb1341984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb13a746640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 0x7fb13419c840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb13a746640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb134111b60 con 0x7fb1341078e0 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb13a746640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341119e0 con 0x7fb134102290 2024-05-08T05:35:36.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb13a746640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb134111ce0 con 0x7fb1341018d0 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb138ce3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 0x7fb13419c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb138ce3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 0x7fb13419c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56666/0 (socket says 172.21.15.59:56666) 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb138ce3640 1 -- 172.21.15.59:0/4173394321 learned_addr learned my addr 172.21.15.59:0/4173394321 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb1337fe640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb134102290 0x7fb1341984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb133fff640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1341018d0 0x7fb134197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb138ce3640 1 -- 172.21.15.59:0/4173394321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1341018d0 msgr2=0x7fb134197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb138ce3640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1341018d0 0x7fb134197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb138ce3640 1 -- 172.21.15.59:0/4173394321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb134102290 msgr2=0x7fb1341984c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb138ce3640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb134102290 0x7fb1341984c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb138ce3640 1 -- 172.21.15.59:0/4173394321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb13419cf70 con 0x7fb1341078e0 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb1337fe640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb134102290 0x7fb1341984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.197+0000 7fb138ce3640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 0x7fb13419c840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00cce0 tx=0x7fb12c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.198+0000 7fb1317fa640 1 -- 172.21.15.59:0/4173394321 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00f450 con 0x7fb1341078e0 2024-05-08T05:35:36.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.198+0000 7fb1317fa640 1 -- 172.21.15.59:0/4173394321 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12c00faa0 con 0x7fb1341078e0 2024-05-08T05:35:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.198+0000 7fb13a746640 1 -- 172.21.15.59:0/4173394321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb13419d260 con 0x7fb1341078e0 2024-05-08T05:35:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.198+0000 7fb1317fa640 1 -- 172.21.15.59:0/4173394321 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12c018b70 con 0x7fb1341078e0 2024-05-08T05:35:36.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.198+0000 7fb13a746640 1 -- 172.21.15.59:0/4173394321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1341a4af0 con 0x7fb1341078e0 2024-05-08T05:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.200+0000 7fb1317fa640 1 -- 172.21.15.59:0/4173394321 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb12c018d10 con 0x7fb1341078e0 2024-05-08T05:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.200+0000 7fb13d466740 1 Processor -- start 2024-05-08T05:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb13d466740 1 -- start start 2024-05-08T05:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb13d466740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 0x562f6a29a190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb13d466740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb13d466740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a96f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb13d466740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562f6a1ff5d0 con 0x562f6a29a6d0 2024-05-08T05:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb13d466740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562f6a1b8890 con 0x562f6a2a7300 2024-05-08T05:35:36.211 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb13d466740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562f6a1fb590 con 0x562f6a299db0 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb133fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb133fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56678/0 (socket says 172.21.15.59:56678) 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb133fff640 1 -- 172.21.15.59:0/1733798865 learned_addr learned my addr 172.21.15.59:0/1733798865 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb1317fa640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb11c0b2a10 0x7fb11c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb1337fe640 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 0x562f6a29a190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb1337fe640 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a96f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb133fff640 1 -- 172.21.15.59:0/1733798865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 msgr2=0x562f6a29a190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb133fff640 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 0x562f6a29a190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb138ce3640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb11c0b2a10 0x7fb11c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb133fff640 1 -- 172.21.15.59:0/1733798865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 msgr2=0x562f6a2a96f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb133fff640 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a96f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb133fff640 1 -- 172.21.15.59:0/1733798865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f69f11210 con 0x562f6a29a6d0 2024-05-08T05:35:36.212 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb1337fe640 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.201+0000 7fb1317fa640 1 -- 172.21.15.59:0/4173394321 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fb12c0e2e60 con 0x7fb1341078e0 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.202+0000 7fb133fff640 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a6dc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb12402a940 tx=0x7fb124031cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e534537c07db8e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.202+0000 7fb138ce3640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb11c0b2a10 0x7fb11c0b4ed0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0a7000 tx=0x7fb12c073630 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.202+0000 7fb11a7fc640 1 -- 172.21.15.59:0/1733798865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12403f070 con 0x562f6a29a6d0 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.202+0000 7fb11a7fc640 1 -- 172.21.15.59:0/1733798865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1240316a0 con 0x562f6a29a6d0 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.202+0000 7fb11a7fc640 1 -- 172.21.15.59:0/1733798865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb124035070 con 0x562f6a29a6d0 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.202+0000 7fb13d466740 1 -- 172.21.15.59:0/1733798865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 msgr2=0x562f6a2a6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.202+0000 7fb13d466740 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a6dc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7fb12402a940 tx=0x7fb124031cd0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.203+0000 7fb13d466740 1 -- 172.21.15.59:0/1733798865 shutdown_connections 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.203+0000 7fb13d466740 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a96f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.203+0000 7fb13d466740 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a6dc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.203+0000 7fb13d466740 1 --2- 172.21.15.59:0/1733798865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 0x562f6a29a190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.203+0000 7fb13d466740 1 -- 172.21.15.59:0/1733798865 >> 172.21.15.59:0/1733798865 conn(0x562f6a296420 msgr2=0x562f6a296800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:36.213 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.203+0000 7fb13d466740 1 -- 172.21.15.59:0/1733798865 shutdown_connections 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.203+0000 7fb13d466740 1 -- 172.21.15.59:0/1733798865 wait complete. 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.203+0000 7fb13d466740 1 -- start start 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.203+0000 7fb13d466740 1 Processor -- start 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb13d466740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 0x562f6a2a4200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb13d466740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb13d466740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a4c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb13d466740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562f6a2a9ed0 con 0x562f6a29a6d0 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb13d466740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562f6a1bc370 con 0x562f6a2a7300 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb13d466740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562f6a2aa050 con 0x562f6a299db0 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb13d466740 10 client.0 _ll_register_callbacks cb 0x562f69f10a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb133fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb133fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56682/0 (socket says 172.21.15.59:56682) 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb133fff640 1 -- 172.21.15.59:0/3073550344 learned_addr learned my addr 172.21.15.59:0/3073550344 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:36.214 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb1337fe640 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 0x562f6a2a4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb1337fe640 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a4c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb133fff640 1 -- 172.21.15.59:0/3073550344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 msgr2=0x562f6a2a4200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb133fff640 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 0x562f6a2a4200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb133fff640 1 -- 172.21.15.59:0/3073550344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 msgr2=0x562f6a2a4c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb133fff640 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a4c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb133fff640 1 -- 172.21.15.59:0/3073550344 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f69fdd620 con 0x562f6a29a6d0 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb1337fe640 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb133fff640 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a4740 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb124000c00 tx=0x7fb124031520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8df3a228b4a367a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb1137fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb1137fe640 1 -- 172.21.15.59:0/3073550344 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12404a070 con 0x562f6a29a6d0 2024-05-08T05:35:36.215 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb1137fe640 1 -- 172.21.15.59:0/3073550344 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fb124004220 con 0x562f6a29a6d0 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.204+0000 7fb13d466740 1 -- 172.21.15.59:0/3073550344 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1240316a0 con 0x562f6a29a6d0 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb1137fe640 1 -- 172.21.15.59:0/3073550344 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb124055050 con 0x562f6a29a6d0 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb13d466740 10 client.16191 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb13d466740 1 -- 172.21.15.59:0/3073550344 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562f6a2b5cb0 con 0x562f6a29a6d0 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb1137fe640 1 -- 172.21.15.59:0/3073550344 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1240316a0 con 0x562f6a29a6d0 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb1137fe640 1 -- 172.21.15.59:0/3073550344 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fb1240038c0 con 0x562f6a29a6d0 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb13d466740 10 client.16191 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb13d466740 10 client.16191 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb13d466740 20 client.16191 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb13d466740 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562f6a2b7000 0x562f6a2b93f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb13d466740 4 client.16191 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:35:36.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb13d466740 1 -- 172.21.15.59:0/3073550344 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562f6a20d1e0 con 0x562f6a2b7000 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.205+0000 7fb1337fe640 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562f6a2b7000 0x562f6a2b93f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.206+0000 7fb1337fe640 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562f6a2b7000 0x562f6a2b93f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15ea5b0542537ee server_cookie=3da241add883c8d1 in_seq=0 out_seq=0 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.206+0000 7fb1137fe640 10 client.16191 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb1137fe640 1 -- 172.21.15.59:0/3073550344 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562f6a20d1e0 con 0x562f6a2b7000 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb1137fe640 10 client.16191 handle_command_reply: tid=0 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 client.16191 shutdown 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 -- 172.21.15.59:0/3073550344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 msgr2=0x562f6a2a4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a4740 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7fb124000c00 tx=0x7fb124031520 comp rx=0 tx=0).stop 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 -- 172.21.15.59:0/3073550344 shutdown_connections 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562f6a2b7000 0x562f6a2b93f0 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2a4c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2a4740 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 --2- 172.21.15.59:0/3073550344 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a299db0 0x562f6a2a4200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb1137fe640 0 client.16191 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 -- 172.21.15.59:0/3073550344 >> 172.21.15.59:0/3073550344 conn(0x562f6a296420 msgr2=0x562f6a29c290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 -- 172.21.15.59:0/3073550344 shutdown_connections 2024-05-08T05:35:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 1 -- 172.21.15.59:0/3073550344 wait complete. 2024-05-08T05:35:36.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.207+0000 7fb13d466740 20 client.16191 trim_cache size 0 max 16384 2024-05-08T05:35:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.222+0000 7fb13d466740 1 Processor -- start 2024-05-08T05:35:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb13d466740 1 -- start start 2024-05-08T05:35:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb13d466740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a299db0 0x562f6a295400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb13d466740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a29a6d0 0x562f6a2be9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb13d466740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2bef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb13d466740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341ad990 con 0x562f6a299db0 2024-05-08T05:35:36.232 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb13d466740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341af4d0 con 0x562f6a2a7300 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb13d466740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341ae060 con 0x562f6a29a6d0 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb1337fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a299db0 0x562f6a295400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb1337fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a299db0 0x562f6a295400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56688/0 (socket says 172.21.15.59:56688) 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb1337fe640 1 -- 172.21.15.59:0/194844890 learned_addr learned my addr 172.21.15.59:0/194844890 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb1337fe640 1 -- 172.21.15.59:0/194844890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a29a6d0 msgr2=0x562f6a2be9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb133fff640 1 --2- 172.21.15.59:0/194844890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a29a6d0 0x562f6a2be9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb1337fe640 1 --2- 172.21.15.59:0/194844890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a29a6d0 0x562f6a2be9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb1337fe640 1 -- 172.21.15.59:0/194844890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 msgr2=0x562f6a2bef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb1337fe640 1 --2- 172.21.15.59:0/194844890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2bef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb1337fe640 1 -- 172.21.15.59:0/194844890 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f6a2b9760 con 0x562f6a299db0 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb133fff640 1 --2- 172.21.15.59:0/194844890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a29a6d0 0x562f6a2be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb1337fe640 1 --2- 172.21.15.59:0/194844890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a299db0 0x562f6a295400 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb128000c00 tx=0x7fb12800b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=328c245fd3205ae6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.223+0000 7fb119ffb640 1 -- 172.21.15.59:0/194844890 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb128022030 con 0x562f6a299db0 2024-05-08T05:35:36.233 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb119ffb640 1 -- 172.21.15.59:0/194844890 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12801e580 con 0x562f6a299db0 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb119ffb640 1 -- 172.21.15.59:0/194844890 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1280039f0 con 0x562f6a299db0 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 -- 172.21.15.59:0/194844890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a299db0 msgr2=0x562f6a295400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 --2- 172.21.15.59:0/194844890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a299db0 0x562f6a295400 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7fb128000c00 tx=0x7fb12800b460 comp rx=0 tx=0).stop 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 -- 172.21.15.59:0/194844890 shutdown_connections 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 --2- 172.21.15.59:0/194844890 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a2a7300 0x562f6a2bef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 --2- 172.21.15.59:0/194844890 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a29a6d0 0x562f6a2be9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 --2- 172.21.15.59:0/194844890 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a299db0 0x562f6a295400 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 -- 172.21.15.59:0/194844890 >> 172.21.15.59:0/194844890 conn(0x562f6a296420 msgr2=0x562f6a294cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 -- 172.21.15.59:0/194844890 shutdown_connections 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 -- 172.21.15.59:0/194844890 wait complete. 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.224+0000 7fb13d466740 1 -- start start 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb13d466740 1 Processor -- start 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb13d466740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a299db0 0x562f6a2af670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.234 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb13d466740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2aca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb13d466740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a2a7300 0x562f6a2acf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb13d466740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341aee00 con 0x562f6a29a6d0 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb13d466740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341ae730 con 0x562f6a299db0 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb13d466740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb1341ab950 con 0x562f6a2a7300 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb13d466740 10 client.0 _ll_register_callbacks cb 0x7fb124031ba0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb133fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2aca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb133fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2aca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:56700/0 (socket says 172.21.15.59:56700) 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb133fff640 1 -- 172.21.15.59:0/661492179 learned_addr learned my addr 172.21.15.59:0/661492179 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb1337fe640 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a2a7300 0x562f6a2acf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb133fff640 1 -- 172.21.15.59:0/661492179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a2a7300 msgr2=0x562f6a2acf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb133fff640 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a2a7300 0x562f6a2acf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb133fff640 1 -- 172.21.15.59:0/661492179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a299db0 msgr2=0x562f6a2af670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:36.235 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb133fff640 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a299db0 0x562f6a2af670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb133fff640 1 -- 172.21.15.59:0/661492179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f6a2ad5b0 con 0x562f6a29a6d0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.225+0000 7fb1337fe640 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a2a7300 0x562f6a2acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb133fff640 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2aca00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb124005250 tx=0x7fb124032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d8cc74814c784d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb1137fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb1137fe640 1 -- 172.21.15.59:0/661492179 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb12404a040 con 0x562f6a29a6d0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb1137fe640 1 -- 172.21.15.59:0/661492179 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fb1240049b0 con 0x562f6a29a6d0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb13d466740 1 -- 172.21.15.59:0/661492179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb12801e580 con 0x562f6a29a6d0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb1137fe640 1 -- 172.21.15.59:0/661492179 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fb124062050 con 0x562f6a29a6d0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb13d466740 10 client.16197 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb13d466740 1 -- 172.21.15.59:0/661492179 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562f6a321390 con 0x562f6a29a6d0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb1137fe640 1 -- 172.21.15.59:0/661492179 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb12801e580 con 0x562f6a29a6d0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb1137fe640 1 -- 172.21.15.59:0/661492179 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fb124003e30 con 0x562f6a29a6d0 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb13d466740 10 client.16197 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:36.236 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb13d466740 10 client.16197 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb13d466740 20 client.16197 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb13d466740 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562f6a321b10 0x562f6a323f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb13d466740 4 client.16197 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.226+0000 7fb13d466740 1 -- 172.21.15.59:0/661492179 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562f6a207050 con 0x562f6a321b10 2024-05-08T05:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.227+0000 7fb1337fe640 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562f6a321b10 0x562f6a323f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.227+0000 7fb1337fe640 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562f6a321b10 0x562f6a323f30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48c4a61da9da6e44 server_cookie=c55663c8c00bcc5d in_seq=0 out_seq=0 2024-05-08T05:35:36.237 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.227+0000 7fb1137fe640 10 client.16197 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:36.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb1137fe640 1 -- 172.21.15.59:0/661492179 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562f6a207050 con 0x562f6a321b10 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb1137fe640 10 client.16197 handle_command_reply: tid=0 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 client.16197 shutdown 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 -- 172.21.15.59:0/661492179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 msgr2=0x562f6a2aca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2aca00 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7fb124005250 tx=0x7fb124032f10 comp rx=0 tx=0).stop 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 -- 172.21.15.59:0/661492179 shutdown_connections 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562f6a321b10 0x562f6a323f30 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562f6a2a7300 0x562f6a2acf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562f6a29a6d0 0x562f6a2aca00 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb1137fe640 0 client.16197 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 --2- 172.21.15.59:0/661492179 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562f6a299db0 0x562f6a2af670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 -- 172.21.15.59:0/661492179 >> 172.21.15.59:0/661492179 conn(0x562f6a296420 msgr2=0x562f6a295580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 -- 172.21.15.59:0/661492179 shutdown_connections 2024-05-08T05:35:36.245 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 1 -- 172.21.15.59:0/661492179 wait complete. 2024-05-08T05:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.243+0000 7fb13d466740 20 client.16197 trim_cache size 0 max 16384 2024-05-08T05:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:35:36.246 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 -- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb11c0b2a10 msgr2=0x7fb11c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb11c0b2a10 0x7fb11c0b4ed0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fb12c0a7000 tx=0x7fb12c073630 comp rx=0 tx=0).stop 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 -- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 msgr2=0x7fb13419c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 0x7fb13419c840 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7fb12c00cce0 tx=0x7fb12c00a620 comp rx=0 tx=0).stop 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 -- 172.21.15.59:0/4173394321 shutdown_connections 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb11c0b2a10 0x7fb11c0b4ed0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb1341078e0 0x7fb13419c840 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb134102290 0x7fb1341984c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 --2- 172.21.15.59:0/4173394321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb1341018d0 0x7fb134197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:36.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 -- 172.21.15.59:0/4173394321 >> 172.21.15.59:0/4173394321 conn(0x7fb134077c30 msgr2=0x7fb1340ff610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 -- 172.21.15.59:0/4173394321 shutdown_connections 2024-05-08T05:35:36.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:36.244+0000 7fb13a746640 1 -- 172.21.15.59:0/4173394321 wait complete. 2024-05-08T05:35:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:37 smithi143 ceph-mon[34337]: pgmap v184: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 202 op/s 2024-05-08T05:35:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:37 smithi096 ceph-mon[34253]: pgmap v184: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 202 op/s 2024-05-08T05:35:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:37 smithi059 ceph-mon[30304]: pgmap v184: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 3.1 MiB/s rd, 1.1 MiB/s wr, 202 op/s 2024-05-08T05:35:39.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:39 smithi143 ceph-mon[34337]: pgmap v185: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 137 KiB/s wr, 30 op/s 2024-05-08T05:35:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:39 smithi096 ceph-mon[34253]: pgmap v185: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 137 KiB/s wr, 30 op/s 2024-05-08T05:35:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:39 smithi059 ceph-mon[30304]: pgmap v185: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 279 KiB/s rd, 137 KiB/s wr, 30 op/s 2024-05-08T05:35:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:41 smithi143 ceph-mon[34337]: pgmap v186: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 146 KiB/s wr, 37 op/s 2024-05-08T05:35:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:41 smithi096 ceph-mon[34253]: pgmap v186: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 146 KiB/s wr, 37 op/s 2024-05-08T05:35:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:41 smithi059 ceph-mon[30304]: pgmap v186: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 287 KiB/s rd, 146 KiB/s wr, 37 op/s 2024-05-08T05:35:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:43 smithi143 ceph-mon[34337]: pgmap v187: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-08T05:35:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:43 smithi096 ceph-mon[34253]: pgmap v187: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-08T05:35:43.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:43 smithi059 ceph-mon[30304]: pgmap v187: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-08T05:35:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:44 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:35:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:44 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:35:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:44 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:35:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:44 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:35:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:44 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:35:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:44 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:35:44.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:44 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:35:44.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:44 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:35:44.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:44 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:35:44.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:44 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:35:44.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:44 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:35:44.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:44 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:35:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:45 smithi143 ceph-mon[34337]: pgmap v188: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-08T05:35:45.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:45 smithi096 ceph-mon[34253]: pgmap v188: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-08T05:35:45.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:45 smithi059 ceph-mon[30304]: pgmap v188: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-08T05:35:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:47 smithi143 ceph-mon[34337]: pgmap v189: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 16 op/s 2024-05-08T05:35:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:47 smithi096 ceph-mon[34253]: pgmap v189: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 16 op/s 2024-05-08T05:35:47.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:47 smithi059 ceph-mon[30304]: pgmap v189: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 25 KiB/s wr, 16 op/s 2024-05-08T05:35:49.347 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.347+0000 7f9314f90640 1 -- 172.21.15.59:0/1856932124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310101120 msgr2=0x7f9310101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.347+0000 7f9314f90640 1 --2- 172.21.15.59:0/1856932124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310101120 0x7f9310101500 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f9300009900 tx=0x7f930002a410 comp rx=0 tx=0).stop 2024-05-08T05:35:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.347+0000 7f9314f90640 1 -- 172.21.15.59:0/1856932124 shutdown_connections 2024-05-08T05:35:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.347+0000 7f9314f90640 1 --2- 172.21.15.59:0/1856932124 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310107ce0 0x7f931010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.347+0000 7f9314f90640 1 --2- 172.21.15.59:0/1856932124 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9310101ad0 0x7f9310107600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.347+0000 7f9314f90640 1 --2- 172.21.15.59:0/1856932124 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9310101120 0x7f9310101500 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.347+0000 7f9314f90640 1 -- 172.21.15.59:0/1856932124 >> 172.21.15.59:0/1856932124 conn(0x7f9310077e40 msgr2=0x7f93100ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.347+0000 7f9314f90640 1 -- 172.21.15.59:0/1856932124 shutdown_connections 2024-05-08T05:35:49.348 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.347+0000 7f9314f90640 1 -- 172.21.15.59:0/1856932124 wait complete. 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f9314f90640 1 Processor -- start 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f9314f90640 1 -- start start 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f9314f90640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9310101120 0x7f93101991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f9314f90640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310107ce0 0x7f93101996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f9314f90640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f931019da20 0x7f931019ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f9314f90640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9310112c90 con 0x7f931019da20 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f9314f90640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9310112b10 con 0x7f9310107ce0 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f9314f90640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9310112e10 con 0x7f9310101120 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f931019da20 0x7f931019ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.350 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930ed9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f931019da20 0x7f931019ded0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46612/0 (socket says 172.21.15.59:46612) 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930ed9d640 1 -- 172.21.15.59:0/1154635066 learned_addr learned my addr 172.21.15.59:0/1154635066 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930e59c640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9310101120 0x7f93101991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930dd9b640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310107ce0 0x7f93101996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930ed9d640 1 -- 172.21.15.59:0/1154635066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9310101120 msgr2=0x7f93101991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930ed9d640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9310101120 0x7f93101991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930ed9d640 1 -- 172.21.15.59:0/1154635066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310107ce0 msgr2=0x7f93101996f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930ed9d640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310107ce0 0x7f93101996f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.348+0000 7f930ed9d640 1 -- 172.21.15.59:0/1154635066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f931019e510 con 0x7f931019da20 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.349+0000 7f930ed9d640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f931019da20 0x7f931019ded0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9308009fd0 tx=0x7f930800b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.349+0000 7f92ff7fe640 1 -- 172.21.15.59:0/1154635066 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308017070 con 0x7f931019da20 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.349+0000 7f92ff7fe640 1 -- 172.21.15.59:0/1154635066 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9308003e20 con 0x7f931019da20 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.349+0000 7f9314f90640 1 -- 172.21.15.59:0/1154635066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f931019e800 con 0x7f931019da20 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.349+0000 7f92ff7fe640 1 -- 172.21.15.59:0/1154635066 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9308010460 con 0x7f931019da20 2024-05-08T05:35:49.351 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.349+0000 7f9314f90640 1 -- 172.21.15.59:0/1154635066 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93101a5ff0 con 0x7f931019da20 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.351+0000 7f92ff7fe640 1 -- 172.21.15.59:0/1154635066 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9308003970 con 0x7f931019da20 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.351+0000 7f9317d87740 1 Processor -- start 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f9317d87740 1 -- start start 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f9317d87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 0x562a513cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f9317d87740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cc730 0x562a513dafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f9317d87740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513db500 0x562a513dd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f9317d87740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562a513315d0 con 0x562a513db500 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f9317d87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a512ea890 con 0x562a513cbe10 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f9317d87740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562a5132d590 con 0x562a513cc730 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f930dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513db500 0x562a513dd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.354 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f930dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513db500 0x562a513dd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46614/0 (socket says 172.21.15.59:46614) 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f930dd9b640 1 -- 172.21.15.59:0/2191906019 learned_addr learned my addr 172.21.15.59:0/2191906019 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f930dd9b640 1 --2- 172.21.15.59:0/2191906019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 0x562a513cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.352+0000 7f930e59c640 1 --2- 172.21.15.59:0/2191906019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cc730 0x562a513dafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f930dd9b640 1 -- 172.21.15.59:0/2191906019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cc730 msgr2=0x562a513dafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f930dd9b640 1 --2- 172.21.15.59:0/2191906019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cc730 0x562a513dafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f930dd9b640 1 -- 172.21.15.59:0/2191906019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 msgr2=0x562a513cc1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f930dd9b640 1 --2- 172.21.15.59:0/2191906019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 0x562a513cc1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f930dd9b640 1 -- 172.21.15.59:0/2191906019 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a51043210 con 0x562a513db500 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f92ff7fe640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f92f80b2a90 0x7f92f80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f930dd9b640 1 --2- 172.21.15.59:0/2191906019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513db500 0x562a513dd8f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9304012760 tx=0x7f93040096c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3bf8fd31bfbe545 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f930e59c640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f92f80b2a90 0x7f92f80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f92fcff9640 1 -- 172.21.15.59:0/2191906019 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304012ce0 con 0x562a513db500 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f92ff7fe640 1 -- 172.21.15.59:0/1154635066 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f93080e2700 con 0x7f931019da20 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f92fcff9640 1 -- 172.21.15.59:0/2191906019 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93040107d0 con 0x562a513db500 2024-05-08T05:35:49.355 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.353+0000 7f92fcff9640 1 -- 172.21.15.59:0/2191906019 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93040044e0 con 0x562a513db500 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f9317d87740 1 -- 172.21.15.59:0/2191906019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513db500 msgr2=0x562a513dd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f930e59c640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f92f80b2a90 0x7f92f80b4f50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9300000c00 tx=0x7f9300005b50 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f9317d87740 1 --2- 172.21.15.59:0/2191906019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513db500 0x562a513dd8f0 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f9304012760 tx=0x7f93040096c0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f9317d87740 1 -- 172.21.15.59:0/2191906019 shutdown_connections 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f9317d87740 1 --2- 172.21.15.59:0/2191906019 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513db500 0x562a513dd8f0 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f9317d87740 1 --2- 172.21.15.59:0/2191906019 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cc730 0x562a513dafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f9317d87740 1 --2- 172.21.15.59:0/2191906019 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 0x562a513cc1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f9317d87740 1 -- 172.21.15.59:0/2191906019 >> 172.21.15.59:0/2191906019 conn(0x562a513c8420 msgr2=0x562a513c8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f9317d87740 1 -- 172.21.15.59:0/2191906019 shutdown_connections 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.354+0000 7f9317d87740 1 -- 172.21.15.59:0/2191906019 wait complete. 2024-05-08T05:35:49.360 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f9317d87740 1 -- start start 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f9317d87740 1 Processor -- start 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f9317d87740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 0x562a513d8490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f9317d87740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513d89d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f9317d87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 0x562a513d8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f9317d87740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x562a513de130 con 0x562a513cc730 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f9317d87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x562a512ee370 con 0x562a513db500 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f9317d87740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x562a513de2b0 con 0x562a513cbe10 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f9317d87740 10 client.0 _ll_register_callbacks cb 0x562a51042a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513d89d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513d89d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46624/0 (socket says 172.21.15.59:46624) 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930dd9b640 1 -- 172.21.15.59:0/457698086 learned_addr learned my addr 172.21.15.59:0/457698086 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930ed9d640 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 0x562a513d8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930e59c640 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 0x562a513d8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.361 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930dd9b640 1 -- 172.21.15.59:0/457698086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 msgr2=0x562a513d8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930dd9b640 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 0x562a513d8490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930dd9b640 1 -- 172.21.15.59:0/457698086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 msgr2=0x562a513d8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930dd9b640 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 0x562a513d8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.355+0000 7f930dd9b640 1 -- 172.21.15.59:0/457698086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a5110f620 con 0x562a513cc730 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f930dd9b640 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513d89d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9304009ee0 tx=0x7f9304005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f812b3bec76682cf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f92f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f92f5ffb640 1 -- 172.21.15.59:0/457698086 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304023020 con 0x562a513cc730 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f9317d87740 1 -- 172.21.15.59:0/457698086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93040107d0 con 0x562a513cc730 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f92f5ffb640 1 -- 172.21.15.59:0/457698086 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f9304004240 con 0x562a513cc730 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f92f5ffb640 1 -- 172.21.15.59:0/457698086 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f930402e050 con 0x562a513cc730 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f9317d87740 10 client.16209 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f9317d87740 1 -- 172.21.15.59:0/457698086 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562a513ea080 con 0x562a513cc730 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f92f5ffb640 1 -- 172.21.15.59:0/457698086 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93040107d0 con 0x562a513cc730 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.356+0000 7f92f5ffb640 1 -- 172.21.15.59:0/457698086 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f93040049a0 con 0x562a513cc730 2024-05-08T05:35:49.362 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.357+0000 7f9317d87740 10 client.16209 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.357+0000 7f9317d87740 10 client.16209 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.357+0000 7f9317d87740 20 client.16209 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.357+0000 7f9317d87740 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562a513eb480 0x562a513ed870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.357+0000 7f9317d87740 4 client.16209 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.357+0000 7f9317d87740 1 -- 172.21.15.59:0/457698086 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a5133f1e0 con 0x562a513eb480 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.357+0000 7f930e59c640 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562a513eb480 0x562a513ed870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.357+0000 7f930e59c640 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562a513eb480 0x562a513ed870 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bddaab93c70524b7 server_cookie=d0ae9fd6ae485e0c in_seq=0 out_seq=0 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.357+0000 7f92f5ffb640 10 client.16209 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f92f5ffb640 1 -- 172.21.15.59:0/457698086 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x562a5133f1e0 con 0x562a513eb480 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f92f5ffb640 10 client.16209 handle_command_reply: tid=0 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 client.16209 shutdown 2024-05-08T05:35:49.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 -- 172.21.15.59:0/457698086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 msgr2=0x562a513d89d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513d89d0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f9304009ee0 tx=0x7f9304005f90 comp rx=0 tx=0).stop 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 -- 172.21.15.59:0/457698086 shutdown_connections 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562a513eb480 0x562a513ed870 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 0x562a513d8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f92f5ffb640 0 client.16209 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513d89d0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 --2- 172.21.15.59:0/457698086 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 0x562a513d8490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 -- 172.21.15.59:0/457698086 >> 172.21.15.59:0/457698086 conn(0x562a513c8420 msgr2=0x562a513ce290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.358+0000 7f9317d87740 1 -- 172.21.15.59:0/457698086 shutdown_connections 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.359+0000 7f9317d87740 1 -- 172.21.15.59:0/457698086 wait complete. 2024-05-08T05:35:49.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.359+0000 7f9317d87740 20 client.16209 trim_cache size 0 max 16384 2024-05-08T05:35:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f9317d87740 1 Processor -- start 2024-05-08T05:35:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f9317d87740 1 -- start start 2024-05-08T05:35:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f9317d87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 0x562a513c7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f9317d87740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513f2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f9317d87740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513db500 0x562a513f34f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f9317d87740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93101adb00 con 0x562a513cc730 2024-05-08T05:35:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f9317d87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93101af640 con 0x562a513cbe10 2024-05-08T05:35:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f9317d87740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f93101ae1d0 con 0x562a513db500 2024-05-08T05:35:49.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f930dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513f2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f930dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513f2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46630/0 (socket says 172.21.15.59:46630) 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.374+0000 7f930dd9b640 1 -- 172.21.15.59:0/1969695290 learned_addr learned my addr 172.21.15.59:0/1969695290 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f930ed9d640 1 --2- 172.21.15.59:0/1969695290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513db500 0x562a513f34f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f930e59c640 1 --2- 172.21.15.59:0/1969695290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 0x562a513c7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f930dd9b640 1 -- 172.21.15.59:0/1969695290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513db500 msgr2=0x562a513f34f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f930dd9b640 1 --2- 172.21.15.59:0/1969695290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513db500 0x562a513f34f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f930dd9b640 1 -- 172.21.15.59:0/1969695290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 msgr2=0x562a513c7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f930dd9b640 1 --2- 172.21.15.59:0/1969695290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 0x562a513c7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f930dd9b640 1 -- 172.21.15.59:0/1969695290 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a513ce970 con 0x562a513cc730 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f930dd9b640 1 --2- 172.21.15.59:0/1969695290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513f2fb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f930400a010 tx=0x7f9304012630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=450f3b1141a21359 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f92f7fff640 1 -- 172.21.15.59:0/1969695290 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304037070 con 0x562a513cc730 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f92f7fff640 1 -- 172.21.15.59:0/1969695290 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9304005bf0 con 0x562a513cc730 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.375+0000 7f92f7fff640 1 -- 172.21.15.59:0/1969695290 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304023040 con 0x562a513cc730 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 -- 172.21.15.59:0/1969695290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 msgr2=0x562a513f2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:49.380 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 --2- 172.21.15.59:0/1969695290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513f2fb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f930400a010 tx=0x7f9304012630 comp rx=0 tx=0).stop 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 -- 172.21.15.59:0/1969695290 shutdown_connections 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 --2- 172.21.15.59:0/1969695290 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513db500 0x562a513f34f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 --2- 172.21.15.59:0/1969695290 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513f2fb0 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 --2- 172.21.15.59:0/1969695290 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513cbe10 0x562a513c7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 -- 172.21.15.59:0/1969695290 >> 172.21.15.59:0/1969695290 conn(0x562a513c8420 msgr2=0x562a513c6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 -- 172.21.15.59:0/1969695290 shutdown_connections 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 -- 172.21.15.59:0/1969695290 wait complete. 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 -- start start 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 Processor -- start 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 0x562a513df420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513df960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 0x562a51454b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f93101aef70 con 0x562a513cc730 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f93101ae8a0 con 0x562a513db500 2024-05-08T05:35:49.381 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f93101abac0 con 0x562a513cbe10 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f9317d87740 10 client.0 _ll_register_callbacks cb 0x7f9304009dd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.376+0000 7f930dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513df960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930dd9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513df960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46634/0 (socket says 172.21.15.59:46634) 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930dd9b640 1 -- 172.21.15.59:0/1391601403 learned_addr learned my addr 172.21.15.59:0/1391601403 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930e59c640 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 0x562a513df420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930ed9d640 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 0x562a51454b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930dd9b640 1 -- 172.21.15.59:0/1391601403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 msgr2=0x562a513df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930dd9b640 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 0x562a513df420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930dd9b640 1 -- 172.21.15.59:0/1391601403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 msgr2=0x562a51454b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930dd9b640 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 0x562a51454b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930dd9b640 1 -- 172.21.15.59:0/1391601403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a513e0200 con 0x562a513cc730 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930ed9d640 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 0x562a51454b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f930dd9b640 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513df960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9304003670 tx=0x7f9304009420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7711162b294a6d74 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:49.382 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f92f5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f92f5ffb640 1 -- 172.21.15.59:0/1391601403 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9304037030 con 0x562a513cc730 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f9317d87740 1 -- 172.21.15.59:0/1391601403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562a51339050 con 0x562a513cc730 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f92f5ffb640 1 -- 172.21.15.59:0/1391601403 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f93040039f0 con 0x562a513cc730 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f92f5ffb640 1 -- 172.21.15.59:0/1391601403 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f9304044050 con 0x562a513cc730 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f9317d87740 10 client.16215 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.377+0000 7f9317d87740 1 -- 172.21.15.59:0/1391601403 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x562a51456180 con 0x562a513cc730 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f92f5ffb640 1 -- 172.21.15.59:0/1391601403 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562a51339050 con 0x562a513cc730 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f92f5ffb640 1 -- 172.21.15.59:0/1391601403 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f9304004240 con 0x562a513cc730 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f9317d87740 10 client.16215 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f9317d87740 10 client.16215 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f9317d87740 20 client.16215 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f9317d87740 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562a514567e0 0x562a51458c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f9317d87740 4 client.16215 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f9317d87740 1 -- 172.21.15.59:0/1391601403 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a512b45f0 con 0x562a514567e0 2024-05-08T05:35:49.383 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f930e59c640 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562a514567e0 0x562a51458c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f930e59c640 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562a514567e0 0x562a51458c00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=58177f07fccd3145 server_cookie=632468e84a36b900 in_seq=0 out_seq=0 2024-05-08T05:35:49.384 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.378+0000 7f92f5ffb640 10 client.16215 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f92f5ffb640 1 -- 172.21.15.59:0/1391601403 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a512b45f0 con 0x562a514567e0 2024-05-08T05:35:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f92f5ffb640 10 client.16215 handle_command_reply: tid=0 2024-05-08T05:35:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 client.16215 shutdown 2024-05-08T05:35:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 -- 172.21.15.59:0/1391601403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 msgr2=0x562a513df960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513df960 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f9304003670 tx=0x7f9304009420 comp rx=0 tx=0).stop 2024-05-08T05:35:49.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 -- 172.21.15.59:0/1391601403 shutdown_connections 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x562a514567e0 0x562a51458c00 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x562a513db500 0x562a51454b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x562a513cc730 0x562a513df960 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f92f5ffb640 0 client.16215 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 --2- 172.21.15.59:0/1391601403 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x562a513cbe10 0x562a513df420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 -- 172.21.15.59:0/1391601403 >> 172.21.15.59:0/1391601403 conn(0x562a513c8420 msgr2=0x562a513ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 -- 172.21.15.59:0/1391601403 shutdown_connections 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 1 -- 172.21.15.59:0/1391601403 wait complete. 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.391+0000 7f9317d87740 20 client.16215 trim_cache size 0 max 16384 2024-05-08T05:35:49.393 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:35:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:35:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:35:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:35:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 -- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f92f80b2a90 msgr2=0x7f92f80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:49.394 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f92f80b2a90 0x7f92f80b4f50 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f9300000c00 tx=0x7f9300005b50 comp rx=0 tx=0).stop 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 -- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f931019da20 msgr2=0x7f931019ded0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f931019da20 0x7f931019ded0 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9308009fd0 tx=0x7f930800b260 comp rx=0 tx=0).stop 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 -- 172.21.15.59:0/1154635066 shutdown_connections 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f92f80b2a90 0x7f92f80b4f50 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f931019da20 0x7f931019ded0 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9310107ce0 0x7f93101996f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 --2- 172.21.15.59:0/1154635066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9310101120 0x7f93101991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 -- 172.21.15.59:0/1154635066 >> 172.21.15.59:0/1154635066 conn(0x7f9310077e40 msgr2=0x7f93100ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 -- 172.21.15.59:0/1154635066 shutdown_connections 2024-05-08T05:35:49.395 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:35:49.392+0000 7f9314f90640 1 -- 172.21.15.59:0/1154635066 wait complete. 2024-05-08T05:35:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:49 smithi143 ceph-mon[34337]: pgmap v190: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-08T05:35:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:49 smithi096 ceph-mon[34253]: pgmap v190: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-08T05:35:49.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:49 smithi059 ceph-mon[30304]: pgmap v190: 129 pgs: 129 active+clean; 42 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 23 KiB/s wr, 11 op/s 2024-05-08T05:35:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:51 smithi143 ceph-mon[34337]: pgmap v191: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-05-08T05:35:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:51 smithi096 ceph-mon[34253]: pgmap v191: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-05-08T05:35:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:51 smithi059 ceph-mon[30304]: pgmap v191: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 33 KiB/s wr, 18 op/s 2024-05-08T05:35:52.996 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:35:53.267 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:35:53.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:53 smithi059 ceph-mon[30304]: pgmap v192: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-08T05:35:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:53 smithi143 ceph-mon[34337]: pgmap v192: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-08T05:35:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:53 smithi096 ceph-mon[34253]: pgmap v192: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-08T05:35:55.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.020+0000 7ff183271700 1 -- 172.21.15.59:0/2897614326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06b180 msgr2=0x7ff17c06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:55.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.020+0000 7ff183271700 1 --2- 172.21.15.59:0/2897614326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06b180 0x7ff17c06b600 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7ff16c009990 tx=0x7ff16c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.020+0000 7ff183271700 1 -- 172.21.15.59:0/2897614326 shutdown_connections 2024-05-08T05:35:55.021 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.020+0000 7ff183271700 1 --2- 172.21.15.59:0/2897614326 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff17c06bb40 0x7ff17c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.020+0000 7ff183271700 1 --2- 172.21.15.59:0/2897614326 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c109510 0x7ff17c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.020+0000 7ff183271700 1 --2- 172.21.15.59:0/2897614326 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06b180 0x7ff17c06b600 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.020+0000 7ff183271700 1 -- 172.21.15.59:0/2897614326 >> 172.21.15.59:0/2897614326 conn(0x7ff17c079350 msgr2=0x7ff17c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.021+0000 7ff183271700 1 -- 172.21.15.59:0/2897614326 shutdown_connections 2024-05-08T05:35:55.022 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.021+0000 7ff183271700 1 -- 172.21.15.59:0/2897614326 wait complete. 2024-05-08T05:35:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff183271700 1 Processor -- start 2024-05-08T05:35:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff183271700 1 -- start start 2024-05-08T05:35:55.023 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff183271700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff17c06b180 0x7ff17c10a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff183271700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06bb40 0x7ff17c10a880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff183271700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c109510 0x7ff17c10adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff183271700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c113720 con 0x7ff17c06bb40 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff183271700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1135a0 con 0x7ff17c06b180 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff183271700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff17c1138a0 con 0x7ff17c109510 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18080c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06bb40 0x7ff17c10a880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18080c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06bb40 0x7ff17c10a880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46650/0 (socket says 172.21.15.59:46650) 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18080c700 1 -- 172.21.15.59:0/2420983040 learned_addr learned my addr 172.21.15.59:0/2420983040 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18100d700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff17c06b180 0x7ff17c10a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18180e700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c109510 0x7ff17c10adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18080c700 1 -- 172.21.15.59:0/2420983040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c109510 msgr2=0x7ff17c10adc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18080c700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c109510 0x7ff17c10adc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18080c700 1 -- 172.21.15.59:0/2420983040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff17c06b180 msgr2=0x7ff17c10a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:55.024 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18080c700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff17c06b180 0x7ff17c10a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.022+0000 7ff18080c700 1 -- 172.21.15.59:0/2420983040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff17c1ac560 con 0x7ff17c06bb40 2024-05-08T05:35:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.023+0000 7ff18080c700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06bb40 0x7ff17c10a880 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0057f0 tx=0x7ff16c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.023+0000 7ff1727fc700 1 -- 172.21.15.59:0/2420983040 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c027070 con 0x7ff17c06bb40 2024-05-08T05:35:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.023+0000 7ff1727fc700 1 -- 172.21.15.59:0/2420983040 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff16c004020 con 0x7ff17c06bb40 2024-05-08T05:35:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.023+0000 7ff1727fc700 1 -- 172.21.15.59:0/2420983040 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff16c022ce0 con 0x7ff17c06bb40 2024-05-08T05:35:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.023+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff17c1ac7f0 con 0x7ff17c06bb40 2024-05-08T05:35:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.023+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff17c1acc80 con 0x7ff17c06bb40 2024-05-08T05:35:55.025 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.024+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff17c104bd0 con 0x7ff17c06bb40 2024-05-08T05:35:55.027 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.027+0000 7ff1727fc700 1 -- 172.21.15.59:0/2420983040 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff16c02a070 con 0x7ff17c06bb40 2024-05-08T05:35:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.028+0000 7ff1727fc700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1680b29f0 0x7ff1680b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.028+0000 7ff18100d700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1680b29f0 0x7ff1680b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.028+0000 7ff1727fc700 1 -- 172.21.15.59:0/2420983040 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7ff16c0e3fb0 con 0x7ff17c06bb40 2024-05-08T05:35:55.028 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.028+0000 7ff18100d700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1680b29f0 0x7ff1680b4eb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff178009a10 tx=0x7ff178009190 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:55.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.032+0000 7ff1727fc700 1 -- 172.21.15.59:0/2420983040 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff16c031020 con 0x7ff17c06bb40 2024-05-08T05:35:55.294 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.294+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff17c1acfa0 con 0x7ff17c06bb40 2024-05-08T05:35:55.295 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:55 smithi059 ceph-mon[30304]: pgmap v193: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-08T05:35:55.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.295+0000 7ff1727fc700 1 -- 172.21.15.59:0/2420983040 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11929 (secure 0 0 0) 0x7ff16c013070 con 0x7ff17c06bb40 2024-05-08T05:35:55.295 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:35:55.296 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:34:26.942121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24443,24473,24421,24433,24419]},"id":1}]} 2024-05-08T05:35:55.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1680b29f0 msgr2=0x7ff1680b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1680b29f0 0x7ff1680b4eb0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7ff178009a10 tx=0x7ff178009190 comp rx=0 tx=0).stop 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06bb40 msgr2=0x7ff17c10a880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06bb40 0x7ff17c10a880 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff16c0057f0 tx=0x7ff16c003680 comp rx=0 tx=0).stop 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 shutdown_connections 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1680b29f0 0x7ff1680b4eb0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff17c109510 0x7ff17c10adc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff17c06b180 0x7ff17c10a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 --2- 172.21.15.59:0/2420983040 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff17c06bb40 0x7ff17c10a880 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 >> 172.21.15.59:0/2420983040 conn(0x7ff17c079350 msgr2=0x7ff17c10e680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 shutdown_connections 2024-05-08T05:35:55.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:55.297+0000 7ff183271700 1 -- 172.21.15.59:0/2420983040 wait complete. 2024-05-08T05:35:55.299 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 36 2024-05-08T05:35:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:55 smithi143 ceph-mon[34337]: pgmap v193: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-08T05:35:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:55 smithi096 ceph-mon[34253]: pgmap v193: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 15 KiB/s rd, 24 KiB/s wr, 11 op/s 2024-05-08T05:35:55.980 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db --set-id 3cff3c00 --release --await' 2024-05-08T05:35:55.980 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db --set-id 3cff3c00 --release --await 2024-05-08T05:35:56.237 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:35:56.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:56 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2420983040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:35:56.505 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:56 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2420983040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:35:56.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:56 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2420983040' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:35:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:57 smithi143 ceph-mon[34337]: pgmap v194: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-05-08T05:35:57.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:57 smithi096 ceph-mon[34253]: pgmap v194: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-05-08T05:35:57.544 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:57 smithi059 ceph-mon[30304]: pgmap v194: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 24 KiB/s wr, 15 op/s 2024-05-08T05:35:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.128+0000 7f7ab447e700 1 -- 172.21.15.59:0/3797656373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac06b180 msgr2=0x7f7aac06b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.128+0000 7f7ab447e700 1 --2- 172.21.15.59:0/3797656373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac06b180 0x7f7aac06b600 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0009990 tx=0x7f7aa0009ca0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.128+0000 7f7ab447e700 1 -- 172.21.15.59:0/3797656373 shutdown_connections 2024-05-08T05:35:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.128+0000 7f7ab447e700 1 --2- 172.21.15.59:0/3797656373 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac06bb40 0x7f7aac110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.128+0000 7f7ab447e700 1 --2- 172.21.15.59:0/3797656373 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7aac109510 0x7f7aac1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.128+0000 7f7ab447e700 1 --2- 172.21.15.59:0/3797656373 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac06b180 0x7f7aac06b600 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.128+0000 7f7ab447e700 1 -- 172.21.15.59:0/3797656373 >> 172.21.15.59:0/3797656373 conn(0x7f7aac079350 msgr2=0x7f7aac079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.128+0000 7f7ab447e700 1 -- 172.21.15.59:0/3797656373 shutdown_connections 2024-05-08T05:35:58.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.128+0000 7f7ab447e700 1 -- 172.21.15.59:0/3797656373 wait complete. 2024-05-08T05:35:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.129+0000 7f7ab447e700 1 Processor -- start 2024-05-08T05:35:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.129+0000 7f7ab447e700 1 -- start start 2024-05-08T05:35:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab447e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac06b180 0x7f7aac19f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab447e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac06bb40 0x7f7aac19fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab447e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7aac109510 0x7f7aac199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab447e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac113640 con 0x7f7aac06b180 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab447e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1134c0 con 0x7f7aac06bb40 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab447e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1137c0 con 0x7f7aac109510 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab1a19700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac06bb40 0x7f7aac19fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab221a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac06b180 0x7f7aac19f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab1a19700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac06bb40 0x7f7aac19fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:42826/0 (socket says 172.21.15.59:42826) 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab1a19700 1 -- 172.21.15.59:0/85620856 learned_addr learned my addr 172.21.15.59:0/85620856 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab2a1b700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7aac109510 0x7f7aac199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab221a700 1 -- 172.21.15.59:0/85620856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7aac109510 msgr2=0x7f7aac199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab221a700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7aac109510 0x7f7aac199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab221a700 1 -- 172.21.15.59:0/85620856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac06bb40 msgr2=0x7f7aac19fd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab221a700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac06bb40 0x7f7aac19fd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab221a700 1 -- 172.21.15.59:0/85620856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7aac19a160 con 0x7f7aac06b180 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.130+0000 7f7ab221a700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac06b180 0x7f7aac19f840 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c970 tx=0x7f7aa800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:58.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.131+0000 7f7a9f7fe700 1 -- 172.21.15.59:0/85620856 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800e880 con 0x7f7aac06b180 2024-05-08T05:35:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.131+0000 7f7ab447e700 1 -- 172.21.15.59:0/85620856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7aac19a450 con 0x7f7aac06b180 2024-05-08T05:35:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.131+0000 7f7a9f7fe700 1 -- 172.21.15.59:0/85620856 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa800f450 con 0x7f7aac06b180 2024-05-08T05:35:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.131+0000 7f7a9f7fe700 1 -- 172.21.15.59:0/85620856 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa8018660 con 0x7f7aac06b180 2024-05-08T05:35:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.131+0000 7f7ab447e700 1 -- 172.21.15.59:0/85620856 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7aac1a6710 con 0x7f7aac06b180 2024-05-08T05:35:58.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.133+0000 7f7a9f7fe700 1 -- 172.21.15.59:0/85620856 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f7aa800f600 con 0x7f7aac06b180 2024-05-08T05:35:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.134+0000 7f7a9f7fe700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7a980b29f0 0x7f7a980b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.134+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7a980b29f0 0x7f7a980b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.134+0000 7f7a9f7fe700 1 -- 172.21.15.59:0/85620856 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7aa80e26f0 con 0x7f7aac06b180 2024-05-08T05:35:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7a980b29f0 0x7f7a980b4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0003800 tx=0x7f7aa0005bc0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:58.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7abed11b80 1 Processor -- start 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7abed11b80 1 -- start start 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 0x55f72887cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 0x55f728876ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728883580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7abed11b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f7287c8a40 con 0x55f728877430 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7abed11b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f72872dea0 con 0x55f728876a70 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7abed11b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f7287c4a00 con 0x55f72887c7c0 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7ab2a1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728883580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7ab2a1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728883580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46670/0 (socket says 172.21.15.59:46670) 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7ab2a1b700 1 -- 172.21.15.59:0/3541055906 learned_addr learned my addr 172.21.15.59:0/3541055906 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.135+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/3541055906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 0x55f728876ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7ab221a700 1 --2- 172.21.15.59:0/3541055906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 0x55f72887cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7ab2a1b700 1 -- 172.21.15.59:0/3541055906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 msgr2=0x55f72887cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:58.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7ab2a1b700 1 --2- 172.21.15.59:0/3541055906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 0x55f72887cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7ab2a1b700 1 -- 172.21.15.59:0/3541055906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 msgr2=0x55f728876ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7ab2a1b700 1 --2- 172.21.15.59:0/3541055906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 0x55f728876ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7ab2a1b700 1 -- 172.21.15.59:0/3541055906 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7287159d0 con 0x55f728877430 2024-05-08T05:35:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7ab2a1b700 1 --2- 172.21.15.59:0/3541055906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728883580 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4003f90 tx=0x7f7aa4009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ba6c71ab642792 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7a9cd70700 1 -- 172.21.15.59:0/3541055906 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4004730 con 0x55f728877430 2024-05-08T05:35:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7a9cd70700 1 -- 172.21.15.59:0/3541055906 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa401a070 con 0x55f728877430 2024-05-08T05:35:58.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.136+0000 7f7a9cd70700 1 -- 172.21.15.59:0/3541055906 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa40156e0 con 0x55f728877430 2024-05-08T05:35:58.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 -- 172.21.15.59:0/3541055906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 msgr2=0x55f728883580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3541055906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728883580 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4003f90 tx=0x7f7aa4009c20 comp rx=0 tx=0).stop 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 -- 172.21.15.59:0/3541055906 shutdown_connections 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3541055906 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728883580 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3541055906 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 0x55f72887cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3541055906 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 0x55f728876ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 -- 172.21.15.59:0/3541055906 >> 172.21.15.59:0/3541055906 conn(0x55f7287a2c70 msgr2=0x55f7288717f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 -- 172.21.15.59:0/3541055906 shutdown_connections 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 -- 172.21.15.59:0/3541055906 wait complete. 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.137+0000 7f7abed11b80 1 -- start start 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7abed11b80 1 Processor -- start 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 0x55f728883120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728877430 0x55f7288804c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.142 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f72887c7c0 0x55f728880a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7abed11b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f728883ac0 con 0x55f72887c7c0 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7abed11b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f728781fd0 con 0x55f728876a70 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7abed11b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f728883c40 con 0x55f728877430 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7abed11b80 10 client.0 _ll_register_callbacks cb 0x55f72875c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab2a1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f72887c7c0 0x55f728880a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab2a1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f72887c7c0 0x55f728880a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46680/0 (socket says 172.21.15.59:46680) 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab2a1b700 1 -- 172.21.15.59:0/3739486301 learned_addr learned my addr 172.21.15.59:0/3739486301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab221a700 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 0x55f728883120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728877430 0x55f7288804c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab2a1b700 1 -- 172.21.15.59:0/3739486301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728877430 msgr2=0x55f7288804c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab2a1b700 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728877430 0x55f7288804c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab2a1b700 1 -- 172.21.15.59:0/3739486301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 msgr2=0x55f728883120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab2a1b700 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 0x55f728883120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.143 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.138+0000 7f7ab2a1b700 1 -- 172.21.15.59:0/3739486301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f72877f860 con 0x55f72887c7c0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7ab2a1b700 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f72887c7c0 0x55f728880a00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4000f80 tx=0x7f7aa4007c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb39ba76a9212fc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7a95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7a95ffb700 1 -- 172.21.15.59:0/3739486301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4007500 con 0x55f72887c7c0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7a95ffb700 1 -- 172.21.15.59:0/3739486301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7aa40076a0 con 0x55f72887c7c0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7abed11b80 1 -- 172.21.15.59:0/3739486301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa401a070 con 0x55f72887c7c0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7a95ffb700 1 -- 172.21.15.59:0/3739486301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7aa40078a0 con 0x55f72887c7c0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7abed11b80 10 client.16233 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7abed11b80 1 -- 172.21.15.59:0/3739486301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f72888e1a0 con 0x55f72887c7c0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7a95ffb700 1 -- 172.21.15.59:0/3739486301 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa401a070 con 0x55f72887c7c0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7a95ffb700 1 -- 172.21.15.59:0/3739486301 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f7aa4015480 con 0x55f72887c7c0 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7abed11b80 10 client.16233 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7abed11b80 10 client.16233 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7abed11b80 20 client.16233 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:58.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f728890720 0x55f728892b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7abed11b80 4 client.16233 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.139+0000 7f7abed11b80 1 -- 172.21.15.59:0/3739486301 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f7287d6420 con 0x55f728890720 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.140+0000 7f7ab221a700 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f728890720 0x55f728892b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.140+0000 7f7ab221a700 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f728890720 0x55f728892b10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1a4c7232b8b3c86 server_cookie=e9291bcf6f20f9c8 in_seq=0 out_seq=0 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.140+0000 7f7a95ffb700 10 client.16233 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7a95ffb700 1 -- 172.21.15.59:0/3739486301 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f7287d6420 con 0x55f728890720 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7a95ffb700 10 client.16233 handle_command_reply: tid=0 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 client.16233 shutdown 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 -- 172.21.15.59:0/3739486301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f72887c7c0 msgr2=0x55f728880a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f72887c7c0 0x55f728880a00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4000f80 tx=0x7f7aa4007c70 comp rx=0 tx=0).stop 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 -- 172.21.15.59:0/3739486301 shutdown_connections 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f728890720 0x55f728892b10 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f72887c7c0 0x55f728880a00 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728876a70 0x55f728883120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 --2- 172.21.15.59:0/3739486301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728877430 0x55f7288804c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7a95ffb700 0 client.16233 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 -- 172.21.15.59:0/3739486301 >> 172.21.15.59:0/3739486301 conn(0x55f7287a2c70 msgr2=0x55f72887bb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 -- 172.21.15.59:0/3739486301 shutdown_connections 2024-05-08T05:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 1 -- 172.21.15.59:0/3739486301 wait complete. 2024-05-08T05:35:58.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.141+0000 7f7abed11b80 20 client.16233 trim_cache size 0 max 16384 2024-05-08T05:35:58.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7abed11b80 1 Processor -- start 2024-05-08T05:35:58.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7abed11b80 1 -- start start 2024-05-08T05:35:58.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728876a70 0x55f72887b980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728898340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.166 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f72887c7c0 0x55f728898880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7abed11b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1af400 con 0x55f728877430 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7abed11b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1b0f40 con 0x55f72887c7c0 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7abed11b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1afad0 con 0x55f728876a70 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7ab1a19700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728898340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7ab1a19700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728898340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46696/0 (socket says 172.21.15.59:46696) 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7ab1a19700 1 -- 172.21.15.59:0/2847820554 learned_addr learned my addr 172.21.15.59:0/2847820554 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.161+0000 7f7ab221a700 1 --2- 172.21.15.59:0/2847820554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728876a70 0x55f72887b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7ab2a1b700 1 --2- 172.21.15.59:0/2847820554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f72887c7c0 0x55f728898880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7ab1a19700 1 -- 172.21.15.59:0/2847820554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728876a70 msgr2=0x55f72887b980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/2847820554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728876a70 0x55f72887b980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7ab1a19700 1 -- 172.21.15.59:0/2847820554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f72887c7c0 msgr2=0x55f728898880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/2847820554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f72887c7c0 0x55f728898880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7ab1a19700 1 -- 172.21.15.59:0/2847820554 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f72887afe0 con 0x55f728877430 2024-05-08T05:35:58.167 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/2847820554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728898340 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00098e0 tx=0x7f7aa001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=402148eb91f0def4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7a97fff700 1 -- 172.21.15.59:0/2847820554 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa0034070 con 0x55f728877430 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7a97fff700 1 -- 172.21.15.59:0/2847820554 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa0024c70 con 0x55f728877430 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.162+0000 7f7a97fff700 1 -- 172.21.15.59:0/2847820554 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa000fc40 con 0x55f728877430 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 -- 172.21.15.59:0/2847820554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 msgr2=0x55f728898340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 --2- 172.21.15.59:0/2847820554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728898340 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f7aa00098e0 tx=0x7f7aa001f000 comp rx=0 tx=0).stop 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 -- 172.21.15.59:0/2847820554 shutdown_connections 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 --2- 172.21.15.59:0/2847820554 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f72887c7c0 0x55f728898880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 --2- 172.21.15.59:0/2847820554 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f728876a70 0x55f72887b980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 --2- 172.21.15.59:0/2847820554 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728877430 0x55f728898340 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 -- 172.21.15.59:0/2847820554 >> 172.21.15.59:0/2847820554 conn(0x55f7287a2c70 msgr2=0x55f72887b270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 -- 172.21.15.59:0/2847820554 shutdown_connections 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 -- 172.21.15.59:0/2847820554 wait complete. 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 -- start start 2024-05-08T05:35:58.168 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 Processor -- start 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728876a70 0x55f7288798d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728877430 0x55f728888e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 0x55f7288893d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1b0870 con 0x55f728876a70 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1b01a0 con 0x55f728877430 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f7aac1ad380 con 0x55f72887c7c0 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.163+0000 7f7abed11b80 10 client.0 _ll_register_callbacks cb 0x55f72875c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab221a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728876a70 0x55f7288798d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab221a700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728876a70 0x55f7288798d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46700/0 (socket says 172.21.15.59:46700) 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab221a700 1 -- 172.21.15.59:0/4133322321 learned_addr learned my addr 172.21.15.59:0/4133322321 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab2a1b700 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 0x55f7288893d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728877430 0x55f728888e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab221a700 1 -- 172.21.15.59:0/4133322321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 msgr2=0x55f7288893d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:58.169 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab221a700 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 0x55f7288893d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab221a700 1 -- 172.21.15.59:0/4133322321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728877430 msgr2=0x55f728888e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab221a700 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728877430 0x55f728888e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab221a700 1 -- 172.21.15.59:0/4133322321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7288781a0 con 0x55f728876a70 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728877430 0x55f728888e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7ab221a700 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728876a70 0x55f7288798d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80a9560 tx=0x7f7aa80a9470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b617e6ccde0b03f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7a95ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7a95ffb700 1 -- 172.21.15.59:0/4133322321 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa800f920 con 0x55f728876a70 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7a95ffb700 1 -- 172.21.15.59:0/4133322321 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7f7aa80e22c0 con 0x55f728876a70 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7abed11b80 1 -- 172.21.15.59:0/4133322321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa0024c70 con 0x55f728876a70 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7a95ffb700 1 -- 172.21.15.59:0/4133322321 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f7aa800fac0 con 0x55f728876a70 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7abed11b80 10 client.16239 fetch_fsmap learned FSMap version 36 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.164+0000 7f7abed11b80 1 -- 172.21.15.59:0/4133322321 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f7288fc300 con 0x55f728876a70 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7a95ffb700 1 -- 172.21.15.59:0/4133322321 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa0024c70 con 0x55f728876a70 2024-05-08T05:35:58.170 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7a95ffb700 1 -- 172.21.15.59:0/4133322321 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f7aa8083ce0 con 0x55f728876a70 2024-05-08T05:35:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7abed11b80 10 client.16239 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:35:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7abed11b80 10 client.16239 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:35:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7abed11b80 20 client.16239 populate_metadata read hostname 'smithi059' 2024-05-08T05:35:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7abed11b80 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f7288fc9c0 0x55f7288fede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:35:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7abed11b80 4 client.16239 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "3cff3c00", "release": true, "await": true}] 2024-05-08T05:35:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7abed11b80 1 -- 172.21.15.59:0/4133322321 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "3cff3c00", "release": true, "await": true}) -- 0x55f7287d04c0 con 0x55f7288fc9c0 2024-05-08T05:35:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f7288fc9c0 0x55f7288fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:35:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7a95ffb700 10 client.16239 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:58.171 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.165+0000 7f7ab1a19700 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f7288fc9c0 0x55f7288fede0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b71e032b4a5458c server_cookie=37fc1c0cb13b7f7c in_seq=0 out_seq=0 2024-05-08T05:35:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.362+0000 7f7a95ffb700 1 -- 172.21.15.59:0/4133322321 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55f7287d04c0 con 0x55f7288fc9c0 2024-05-08T05:35:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.362+0000 7f7a95ffb700 10 client.16239 handle_command_reply: tid=0 2024-05-08T05:35:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.362+0000 7f7abed11b80 1 client.16239 shutdown 2024-05-08T05:35:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 -- 172.21.15.59:0/4133322321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728876a70 msgr2=0x55f7288798d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728876a70 0x55f7288798d0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7aa80a9560 tx=0x7f7aa80a9470 comp rx=0 tx=0).stop 2024-05-08T05:35:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 -- 172.21.15.59:0/4133322321 shutdown_connections 2024-05-08T05:35:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f7288fc9c0 0x55f7288fede0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.364 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f72887c7c0 0x55f7288893d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f728876a70 0x55f7288798d0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 --2- 172.21.15.59:0/4133322321 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f728877430 0x55f728888e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 -- 172.21.15.59:0/4133322321 >> 172.21.15.59:0/4133322321 conn(0x55f7287a2c70 msgr2=0x55f72887baa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7a95ffb700 0 client.16239 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 -- 172.21.15.59:0/4133322321 shutdown_connections 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 1 -- 172.21.15.59:0/4133322321 wait complete. 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7abed11b80 20 client.16239 trim_cache size 0 max 16384 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 36, 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:35:58.365 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 20, 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "3cff3c00": { 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 20, 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 85.7, 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:35:58.366 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:35:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:35:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:35:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:35:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:58.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:35:58.367 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7ab447e700 1 -- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7a980b29f0 msgr2=0x7f7a980b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.363+0000 7f7ab447e700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7a980b29f0 0x7f7a980b4eb0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f7aa0003800 tx=0x7f7aa0005bc0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 -- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac06b180 msgr2=0x7f7aac19f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac06b180 0x7f7aac19f840 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f7aa800c970 tx=0x7f7aa800cc80 comp rx=0 tx=0).stop 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 -- 172.21.15.59:0/85620856 shutdown_connections 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f7a980b29f0 0x7f7a980b4eb0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f7aac109510 0x7f7aac199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f7aac06b180 0x7f7aac19f840 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 --2- 172.21.15.59:0/85620856 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f7aac06bb40 0x7f7aac19fd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 -- 172.21.15.59:0/85620856 >> 172.21.15.59:0/85620856 conn(0x7f7aac079350 msgr2=0x7f7aac101080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:35:58.368 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 -- 172.21.15.59:0/85620856 shutdown_connections 2024-05-08T05:35:58.369 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:35:58.364+0000 7f7ab447e700 1 -- 172.21.15.59:0/85620856 wait complete. 2024-05-08T05:35:59.369 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '3cff3c00', seconds elapsed: 42.2 2024-05-08T05:35:59.369 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 25.7 2024-05-08T05:35:59.369 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:35:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:35:59 smithi143 ceph-mon[34337]: pgmap v195: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2024-05-08T05:35:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:35:59 smithi096 ceph-mon[34253]: pgmap v195: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2024-05-08T05:35:59.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:35:59 smithi059 ceph-mon[30304]: pgmap v195: 129 pgs: 129 active+clean; 43 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.6 KiB/s wr, 10 op/s 2024-05-08T05:35:59.681 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:36:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:01 smithi143 ceph-mon[34337]: pgmap v196: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 316 KiB/s wr, 66 op/s 2024-05-08T05:36:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:01 smithi096 ceph-mon[34253]: pgmap v196: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 316 KiB/s wr, 66 op/s 2024-05-08T05:36:01.554 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:01 smithi059 ceph-mon[30304]: pgmap v196: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 497 KiB/s rd, 316 KiB/s wr, 66 op/s 2024-05-08T05:36:01.578 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.578+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/4267149394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc1037b0 msgr2=0x7fdbdc103b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.578+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/4267149394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc1037b0 0x7fdbdc103b90 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8009920 tx=0x7fdbd8009c30 comp rx=0 tx=0).stop 2024-05-08T05:36:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.578+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/4267149394 shutdown_connections 2024-05-08T05:36:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.578+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/4267149394 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbdc108880 0x7fdbdc110400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.578+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/4267149394 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc1037b0 0x7fdbdc103b90 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.578+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/4267149394 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbdc104160 0x7fdbdc108150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.578+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/4267149394 >> 172.21.15.59:0/4267149394 conn(0x7fdbdc07a530 msgr2=0x7fdbdc07a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.578+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/4267149394 shutdown_connections 2024-05-08T05:36:01.579 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.578+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/4267149394 wait complete. 2024-05-08T05:36:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe3f6b700 1 Processor -- start 2024-05-08T05:36:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe3f6b700 1 -- start start 2024-05-08T05:36:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe3f6b700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbdc1037b0 0x7fdbdc19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe3f6b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbdc104160 0x7fdbdc19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:01.581 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe3f6b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc108880 0x7fdbdc199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe3f6b700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc113650 con 0x7fdbdc104160 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe3f6b700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc1134d0 con 0x7fdbdc108880 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe3f6b700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdbdc1137d0 con 0x7fdbdc1037b0 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe1506700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbdc104160 0x7fdbdc19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe1506700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbdc104160 0x7fdbdc19fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49784/0 (socket says 172.21.15.59:49784) 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe1d07700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbdc1037b0 0x7fdbdc19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe1506700 1 -- 172.21.15.59:0/114066322 learned_addr learned my addr 172.21.15.59:0/114066322 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.579+0000 7fdbe2508700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc108880 0x7fdbdc199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbe1d07700 1 -- 172.21.15.59:0/114066322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc108880 msgr2=0x7fdbdc199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbe1d07700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc108880 0x7fdbdc199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbe1d07700 1 -- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbdc104160 msgr2=0x7fdbdc19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbe1d07700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbdc104160 0x7fdbdc19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:01.582 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbe1d07700 1 -- 172.21.15.59:0/114066322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdbdc19a0d0 con 0x7fdbdc1037b0 2024-05-08T05:36:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbe1506700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbdc104160 0x7fdbdc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:36:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbe1d07700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbdc1037b0 0x7fdbdc19f7b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8000c00 tx=0x7fdbd800f950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbd2ffd700 1 -- 172.21.15.59:0/114066322 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8027070 con 0x7fdbdc1037b0 2024-05-08T05:36:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdbdc19a360 con 0x7fdbdc1037b0 2024-05-08T05:36:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbd2ffd700 1 -- 172.21.15.59:0/114066322 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdbd80054f0 con 0x7fdbdc1037b0 2024-05-08T05:36:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbd2ffd700 1 -- 172.21.15.59:0/114066322 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdbd8004930 con 0x7fdbdc1037b0 2024-05-08T05:36:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.580+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdbdc1a6680 con 0x7fdbdc1037b0 2024-05-08T05:36:01.583 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.581+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdbdc104c10 con 0x7fdbdc1037b0 2024-05-08T05:36:01.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.583+0000 7fdbd2ffd700 1 -- 172.21.15.59:0/114066322 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdbd802a070 con 0x7fdbdc1037b0 2024-05-08T05:36:01.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.583+0000 7fdbd2ffd700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdbc80b29f0 0x7fdbc80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:01.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.584+0000 7fdbe1506700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdbc80b29f0 0x7fdbc80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:01.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.584+0000 7fdbd2ffd700 1 -- 172.21.15.59:0/114066322 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9408+0+0 (secure 0 0 0) 0x7fdbd80e3360 con 0x7fdbdc1037b0 2024-05-08T05:36:01.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.584+0000 7fdbe1506700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdbc80b29f0 0x7fdbc80b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc103170 tx=0x7fdbcc009310 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:01.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.587+0000 7fdbd2ffd700 1 -- 172.21.15.59:0/114066322 <== mon.1 v2:172.21.15.143:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdbd8013030 con 0x7fdbdc1037b0 2024-05-08T05:36:01.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.937+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdbdc02e510 con 0x7fdbdc1037b0 2024-05-08T05:36:01.939 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.939+0000 7fdbd2ffd700 1 -- 172.21.15.59:0/114066322 <== mon.1 v2:172.21.15.143:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11929 (secure 0 0 0) 0x7fdbd80aaa60 con 0x7fdbdc1037b0 2024-05-08T05:36:01.939 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:36:01.939 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:34:26.942121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24443,24473,24421,24433,24419]},"id":1}]} 2024-05-08T05:36:01.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdbc80b29f0 msgr2=0x7fdbc80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:01.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdbc80b29f0 0x7fdbc80b4eb0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7fdbdc103170 tx=0x7fdbcc009310 comp rx=0 tx=0).stop 2024-05-08T05:36:01.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbdc1037b0 msgr2=0x7fdbdc19f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:01.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbdc1037b0 0x7fdbdc19f7b0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fdbd8000c00 tx=0x7fdbd800f950 comp rx=0 tx=0).stop 2024-05-08T05:36:01.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 shutdown_connections 2024-05-08T05:36:01.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdbc80b29f0 0x7fdbc80b4eb0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:01.942 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdbdc108880 0x7fdbdc199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:01.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdbdc1037b0 0x7fdbdc19f7b0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:01.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 --2- 172.21.15.59:0/114066322 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdbdc104160 0x7fdbdc19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:01.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 >> 172.21.15.59:0/114066322 conn(0x7fdbdc07a530 msgr2=0x7fdbdc1018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:01.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 shutdown_connections 2024-05-08T05:36:01.943 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:01.941+0000 7fdbe3f6b700 1 -- 172.21.15.59:0/114066322 wait complete. 2024-05-08T05:36:01.943 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 36 2024-05-08T05:36:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:02 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/114066322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:36:02.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:02 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/114066322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:36:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 -- 172.21.15.59:0/3153812115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0358102100 msgr2=0x7f035810ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 --2- 172.21.15.59:0/3153812115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0358102100 0x7f035810ee50 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f0350009ca0 tx=0x7f035002a8f0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 -- 172.21.15.59:0/3153812115 shutdown_connections 2024-05-08T05:36:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 --2- 172.21.15.59:0/3153812115 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f035810f390 0x7f0358111780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 --2- 172.21.15.59:0/3153812115 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f0358102100 0x7f035810ee50 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 --2- 172.21.15.59:0/3153812115 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03581017e0 0x7f0358101bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.549 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 -- 172.21.15.59:0/3153812115 >> 172.21.15.59:0/3153812115 conn(0x7f0358077c30 msgr2=0x7f0358078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 -- 172.21.15.59:0/3153812115 shutdown_connections 2024-05-08T05:36:02.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 -- 172.21.15.59:0/3153812115 wait complete. 2024-05-08T05:36:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 Processor -- start 2024-05-08T05:36:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.546+0000 7f035e864640 1 -- start start 2024-05-08T05:36:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035e864640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03581017e0 0x7f035819b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035e864640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0358102100 0x7f035819ba30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035e864640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035810f390 0x7f035819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035e864640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f0358114fd0 con 0x7f035810f390 2024-05-08T05:36:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035e864640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f0358114e50 con 0x7f0358102100 2024-05-08T05:36:02.551 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035e864640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f0358115150 con 0x7f03581017e0 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f0357fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03581017e0 0x7f035819b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f03577fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0358102100 0x7f035819ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f0357fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03581017e0 0x7f035819b4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:34922/0 (socket says 172.21.15.59:34922) 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f0357fff640 1 -- 172.21.15.59:0/3787178158 learned_addr learned my addr 172.21.15.59:0/3787178158 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035ce01640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035810f390 0x7f035819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035ce01640 1 -- 172.21.15.59:0/3787178158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03581017e0 msgr2=0x7f035819b4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035ce01640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03581017e0 0x7f035819b4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035ce01640 1 -- 172.21.15.59:0/3787178158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0358102100 msgr2=0x7f035819ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.547+0000 7f035ce01640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0358102100 0x7f035819ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.548+0000 7f035ce01640 1 -- 172.21.15.59:0/3787178158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03581a04e0 con 0x7f035810f390 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.548+0000 7f035ce01640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035810f390 0x7f035819fdb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f035806afd0 tx=0x7f034c00b870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.548+0000 7f03557fa640 1 -- 172.21.15.59:0/3787178158 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c015020 con 0x7f035810f390 2024-05-08T05:36:02.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.548+0000 7f035e864640 1 -- 172.21.15.59:0/3787178158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03581a07d0 con 0x7f035810f390 2024-05-08T05:36:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.548+0000 7f03557fa640 1 -- 172.21.15.59:0/3787178158 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034c011ce0 con 0x7f035810f390 2024-05-08T05:36:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.548+0000 7f03557fa640 1 -- 172.21.15.59:0/3787178158 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c010db0 con 0x7f035810f390 2024-05-08T05:36:02.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.548+0000 7f035e864640 1 -- 172.21.15.59:0/3787178158 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03581a8060 con 0x7f035810f390 2024-05-08T05:36:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.550+0000 7f036157f740 1 Processor -- start 2024-05-08T05:36:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.550+0000 7f03557fa640 1 -- 172.21.15.59:0/3787178158 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f034c018020 con 0x7f035810f390 2024-05-08T05:36:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f036157f740 1 -- start start 2024-05-08T05:36:02.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f036157f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 0x55fc4ac722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f036157f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac7cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f036157f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac7f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f036157f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fc4abd75d0 con 0x55fc4ac72830 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f036157f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc4ab90890 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f036157f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fc4abd3590 con 0x55fc4ac71f10 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f0357fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac7cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f0357fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac7cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49808/0 (socket says 172.21.15.59:49808) 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f0357fff640 1 -- 172.21.15.59:0/1673534812 learned_addr learned my addr 172.21.15.59:0/1673534812 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f03577fe640 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 0x55fc4ac722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f03577fe640 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac7f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f0357fff640 1 -- 172.21.15.59:0/1673534812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 msgr2=0x55fc4ac722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f0357fff640 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 0x55fc4ac722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.560 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f0357fff640 1 -- 172.21.15.59:0/1673534812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 msgr2=0x55fc4ac7f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f0357fff640 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac7f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f0357fff640 1 -- 172.21.15.59:0/1673534812 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4a8e9210 con 0x55fc4ac72830 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f03577fe640 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 0x55fc4ac722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.551+0000 7f03557fa640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03400b2a10 0x7f03400b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.552+0000 7f0357fff640 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac7cbb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0348009b10 tx=0x7f034800e1d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c34b876c1a24c208 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.552+0000 7f03577fe640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03400b2a10 0x7f03400b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.552+0000 7f03557fa640 1 -- 172.21.15.59:0/3787178158 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9584+0+0 (secure 0 0 0) 0x7f034c0e2030 con 0x7f035810f390 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.552+0000 7f033e7fc640 1 -- 172.21.15.59:0/1673534812 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348017070 con 0x55fc4ac72830 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.552+0000 7f033e7fc640 1 -- 172.21.15.59:0/1673534812 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034800ebe0 con 0x55fc4ac72830 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.552+0000 7f033e7fc640 1 -- 172.21.15.59:0/1673534812 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348012630 con 0x55fc4ac72830 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.552+0000 7f036157f740 1 -- 172.21.15.59:0/1673534812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 msgr2=0x55fc4ac7cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.552+0000 7f03577fe640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03400b2a10 0x7f03400b4ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0350000f80 tx=0x7f035003f040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:02.561 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac7cbb0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f0348009b10 tx=0x7f034800e1d0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 -- 172.21.15.59:0/1673534812 shutdown_connections 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac7f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac7cbb0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 --2- 172.21.15.59:0/1673534812 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 0x55fc4ac722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 -- 172.21.15.59:0/1673534812 >> 172.21.15.59:0/1673534812 conn(0x55fc4ac6e420 msgr2=0x55fc4ac6e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 -- 172.21.15.59:0/1673534812 shutdown_connections 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 -- 172.21.15.59:0/1673534812 wait complete. 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 -- start start 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 Processor -- start 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac71f10 0x55fc4ac7a020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 0x55fc4ac7a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.553+0000 7f036157f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac7aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.562 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f036157f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fc4ac7fd20 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f036157f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc4ab94370 con 0x55fc4ac71f10 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f036157f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fc4ac7fea0 con 0x55fc4ac72830 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f036157f740 10 client.0 _ll_register_callbacks cb 0x55fc4a8e8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f035ce01640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac7aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f0357fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac71f10 0x55fc4ac7a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f0357fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac71f10 0x55fc4ac7a020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:42946/0 (socket says 172.21.15.59:42946) 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f0357fff640 1 -- 172.21.15.59:0/93614536 learned_addr learned my addr 172.21.15.59:0/93614536 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f03577fe640 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 0x55fc4ac7a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f035ce01640 1 -- 172.21.15.59:0/93614536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 msgr2=0x55fc4ac7a560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f035ce01640 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 0x55fc4ac7a560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f035ce01640 1 -- 172.21.15.59:0/93614536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac71f10 msgr2=0x55fc4ac7a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f035ce01640 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac71f10 0x55fc4ac7a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f035ce01640 1 -- 172.21.15.59:0/93614536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4a9b5620 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f03577fe640 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 0x55fc4ac7a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f035ce01640 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac7aaa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f034c0df470 tx=0x7f034c0a8950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ad8dfa93d39864 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f03377fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f03377fe640 1 -- 172.21.15.59:0/93614536 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034c0a9700 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f03377fe640 1 -- 172.21.15.59:0/93614536 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9584+0+0 (secure 0 0 0) 0x7f034c011830 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f036157f740 1 -- 172.21.15.59:0/93614536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f034800ebe0 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f03377fe640 1 -- 172.21.15.59:0/93614536 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f034c0a9000 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f036157f740 10 client.16257 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.554+0000 7f036157f740 1 -- 172.21.15.59:0/93614536 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fc4ac94400 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f03377fe640 1 -- 172.21.15.59:0/93614536 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f034800ebe0 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f03377fe640 1 -- 172.21.15.59:0/93614536 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f034c0a4a00 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f036157f740 10 client.16257 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:02.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f036157f740 10 client.16257 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f036157f740 20 client.16257 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f036157f740 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc4ac958c0 0x55fc4ac97d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f036157f740 4 client.16257 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f036157f740 1 -- 172.21.15.59:0/93614536 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc4abe51e0 con 0x55fc4ac958c0 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f0357fff640 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc4ac958c0 0x55fc4ac97d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f03377fe640 10 client.16257 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.555+0000 7f0357fff640 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc4ac958c0 0x55fc4ac97d10 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17c10540237be02a server_cookie=cfdc9aebf90ef1ec in_seq=0 out_seq=0 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.556+0000 7f03377fe640 1 -- 172.21.15.59:0/93614536 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc4abe51e0 con 0x55fc4ac958c0 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.556+0000 7f03377fe640 10 client.16257 handle_command_reply: tid=0 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.556+0000 7f036157f740 1 client.16257 shutdown 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.556+0000 7f036157f740 1 -- 172.21.15.59:0/93614536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac7d0f0 msgr2=0x55fc4ac7aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.556+0000 7f036157f740 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac7aaa0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f034c0df470 tx=0x7f034c0a8950 comp rx=0 tx=0).stop 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f036157f740 1 -- 172.21.15.59:0/93614536 shutdown_connections 2024-05-08T05:36:02.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f036157f740 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc4ac958c0 0x55fc4ac97d10 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f036157f740 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac7aaa0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f036157f740 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 0x55fc4ac7a560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f03377fe640 0 client.16257 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f036157f740 1 --2- 172.21.15.59:0/93614536 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac71f10 0x55fc4ac7a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f036157f740 1 -- 172.21.15.59:0/93614536 >> 172.21.15.59:0/93614536 conn(0x55fc4ac6e420 msgr2=0x55fc4ac74150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f036157f740 1 -- 172.21.15.59:0/93614536 shutdown_connections 2024-05-08T05:36:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f036157f740 1 -- 172.21.15.59:0/93614536 wait complete. 2024-05-08T05:36:02.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.557+0000 7f036157f740 20 client.16257 trim_cache size 0 max 16384 2024-05-08T05:36:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f036157f740 1 Processor -- start 2024-05-08T05:36:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f036157f740 1 -- start start 2024-05-08T05:36:02.577 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f036157f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac71f10 0x55fc4ac748d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f036157f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 0x55fc4ac9d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f036157f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac9db60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f036157f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03581add40 con 0x55fc4ac71f10 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f036157f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03581af880 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f036157f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f03581ae410 con 0x55fc4ac72830 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f0357fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac71f10 0x55fc4ac748d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f0357fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac71f10 0x55fc4ac748d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49830/0 (socket says 172.21.15.59:49830) 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.572+0000 7f0357fff640 1 -- 172.21.15.59:0/3894567580 learned_addr learned my addr 172.21.15.59:0/3894567580 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f035ce01640 1 --2- 172.21.15.59:0/3894567580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac9db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f03577fe640 1 --2- 172.21.15.59:0/3894567580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 0x55fc4ac9d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f0357fff640 1 -- 172.21.15.59:0/3894567580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 msgr2=0x55fc4ac9d620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f0357fff640 1 --2- 172.21.15.59:0/3894567580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 0x55fc4ac9d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.578 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f0357fff640 1 -- 172.21.15.59:0/3894567580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 msgr2=0x55fc4ac9db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f0357fff640 1 --2- 172.21.15.59:0/3894567580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac9db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f0357fff640 1 -- 172.21.15.59:0/3894567580 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc4ac72660 con 0x55fc4ac71f10 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f0357fff640 1 --2- 172.21.15.59:0/3894567580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac71f10 0x55fc4ac748d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0348013ec0 tx=0x7f0348008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfef38f42575400 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f033dffb640 1 -- 172.21.15.59:0/3894567580 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f034802b020 con 0x55fc4ac71f10 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f033dffb640 1 -- 172.21.15.59:0/3894567580 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03480139e0 con 0x55fc4ac71f10 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.573+0000 7f033dffb640 1 -- 172.21.15.59:0/3894567580 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0348017040 con 0x55fc4ac71f10 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 -- 172.21.15.59:0/3894567580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac71f10 msgr2=0x55fc4ac748d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 --2- 172.21.15.59:0/3894567580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac71f10 0x55fc4ac748d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f0348013ec0 tx=0x7f0348008040 comp rx=0 tx=0).stop 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 -- 172.21.15.59:0/3894567580 shutdown_connections 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 --2- 172.21.15.59:0/3894567580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac9db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 --2- 172.21.15.59:0/3894567580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac72830 0x55fc4ac9d620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 --2- 172.21.15.59:0/3894567580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac71f10 0x55fc4ac748d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.579 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 -- 172.21.15.59:0/3894567580 >> 172.21.15.59:0/3894567580 conn(0x55fc4ac6e420 msgr2=0x55fc4ac6d580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 -- 172.21.15.59:0/3894567580 shutdown_connections 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 -- 172.21.15.59:0/3894567580 wait complete. 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 -- start start 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 Processor -- start 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.574+0000 7f036157f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 0x55fc4ac895b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f036157f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac89af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f036157f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac8a030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f036157f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f03581af1b0 con 0x55fc4ac72830 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f036157f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f03581aeae0 con 0x55fc4ac7d0f0 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f036157f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f03581abd20 con 0x55fc4ac71f10 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03577fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac89af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f036157f740 10 client.0 _ll_register_callbacks cb 0x7f034c00bef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03577fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac89af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49840/0 (socket says 172.21.15.59:49840) 2024-05-08T05:36:02.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03577fe640 1 -- 172.21.15.59:0/1684856194 learned_addr learned my addr 172.21.15.59:0/1684856194 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f035ce01640 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac8a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f0357fff640 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 0x55fc4ac895b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03577fe640 1 -- 172.21.15.59:0/1684856194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 msgr2=0x55fc4ac895b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03577fe640 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 0x55fc4ac895b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03577fe640 1 -- 172.21.15.59:0/1684856194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 msgr2=0x55fc4ac8a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03577fe640 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac8a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03577fe640 1 -- 172.21.15.59:0/1684856194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc4acff2a0 con 0x55fc4ac72830 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03577fe640 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac89af0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0350035000 tx=0x7f0350005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49442eb4119cedc8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03377fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03377fe640 1 -- 172.21.15.59:0/1684856194 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0350004450 con 0x55fc4ac72830 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03377fe640 1 -- 172.21.15.59:0/1684856194 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9584+0+0 (secure 0 0 0) 0x7f03500045f0 con 0x55fc4ac72830 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f036157f740 1 -- 172.21.15.59:0/1684856194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f03480139e0 con 0x55fc4ac72830 2024-05-08T05:36:02.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f03377fe640 1 -- 172.21.15.59:0/1684856194 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f03500047f0 con 0x55fc4ac72830 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f036157f740 10 client.16263 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.575+0000 7f036157f740 1 -- 172.21.15.59:0/1684856194 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55fc4ad005c0 con 0x55fc4ac72830 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f03377fe640 1 -- 172.21.15.59:0/1684856194 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03480139e0 con 0x55fc4ac72830 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f03377fe640 1 -- 172.21.15.59:0/1684856194 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f0350030860 con 0x55fc4ac72830 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f036157f740 10 client.16263 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f036157f740 10 client.16263 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f036157f740 20 client.16263 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f036157f740 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc4ac958c0 0x55fc4ad02db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f036157f740 4 client.16263 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f036157f740 1 -- 172.21.15.59:0/1684856194 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fc4abdf050 con 0x55fc4ac958c0 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f0357fff640 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc4ac958c0 0x55fc4ad02db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f03377fe640 10 client.16263 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:02.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.576+0000 7f0357fff640 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc4ac958c0 0x55fc4ad02db0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=636bcc166699ee95 server_cookie=a96ffc90489f6c9d in_seq=0 out_seq=0 2024-05-08T05:36:02.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:02 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/114066322' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:36:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.651+0000 7f03377fe640 1 -- 172.21.15.59:0/1684856194 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fc4abdf050 con 0x55fc4ac958c0 2024-05-08T05:36:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.651+0000 7f03377fe640 10 client.16263 handle_command_reply: tid=0 2024-05-08T05:36:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.651+0000 7f036157f740 1 client.16263 shutdown 2024-05-08T05:36:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.651+0000 7f036157f740 1 -- 172.21.15.59:0/1684856194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 msgr2=0x55fc4ac89af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.651+0000 7f036157f740 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac89af0 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7f0350035000 tx=0x7f0350005ef0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.651+0000 7f036157f740 1 -- 172.21.15.59:0/1684856194 shutdown_connections 2024-05-08T05:36:02.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.651+0000 7f036157f740 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc4ac958c0 0x55fc4ad02db0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.652+0000 7f03377fe640 0 client.16263 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.652+0000 7f036157f740 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc4ac7d0f0 0x55fc4ac8a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.652+0000 7f036157f740 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc4ac72830 0x55fc4ac89af0 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.652+0000 7f036157f740 1 --2- 172.21.15.59:0/1684856194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc4ac71f10 0x55fc4ac895b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.652+0000 7f036157f740 1 -- 172.21.15.59:0/1684856194 >> 172.21.15.59:0/1684856194 conn(0x55fc4ac6e420 msgr2=0x55fc4ac6ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.652+0000 7f036157f740 1 -- 172.21.15.59:0/1684856194 shutdown_connections 2024-05-08T05:36:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.652+0000 7f036157f740 1 -- 172.21.15.59:0/1684856194 wait complete. 2024-05-08T05:36:02.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.653+0000 7f036157f740 20 client.16263 trim_cache size 0 max 16384 2024-05-08T05:36:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:36:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:36:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:36:02.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:36:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.655+0000 7f035e864640 1 -- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03400b2a10 msgr2=0x7f03400b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.655+0000 7f035e864640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03400b2a10 0x7f03400b4ed0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f0350000f80 tx=0x7f035003f040 comp rx=0 tx=0).stop 2024-05-08T05:36:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.655+0000 7f035e864640 1 -- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035810f390 msgr2=0x7f035819fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:02.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.655+0000 7f035e864640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035810f390 0x7f035819fdb0 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f035806afd0 tx=0x7f034c00b870 comp rx=0 tx=0).stop 2024-05-08T05:36:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.656+0000 7f035e864640 1 -- 172.21.15.59:0/3787178158 shutdown_connections 2024-05-08T05:36:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.656+0000 7f035e864640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f03400b2a10 0x7f03400b4ed0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.656+0000 7f035e864640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f035810f390 0x7f035819fdb0 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.656+0000 7f035e864640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f0358102100 0x7f035819ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.656+0000 7f035e864640 1 --2- 172.21.15.59:0/3787178158 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f03581017e0 0x7f035819b4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:02.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.656+0000 7f035e864640 1 -- 172.21.15.59:0/3787178158 >> 172.21.15.59:0/3787178158 conn(0x7f0358077c30 msgr2=0x7f0358110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.656+0000 7f035e864640 1 -- 172.21.15.59:0/3787178158 shutdown_connections 2024-05-08T05:36:02.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:02.656+0000 7f035e864640 1 -- 172.21.15.59:0/3787178158 wait complete. 2024-05-08T05:36:03.172 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db / --timeout 60 --expiration 85.7 --await-for 42.0' 2024-05-08T05:36:03.173 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db / --timeout 60 --expiration 85.7 --await-for 42.0 2024-05-08T05:36:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:03 smithi143 ceph-mon[34337]: pgmap v197: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 307 KiB/s wr, 59 op/s 2024-05-08T05:36:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:03 smithi143 ceph-mon[34337]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T05:36:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:03 smithi096 ceph-mon[34253]: pgmap v197: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 307 KiB/s wr, 59 op/s 2024-05-08T05:36:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:03 smithi096 ceph-mon[34253]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T05:36:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:03 smithi059 ceph-mon[30304]: pgmap v197: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 488 KiB/s rd, 307 KiB/s wr, 59 op/s 2024-05-08T05:36:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:03 smithi059 ceph-mon[30304]: osdmap e92: 12 total, 12 up, 12 in 2024-05-08T05:36:03.679 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:36:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:05 smithi059 ceph-mon[30304]: pgmap v199: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 368 KiB/s wr, 71 op/s 2024-05-08T05:36:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:05 smithi096 ceph-mon[34253]: pgmap v199: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 368 KiB/s wr, 71 op/s 2024-05-08T05:36:05.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:05 smithi143 ceph-mon[34337]: pgmap v199: 129 pgs: 129 active+clean; 46 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 585 KiB/s rd, 368 KiB/s wr, 71 op/s 2024-05-08T05:36:06.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.084+0000 7f4274604700 1 -- 172.21.15.59:0/2750152714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f426c1076d0 msgr2=0x7f426c10f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:06.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.084+0000 7f4274604700 1 --2- 172.21.15.59:0/2750152714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f426c1076d0 0x7f426c10f2a0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f4264009fc0 tx=0x7f426400b330 comp rx=0 tx=0).stop 2024-05-08T05:36:06.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.084+0000 7f4274604700 1 -- 172.21.15.59:0/2750152714 shutdown_connections 2024-05-08T05:36:06.090 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.084+0000 7f4274604700 1 --2- 172.21.15.59:0/2750152714 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f426c1076d0 0x7f426c10f2a0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.084+0000 7f4274604700 1 --2- 172.21.15.59:0/2750152714 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f426c102600 0x7f426c1029e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.084+0000 7f4274604700 1 --2- 172.21.15.59:0/2750152714 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c102fb0 0x7f426c106ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.084+0000 7f4274604700 1 -- 172.21.15.59:0/2750152714 >> 172.21.15.59:0/2750152714 conn(0x7f426c079340 msgr2=0x7f426c079750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:06.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.085+0000 7f4274604700 1 -- 172.21.15.59:0/2750152714 shutdown_connections 2024-05-08T05:36:06.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.085+0000 7f4274604700 1 -- 172.21.15.59:0/2750152714 wait complete. 2024-05-08T05:36:06.091 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.086+0000 7f4274604700 1 Processor -- start 2024-05-08T05:36:06.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.086+0000 7f4274604700 1 -- start start 2024-05-08T05:36:06.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.086+0000 7f4274604700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f426c102600 0x7f426c1148f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.086+0000 7f4274604700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f426c102fb0 0x7f426c112d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.086+0000 7f4274604700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c1076d0 0x7f426c1132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.092 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.086+0000 7f4274604700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f426c1125c0 con 0x7f426c1076d0 2024-05-08T05:36:06.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.086+0000 7f4274604700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f426c112440 con 0x7f426c102600 2024-05-08T05:36:06.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.086+0000 7f4274604700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f426c112740 con 0x7f426c102fb0 2024-05-08T05:36:06.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f42723a0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f426c102600 0x7f426c1148f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f4272ba1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c1076d0 0x7f426c1132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f4272ba1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c1076d0 0x7f426c1132c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49854/0 (socket says 172.21.15.59:49854) 2024-05-08T05:36:06.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f4272ba1700 1 -- 172.21.15.59:0/2605832337 learned_addr learned my addr 172.21.15.59:0/2605832337 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:06.093 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f4271b9f700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f426c102fb0 0x7f426c112d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f42723a0700 1 -- 172.21.15.59:0/2605832337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f426c102fb0 msgr2=0x7f426c112d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f42723a0700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f426c102fb0 0x7f426c112d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f42723a0700 1 -- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c1076d0 msgr2=0x7f426c1132c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f42723a0700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c1076d0 0x7f426c1132c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f42723a0700 1 -- 172.21.15.59:0/2605832337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f426c113b60 con 0x7f426c102600 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f4272ba1700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c1076d0 0x7f426c1132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f4271b9f700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f426c102fb0 0x7f426c112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.087+0000 7f42723a0700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f426c102600 0x7f426c1148f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f426800e9f0 tx=0x7f426800ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.088+0000 7f42637fe700 1 -- 172.21.15.59:0/2605832337 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426800c9c0 con 0x7f426c102600 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.088+0000 7f4274604700 1 -- 172.21.15.59:0/2605832337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f426c1a81e0 con 0x7f426c102600 2024-05-08T05:36:06.094 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.088+0000 7f4274604700 1 -- 172.21.15.59:0/2605832337 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f426c1a8770 con 0x7f426c102600 2024-05-08T05:36:06.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.088+0000 7f42637fe700 1 -- 172.21.15.59:0/2605832337 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f426800cb60 con 0x7f426c102600 2024-05-08T05:36:06.095 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.089+0000 7f42637fe700 1 -- 172.21.15.59:0/2605832337 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4268018680 con 0x7f426c102600 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.090+0000 7f42637fe700 1 -- 172.21.15.59:0/2605832337 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f4268018820 con 0x7f426c102600 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.091+0000 7f42637fe700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f42580b2b20 0x7f42580b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.092+0000 7f4271b9f700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f42580b2b20 0x7f42580b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.092+0000 7f42637fe700 1 -- 172.21.15.59:0/2605832337 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9584+0+0 (secure 0 0 0) 0x7f42680e2ad0 con 0x7f426c102600 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.092+0000 7f4271b9f700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f42580b2b20 0x7f42580b4fe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f426c10f570 tx=0x7f425c009310 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f427ee97b80 1 Processor -- start 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f427ee97b80 1 -- start start 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d56c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d50f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 0x5634c0d5d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f427ee97b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5634c0ca2ad0 con 0x5634c0d514c0 2024-05-08T05:36:06.102 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f427ee97b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5634c0c07f30 con 0x5634c0d56850 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f427ee97b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5634c0c9ea90 con 0x5634c0d50b00 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f42723a0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d56c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f4271b9f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d50f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f4272ba1700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 0x5634c0d5d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f42723a0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d56c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:42988/0 (socket says 172.21.15.59:42988) 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f4271b9f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d50f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:34974/0 (socket says 172.21.15.59:34974) 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f42723a0700 1 -- 172.21.15.59:0/3032464135 learned_addr learned my addr 172.21.15.59:0/3032464135 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f4271b9f700 1 -- 172.21.15.59:0/3032464135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 msgr2=0x5634c0d56c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f4271b9f700 1 --2- 172.21.15.59:0/3032464135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d56c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f4271b9f700 1 -- 172.21.15.59:0/3032464135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 msgr2=0x5634c0d5d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f4271b9f700 1 --2- 172.21.15.59:0/3032464135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 0x5634c0d5d610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.093+0000 7f4271b9f700 1 -- 172.21.15.59:0/3032464135 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c0befa60 con 0x5634c0d50b00 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.094+0000 7f4271b9f700 1 --2- 172.21.15.59:0/3032464135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d50f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f426c06b000 tx=0x7f425c014000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c95c85cb8d392e5c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.094+0000 7f4260ff9700 1 -- 172.21.15.59:0/3032464135 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c004220 con 0x5634c0d50b00 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.094+0000 7f4260ff9700 1 -- 172.21.15.59:0/3032464135 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f425c011e50 con 0x5634c0d50b00 2024-05-08T05:36:06.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.094+0000 7f4260ff9700 1 -- 172.21.15.59:0/3032464135 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c0266e0 con 0x5634c0d50b00 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 -- 172.21.15.59:0/3032464135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 msgr2=0x5634c0d50f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 --2- 172.21.15.59:0/3032464135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d50f80 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f426c06b000 tx=0x7f425c014000 comp rx=0 tx=0).stop 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 -- 172.21.15.59:0/3032464135 shutdown_connections 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 --2- 172.21.15.59:0/3032464135 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 0x5634c0d5d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 --2- 172.21.15.59:0/3032464135 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d56c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 --2- 172.21.15.59:0/3032464135 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d50f80 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 -- 172.21.15.59:0/3032464135 >> 172.21.15.59:0/3032464135 conn(0x5634c0c7cd00 msgr2=0x5634c0d4b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 -- 172.21.15.59:0/3032464135 shutdown_connections 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 -- 172.21.15.59:0/3032464135 wait complete. 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 -- start start 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.095+0000 7f427ee97b80 1 Processor -- start 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d5d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 0x5634c0d5a560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d5aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f427ee97b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5634c0d5db50 con 0x5634c0d514c0 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f427ee97b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5634c0c5c060 con 0x5634c0d56850 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f427ee97b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5634c0d5dcd0 con 0x5634c0d50b00 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f427ee97b80 10 client.0 _ll_register_callbacks cb 0x5634c0c36740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f42723a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d5d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f4272ba1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d5aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f4271b9f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 0x5634c0d5a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f4272ba1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d5aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:43002/0 (socket says 172.21.15.59:43002) 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f42723a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d5d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:34982/0 (socket says 172.21.15.59:34982) 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.096+0000 7f4272ba1700 1 -- 172.21.15.59:0/2658425426 learned_addr learned my addr 172.21.15.59:0/2658425426 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.097+0000 7f4271b9f700 1 -- 172.21.15.59:0/2658425426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 msgr2=0x5634c0d5d1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.097+0000 7f4271b9f700 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d5d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.097+0000 7f4271b9f700 1 -- 172.21.15.59:0/2658425426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 msgr2=0x5634c0d5aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:06.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.097+0000 7f42723a0700 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d5d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-05-08T05:36:06.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.097+0000 7f4271b9f700 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d5aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.098+0000 7f4271b9f700 1 -- 172.21.15.59:0/2658425426 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c0c598f0 con 0x5634c0d514c0 2024-05-08T05:36:06.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.098+0000 7f4272ba1700 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d5aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:36:06.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.098+0000 7f4271b9f700 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 0x5634c0d5a560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f426c06b000 tx=0x7f425c003730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77361da8a6d3d061 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:06.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.098+0000 7f4255ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:06.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.098+0000 7f4255ffb700 1 -- 172.21.15.59:0/2658425426 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f425c004020 con 0x5634c0d514c0 2024-05-08T05:36:06.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.098+0000 7f4255ffb700 1 -- 172.21.15.59:0/2658425426 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9584+0+0 (secure 0 0 0) 0x7f425c0041c0 con 0x5634c0d514c0 2024-05-08T05:36:06.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.098+0000 7f427ee97b80 1 -- 172.21.15.59:0/2658425426 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f425c011e50 con 0x5634c0d514c0 2024-05-08T05:36:06.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.099+0000 7f4255ffb700 1 -- 172.21.15.59:0/2658425426 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f425c039050 con 0x5634c0d514c0 2024-05-08T05:36:06.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.099+0000 7f427ee97b80 10 client.16275 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:06.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.099+0000 7f427ee97b80 1 -- 172.21.15.59:0/2658425426 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5634c0d682e0 con 0x5634c0d514c0 2024-05-08T05:36:06.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.099+0000 7f4255ffb700 1 -- 172.21.15.59:0/2658425426 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f425c011e50 con 0x5634c0d514c0 2024-05-08T05:36:06.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.100+0000 7f4255ffb700 1 -- 172.21.15.59:0/2658425426 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f425c01d410 con 0x5634c0d514c0 2024-05-08T05:36:06.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.100+0000 7f427ee97b80 10 client.16275 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:06.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.100+0000 7f427ee97b80 10 client.16275 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:36:06.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.100+0000 7f427ee97b80 20 client.16275 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:06.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.100+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5634c0d6a860 0x5634c0d6cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.100+0000 7f427ee97b80 4 client.16275 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:36:06.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.100+0000 7f427ee97b80 1 -- 172.21.15.59:0/2658425426 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5634c0cb04b0 con 0x5634c0d6a860 2024-05-08T05:36:06.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.101+0000 7f42723a0700 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5634c0d6a860 0x5634c0d6cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.101+0000 7f42723a0700 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5634c0d6a860 0x5634c0d6cc50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=48816d0fb039c5f3 server_cookie=ab73a9c7bf52e739 in_seq=0 out_seq=0 2024-05-08T05:36:06.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.101+0000 7f4255ffb700 10 client.16275 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f4255ffb700 1 -- 172.21.15.59:0/2658425426 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5634c0cb04b0 con 0x5634c0d6a860 2024-05-08T05:36:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f4255ffb700 10 client.16275 handle_command_reply: tid=0 2024-05-08T05:36:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f427ee97b80 1 client.16275 shutdown 2024-05-08T05:36:06.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f427ee97b80 1 -- 172.21.15.59:0/2658425426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 msgr2=0x5634c0d5a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 0x5634c0d5a560 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f426c06b000 tx=0x7f425c003730 comp rx=0 tx=0).stop 2024-05-08T05:36:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f427ee97b80 1 -- 172.21.15.59:0/2658425426 shutdown_connections 2024-05-08T05:36:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5634c0d6a860 0x5634c0d6cc50 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d56850 0x5634c0d5aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d5d1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2658425426 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d514c0 0x5634c0d5a560 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f427ee97b80 1 -- 172.21.15.59:0/2658425426 >> 172.21.15.59:0/2658425426 conn(0x5634c0c7cd00 msgr2=0x5634c0d45e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.102+0000 7f4255ffb700 0 client.16275 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.103+0000 7f427ee97b80 1 -- 172.21.15.59:0/2658425426 shutdown_connections 2024-05-08T05:36:06.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.103+0000 7f427ee97b80 1 -- 172.21.15.59:0/2658425426 wait complete. 2024-05-08T05:36:06.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.103+0000 7f427ee97b80 20 client.16275 trim_cache size 0 max 16384 2024-05-08T05:36:06.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.128+0000 7f427ee97b80 1 Processor -- start 2024-05-08T05:36:06.128 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.128+0000 7f427ee97b80 1 -- start start 2024-05-08T05:36:06.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.128+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0c7cd00 0x5634c0d5aad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.128+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d66bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.128+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0d55970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.128+0000 7f427ee97b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f425000f030 con 0x5634c0c7cd00 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.128+0000 7f427ee97b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f4250010b70 con 0x5634c0d514c0 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.128+0000 7f427ee97b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f425000f700 con 0x5634c0d50b00 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f4272ba1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0d55970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f4272ba1700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0d55970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:43018/0 (socket says 172.21.15.59:43018) 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f4272ba1700 1 -- 172.21.15.59:0/2695008684 learned_addr learned my addr 172.21.15.59:0/2695008684 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f42723a0700 1 --2- 172.21.15.59:0/2695008684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0c7cd00 0x5634c0d5aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f4272ba1700 1 -- 172.21.15.59:0/2695008684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 msgr2=0x5634c0d66bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f4272ba1700 1 --2- 172.21.15.59:0/2695008684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d66bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f4272ba1700 1 -- 172.21.15.59:0/2695008684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0c7cd00 msgr2=0x5634c0d5aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f4272ba1700 1 --2- 172.21.15.59:0/2695008684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0c7cd00 0x5634c0d5aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f4272ba1700 1 -- 172.21.15.59:0/2695008684 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5634c0d56430 con 0x5634c0d514c0 2024-05-08T05:36:06.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.129+0000 7f4272ba1700 1 --2- 172.21.15.59:0/2695008684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0d55970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4264009da0 tx=0x7f42640096e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1497dc13a983eee9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:06.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.130+0000 7f4257fff700 1 -- 172.21.15.59:0/2695008684 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f426402b020 con 0x5634c0d514c0 2024-05-08T05:36:06.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.131+0000 7f4257fff700 1 -- 172.21.15.59:0/2695008684 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f426400bcb0 con 0x5634c0d514c0 2024-05-08T05:36:06.131 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.131+0000 7f4257fff700 1 -- 172.21.15.59:0/2695008684 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4264005280 con 0x5634c0d514c0 2024-05-08T05:36:06.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.131+0000 7f427ee97b80 1 -- 172.21.15.59:0/2695008684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 msgr2=0x5634c0d55970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:06.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.132+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2695008684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0d55970 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4264009da0 tx=0x7f42640096e0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.132+0000 7f427ee97b80 1 -- 172.21.15.59:0/2695008684 shutdown_connections 2024-05-08T05:36:06.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.132+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2695008684 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0d55970 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.132+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2695008684 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0c7cd00 0x5634c0d5aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.132+0000 7f427ee97b80 1 --2- 172.21.15.59:0/2695008684 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0d50b00 0x5634c0d66bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.132+0000 7f427ee97b80 1 -- 172.21.15.59:0/2695008684 >> 172.21.15.59:0/2695008684 conn(0x5634c0d4b9c0 msgr2=0x5634c0d50380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:06.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.132+0000 7f427ee97b80 1 -- 172.21.15.59:0/2695008684 shutdown_connections 2024-05-08T05:36:06.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.132+0000 7f427ee97b80 1 -- 172.21.15.59:0/2695008684 wait complete. 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.133+0000 7f427ee97b80 1 -- start start 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.133+0000 7f427ee97b80 1 Processor -- start 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.133+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0c7cd00 0x5634c0d631f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.133+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d50b00 0x5634c0dd0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.133+0000 7f427ee97b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0dd05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.133+0000 7f427ee97b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f42500104a0 con 0x5634c0d50b00 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.133+0000 7f427ee97b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f425000fdd0 con 0x5634c0d514c0 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.133+0000 7f427ee97b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f425000cfd0 con 0x5634c0c7cd00 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.133+0000 7f427ee97b80 10 client.0 _ll_register_callbacks cb 0x5634c0c36740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f42723a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0c7cd00 0x5634c0d631f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f42723a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0c7cd00 0x5634c0d631f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:35006/0 (socket says 172.21.15.59:35006) 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f42723a0700 1 -- 172.21.15.59:0/826732199 learned_addr learned my addr 172.21.15.59:0/826732199 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f4271b9f700 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d50b00 0x5634c0dd0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.136 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f4272ba1700 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0dd05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f42723a0700 1 -- 172.21.15.59:0/826732199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 msgr2=0x5634c0dd05b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f42723a0700 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0dd05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f42723a0700 1 -- 172.21.15.59:0/826732199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d50b00 msgr2=0x5634c0dd0070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f42723a0700 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d50b00 0x5634c0dd0070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.134+0000 7f42723a0700 1 -- 172.21.15.59:0/826732199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5634c0dd0af0 con 0x5634c0c7cd00 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.135+0000 7f4271b9f700 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d50b00 0x5634c0dd0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.135+0000 7f42723a0700 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0c7cd00 0x5634c0d631f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f42680a7b80 tx=0x7f426801f000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=96c2f7d2592ec089 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.135+0000 7f4255ffb700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.135+0000 7f4255ffb700 1 -- 172.21.15.59:0/826732199 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42680ab010 con 0x5634c0c7cd00 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.135+0000 7f4255ffb700 1 -- 172.21.15.59:0/826732199 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(92..92 src has 1..92) ==== 9584+0+0 (secure 0 0 0) 0x7f42680a6f00 con 0x5634c0c7cd00 2024-05-08T05:36:06.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.135+0000 7f427ee97b80 1 -- 172.21.15.59:0/826732199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f426400bcb0 con 0x5634c0c7cd00 2024-05-08T05:36:06.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f4255ffb700 1 -- 172.21.15.59:0/826732199 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f42680ab010 con 0x5634c0c7cd00 2024-05-08T05:36:06.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f427ee97b80 10 client.25816 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:06.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f427ee97b80 1 -- 172.21.15.59:0/826732199 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5634c0d61f50 con 0x5634c0c7cd00 2024-05-08T05:36:06.138 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f4255ffb700 1 -- 172.21.15.59:0/826732199 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f426400bcb0 con 0x5634c0c7cd00 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f4255ffb700 1 -- 172.21.15.59:0/826732199 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f42680aab70 con 0x5634c0c7cd00 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f427ee97b80 10 client.25816 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f427ee97b80 10 client.25816 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f427ee97b80 20 client.25816 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f427ee97b80 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5634c0d56850 0x5634c0d626b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f427ee97b80 4 client.25816 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.7, "await_for": 42.0}] 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.136+0000 7f427ee97b80 1 -- 172.21.15.59:0/826732199 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 85.7, "await_for": 42.0}) -- 0x5634c0caa550 con 0x5634c0d56850 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.137+0000 7f4271b9f700 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5634c0d56850 0x5634c0d626b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.137+0000 7f4271b9f700 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5634c0d56850 0x5634c0d626b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ffa0fb3ec51db58f server_cookie=bd24edb61253e976 in_seq=0 out_seq=0 2024-05-08T05:36:06.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:06.138+0000 7f4255ffb700 10 client.25816 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f4255ffb700 1 -- 172.21.15.59:0/826732199 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x5634c0caa550 con 0x5634c0d56850 2024-05-08T05:36:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f4255ffb700 10 client.25816 handle_command_reply: tid=0 2024-05-08T05:36:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 client.25816 shutdown 2024-05-08T05:36:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 -- 172.21.15.59:0/826732199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0c7cd00 msgr2=0x5634c0d631f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0c7cd00 0x5634c0d631f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f42680a7b80 tx=0x7f426801f000 comp rx=0 tx=0).stop 2024-05-08T05:36:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 -- 172.21.15.59:0/826732199 shutdown_connections 2024-05-08T05:36:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5634c0d56850 0x5634c0d626b0 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5634c0d514c0 0x5634c0dd05b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:07.099 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5634c0c7cd00 0x5634c0d631f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:07.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f4255ffb700 0 client.25816 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:07.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 --2- 172.21.15.59:0/826732199 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5634c0d50b00 0x5634c0dd0070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:07.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 -- 172.21.15.59:0/826732199 >> 172.21.15.59:0/826732199 conn(0x5634c0d4b9c0 msgr2=0x5634c0d57390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:07.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 -- 172.21.15.59:0/826732199 shutdown_connections 2024-05-08T05:36:07.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 1 -- 172.21.15.59:0/826732199 wait complete. 2024-05-08T05:36:07.100 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.098+0000 7f427ee97b80 20 client.25816 trim_cache size 0 max 16384 2024-05-08T05:36:07.100 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:36:07.100 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 36, 2024-05-08T05:36:07.100 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 22, 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "64a1b109": { 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 22, 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 85.7, 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:36:07.101 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:36:07.102 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:36:07.102 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:36:07.102 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:36:07.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:07.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:07.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:07.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:07.102 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:07.102 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 -- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f42580b2b20 msgr2=0x7f42580b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f42580b2b20 0x7f42580b4fe0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f426c10f570 tx=0x7f425c009310 comp rx=0 tx=0).stop 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 -- 172.21.15.59:0/2605832337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f426c102600 msgr2=0x7f426c1148f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f426c102600 0x7f426c1148f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f426800e9f0 tx=0x7f426800ed00 comp rx=0 tx=0).stop 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 -- 172.21.15.59:0/2605832337 shutdown_connections 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f42580b2b20 0x7f42580b4fe0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f426c1076d0 0x7f426c1132c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f426c102600 0x7f426c1148f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 --2- 172.21.15.59:0/2605832337 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f426c102fb0 0x7f426c112d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 -- 172.21.15.59:0/2605832337 >> 172.21.15.59:0/2605832337 conn(0x7f426c079340 msgr2=0x7f426c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.099+0000 7f4274604700 1 -- 172.21.15.59:0/2605832337 shutdown_connections 2024-05-08T05:36:07.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:07.100+0000 7f4274604700 1 -- 172.21.15.59:0/2605832337 wait complete. 2024-05-08T05:36:07.437 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:07 smithi059 ceph-mon[30304]: pgmap v200: 129 pgs: 129 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-05-08T05:36:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:07 smithi096 ceph-mon[34253]: pgmap v200: 129 pgs: 129 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-05-08T05:36:07.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:07 smithi143 ceph-mon[34337]: pgmap v200: 129 pgs: 129 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-05-08T05:36:07.899 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set '64a1b109', quiesce took 8.5 seconds. Will release after: 25.7 2024-05-08T05:36:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:09 smithi096 ceph-mon[34253]: pgmap v201: 129 pgs: 129 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-05-08T05:36:09.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:09 smithi059 ceph-mon[30304]: pgmap v201: 129 pgs: 129 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-05-08T05:36:09.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:09 smithi143 ceph-mon[34337]: pgmap v201: 129 pgs: 129 active+clean; 40 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.3 MiB/s wr, 217 op/s 2024-05-08T05:36:11.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:11 smithi096 ceph-mon[34253]: pgmap v202: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-05-08T05:36:11.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:11 smithi096 ceph-mon[34253]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T05:36:11.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:11 smithi059 ceph-mon[30304]: pgmap v202: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-05-08T05:36:11.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:11 smithi059 ceph-mon[30304]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T05:36:11.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:11 smithi143 ceph-mon[34337]: pgmap v202: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-05-08T05:36:11.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:11 smithi143 ceph-mon[34337]: osdmap e93: 12 total, 12 up, 12 in 2024-05-08T05:36:13.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:13 smithi096 ceph-mon[34253]: pgmap v204: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-05-08T05:36:13.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:13 smithi059 ceph-mon[30304]: pgmap v204: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-05-08T05:36:13.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:13 smithi143 ceph-mon[34337]: pgmap v204: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.7 MiB/s rd, 1.4 MiB/s wr, 206 op/s 2024-05-08T05:36:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:15 smithi096 ceph-mon[34253]: pgmap v205: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-05-08T05:36:15.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:15 smithi143 ceph-mon[34337]: pgmap v205: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-05-08T05:36:15.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:15 smithi059 ceph-mon[30304]: pgmap v205: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 2.6 MiB/s rd, 1.3 MiB/s wr, 200 op/s 2024-05-08T05:36:15.767 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.766+0000 7f344196a640 1 -- 172.21.15.59:0/1990158987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1078a0 msgr2=0x7f343c107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.766+0000 7f344196a640 1 --2- 172.21.15.59:0/1990158987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1078a0 0x7f343c107c80 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f342c009930 tx=0x7f342c02a420 comp rx=0 tx=0).stop 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.767+0000 7f344196a640 1 -- 172.21.15.59:0/1990158987 shutdown_connections 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.767+0000 7f344196a640 1 --2- 172.21.15.59:0/1990158987 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f343c1022a0 0x7f343c10e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.767+0000 7f344196a640 1 --2- 172.21.15.59:0/1990158987 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f343c1018e0 0x7f343c101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.767+0000 7f344196a640 1 --2- 172.21.15.59:0/1990158987 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1078a0 0x7f343c107c80 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.767+0000 7f344196a640 1 -- 172.21.15.59:0/1990158987 >> 172.21.15.59:0/1990158987 conn(0x7f343c078a30 msgr2=0x7f343c078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.767+0000 7f344196a640 1 -- 172.21.15.59:0/1990158987 shutdown_connections 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.767+0000 7f344196a640 1 -- 172.21.15.59:0/1990158987 wait complete. 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.767+0000 7f344196a640 1 Processor -- start 2024-05-08T05:36:15.768 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.767+0000 7f344196a640 1 -- start start 2024-05-08T05:36:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f344196a640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f343c1018e0 0x7f343c199ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f344196a640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1022a0 0x7f343c19a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f344196a640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f343c1078a0 0x7f343c1a4f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.770 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f344196a640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f343c111b70 con 0x7f343c1022a0 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f344196a640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1119f0 con 0x7f343c1078a0 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f344196a640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f343c111cf0 con 0x7f343c1018e0 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343a7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1022a0 0x7f343c19a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343a7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1022a0 0x7f343c19a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46306/0 (socket says 172.21.15.59:46306) 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343a7fc640 1 -- 172.21.15.59:0/172368427 learned_addr learned my addr 172.21.15.59:0/172368427 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343affd640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f343c1018e0 0x7f343c199ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343b7fe640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f343c1078a0 0x7f343c1a4f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343a7fc640 1 -- 172.21.15.59:0/172368427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f343c1018e0 msgr2=0x7f343c199ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343a7fc640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f343c1018e0 0x7f343c199ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343a7fc640 1 -- 172.21.15.59:0/172368427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f343c1078a0 msgr2=0x7f343c1a4f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343a7fc640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f343c1078a0 0x7f343c1a4f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343a7fc640 1 -- 172.21.15.59:0/172368427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f343c1a54b0 con 0x7f343c1022a0 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343affd640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f343c1018e0 0x7f343c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:36:15.771 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.768+0000 7f343a7fc640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1022a0 0x7f343c19a430 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f342400d950 tx=0x7f342400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.769+0000 7f3440968640 1 -- 172.21.15.59:0/172368427 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424017030 con 0x7f343c1022a0 2024-05-08T05:36:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.769+0000 7f3440968640 1 -- 172.21.15.59:0/172368427 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3424004020 con 0x7f343c1022a0 2024-05-08T05:36:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.769+0000 7f344196a640 1 -- 172.21.15.59:0/172368427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f343c1a5680 con 0x7f343c1022a0 2024-05-08T05:36:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.769+0000 7f3440968640 1 -- 172.21.15.59:0/172368427 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34240047a0 con 0x7f343c1022a0 2024-05-08T05:36:15.772 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.769+0000 7f344196a640 1 -- 172.21.15.59:0/172368427 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f343c1a5bd0 con 0x7f343c1022a0 2024-05-08T05:36:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.771+0000 7f3444680740 1 Processor -- start 2024-05-08T05:36:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.771+0000 7f3440968640 1 -- 172.21.15.59:0/172368427 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3424009a30 con 0x7f343c1022a0 2024-05-08T05:36:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.771+0000 7f3444680740 1 -- start start 2024-05-08T05:36:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.771+0000 7f3444680740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 0x5643ea42d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f3444680740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea42d830 0x5643ea437bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f3444680740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea4380f0 0x5643ea43a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f3444680740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5643ea3925d0 con 0x5643ea4380f0 2024-05-08T05:36:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f3444680740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643ea34b890 con 0x5643ea42cf10 2024-05-08T05:36:15.779 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f3444680740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5643ea38e590 con 0x5643ea42d830 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343a7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea4380f0 0x5643ea43a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343a7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea4380f0 0x5643ea43a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46316/0 (socket says 172.21.15.59:46316) 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343a7fc640 1 -- 172.21.15.59:0/242022173 learned_addr learned my addr 172.21.15.59:0/242022173 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343affd640 1 --2- 172.21.15.59:0/242022173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea42d830 0x5643ea437bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343b7fe640 1 --2- 172.21.15.59:0/242022173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 0x5643ea42d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343a7fc640 1 -- 172.21.15.59:0/242022173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea42d830 msgr2=0x5643ea437bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343a7fc640 1 --2- 172.21.15.59:0/242022173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea42d830 0x5643ea437bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343a7fc640 1 -- 172.21.15.59:0/242022173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 msgr2=0x5643ea42d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343a7fc640 1 --2- 172.21.15.59:0/242022173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 0x5643ea42d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343a7fc640 1 -- 172.21.15.59:0/242022173 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643ea0a4210 con 0x5643ea4380f0 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.772+0000 7f343a7fc640 1 --2- 172.21.15.59:0/242022173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea4380f0 0x5643ea43a4e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f34240df820 tx=0x7f34240e89f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f1abeef8d721e10 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.773+0000 7f3440968640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f34280b2a10 0x7f34280b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.773+0000 7f3431ffb640 1 -- 172.21.15.59:0/242022173 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34240f5070 con 0x5643ea4380f0 2024-05-08T05:36:15.780 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.773+0000 7f343b7fe640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f34280b2a10 0x7f34280b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.773+0000 7f3431ffb640 1 -- 172.21.15.59:0/242022173 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34240a8f20 con 0x5643ea4380f0 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.773+0000 7f3431ffb640 1 -- 172.21.15.59:0/242022173 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3424009d20 con 0x5643ea4380f0 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.773+0000 7f3440968640 1 -- 172.21.15.59:0/172368427 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7f34240e2500 con 0x7f343c1022a0 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.773+0000 7f3444680740 1 -- 172.21.15.59:0/242022173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea4380f0 msgr2=0x5643ea43a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.773+0000 7f343b7fe640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f34280b2a10 0x7f34280b4ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3434007e50 tx=0x7f3434007600 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.773+0000 7f3444680740 1 --2- 172.21.15.59:0/242022173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea4380f0 0x5643ea43a4e0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f34240df820 tx=0x7f34240e89f0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.774+0000 7f3444680740 1 -- 172.21.15.59:0/242022173 shutdown_connections 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.774+0000 7f3444680740 1 --2- 172.21.15.59:0/242022173 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea4380f0 0x5643ea43a4e0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.774+0000 7f3444680740 1 --2- 172.21.15.59:0/242022173 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea42d830 0x5643ea437bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.774+0000 7f3444680740 1 --2- 172.21.15.59:0/242022173 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 0x5643ea42d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.774+0000 7f3444680740 1 -- 172.21.15.59:0/242022173 >> 172.21.15.59:0/242022173 conn(0x5643ea429420 msgr2=0x5643ea429800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.774+0000 7f3444680740 1 -- 172.21.15.59:0/242022173 shutdown_connections 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.774+0000 7f3444680740 1 -- 172.21.15.59:0/242022173 wait complete. 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.774+0000 7f3444680740 1 -- start start 2024-05-08T05:36:15.781 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.774+0000 7f3444680740 1 Processor -- start 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f3444680740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 0x5643ea435040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f3444680740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42d830 0x5643ea435580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f3444680740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea435ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f3444680740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5643ea43ad20 con 0x5643ea42d830 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f3444680740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5643ea34f370 con 0x5643ea42cf10 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f3444680740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5643ea43aea0 con 0x5643ea4380f0 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f3444680740 10 client.0 _ll_register_callbacks cb 0x5643ea0a3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343a7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42d830 0x5643ea435580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343a7fc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42d830 0x5643ea435580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46326/0 (socket says 172.21.15.59:46326) 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343a7fc640 1 -- 172.21.15.59:0/3214345865 learned_addr learned my addr 172.21.15.59:0/3214345865 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343b7fe640 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea435ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343affd640 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 0x5643ea435040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343a7fc640 1 -- 172.21.15.59:0/3214345865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 msgr2=0x5643ea435ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:15.782 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343a7fc640 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea435ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343a7fc640 1 -- 172.21.15.59:0/3214345865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 msgr2=0x5643ea435040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343a7fc640 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 0x5643ea435040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343a7fc640 1 -- 172.21.15.59:0/3214345865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643ea170620 con 0x5643ea42d830 2024-05-08T05:36:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f343a7fc640 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42d830 0x5643ea435580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3424020000 tx=0x7f342400df40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0879f710dce3fa4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f341affd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.775+0000 7f341affd640 1 -- 172.21.15.59:0/3214345865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f34240213d0 con 0x5643ea42d830 2024-05-08T05:36:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f341affd640 1 -- 172.21.15.59:0/3214345865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7f34240211d0 con 0x5643ea42d830 2024-05-08T05:36:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f3444680740 1 -- 172.21.15.59:0/3214345865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f34240a8f20 con 0x5643ea42d830 2024-05-08T05:36:15.783 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f341affd640 1 -- 172.21.15.59:0/3214345865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f3424021630 con 0x5643ea42d830 2024-05-08T05:36:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f3444680740 10 client.16293 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f3444680740 1 -- 172.21.15.59:0/3214345865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5643ea43ad20 con 0x5643ea42d830 2024-05-08T05:36:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f341affd640 1 -- 172.21.15.59:0/3214345865 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f34240a8f20 con 0x5643ea42d830 2024-05-08T05:36:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f341affd640 1 -- 172.21.15.59:0/3214345865 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f34240dfb00 con 0x5643ea42d830 2024-05-08T05:36:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f3444680740 10 client.16293 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f3444680740 10 client.16293 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f3444680740 20 client.16293 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f3444680740 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5643ea447d20 0x5643ea44a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.784 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f3444680740 4 client.16293 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.776+0000 7f3444680740 1 -- 172.21.15.59:0/3214345865 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643ea3a01e0 con 0x5643ea447d20 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.777+0000 7f343affd640 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5643ea447d20 0x5643ea44a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.777+0000 7f341affd640 10 client.16293 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.777+0000 7f343affd640 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5643ea447d20 0x5643ea44a110 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aff0354646b53b8a server_cookie=d3436d9312934eaf in_seq=0 out_seq=0 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f341affd640 1 -- 172.21.15.59:0/3214345865 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5643ea3a01e0 con 0x5643ea447d20 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f341affd640 10 client.16293 handle_command_reply: tid=0 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 client.16293 shutdown 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 -- 172.21.15.59:0/3214345865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42d830 msgr2=0x5643ea435580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42d830 0x5643ea435580 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f3424020000 tx=0x7f342400df40 comp rx=0 tx=0).stop 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 -- 172.21.15.59:0/3214345865 shutdown_connections 2024-05-08T05:36:15.785 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5643ea447d20 0x5643ea44a110 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea435ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42d830 0x5643ea435580 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f341affd640 0 client.16293 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 --2- 172.21.15.59:0/3214345865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42cf10 0x5643ea435040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 -- 172.21.15.59:0/3214345865 >> 172.21.15.59:0/3214345865 conn(0x5643ea429420 msgr2=0x5643ea42f150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 -- 172.21.15.59:0/3214345865 shutdown_connections 2024-05-08T05:36:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 1 -- 172.21.15.59:0/3214345865 wait complete. 2024-05-08T05:36:15.786 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.778+0000 7f3444680740 20 client.16293 trim_cache size 0 max 16384 2024-05-08T05:36:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f3444680740 1 Processor -- start 2024-05-08T05:36:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f3444680740 1 -- start start 2024-05-08T05:36:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f3444680740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea44f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f3444680740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea44fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f3444680740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea427cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f3444680740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1afd00 con 0x5643ea42cf10 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f3444680740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1b1840 con 0x5643ea42d830 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f3444680740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1b03d0 con 0x5643ea4380f0 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343affd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea44f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343affd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea44f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46334/0 (socket says 172.21.15.59:46334) 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343affd640 1 -- 172.21.15.59:0/3418301929 learned_addr learned my addr 172.21.15.59:0/3418301929 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343b7fe640 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea427cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343a7fc640 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea44fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343affd640 1 -- 172.21.15.59:0/3418301929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 msgr2=0x5643ea427cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343affd640 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea427cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343affd640 1 -- 172.21.15.59:0/3418301929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 msgr2=0x5643ea44fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343affd640 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea44fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343affd640 1 -- 172.21.15.59:0/3418301929 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643ea42d660 con 0x5643ea42cf10 2024-05-08T05:36:15.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343b7fe640 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea427cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343a7fc640 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea44fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.794+0000 7f343affd640 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea44f7d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f342c02aa10 tx=0x7f342c031400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd500ad04eaf34dc server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f34317fa640 1 -- 172.21.15.59:0/3418301929 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c035040 con 0x5643ea42cf10 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f34317fa640 1 -- 172.21.15.59:0/3418301929 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f342c049070 con 0x5643ea42cf10 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f34317fa640 1 -- 172.21.15.59:0/3418301929 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c004ab0 con 0x5643ea42cf10 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f3444680740 1 -- 172.21.15.59:0/3418301929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 msgr2=0x5643ea44f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f3444680740 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea44f7d0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f342c02aa10 tx=0x7f342c031400 comp rx=0 tx=0).stop 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f3444680740 1 -- 172.21.15.59:0/3418301929 shutdown_connections 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f3444680740 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea427cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f3444680740 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea44fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f3444680740 1 --2- 172.21.15.59:0/3418301929 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea44f7d0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f3444680740 1 -- 172.21.15.59:0/3418301929 >> 172.21.15.59:0/3418301929 conn(0x5643ea429420 msgr2=0x5643ea42dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f3444680740 1 -- 172.21.15.59:0/3418301929 shutdown_connections 2024-05-08T05:36:15.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.795+0000 7f3444680740 1 -- 172.21.15.59:0/3418301929 wait complete. 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f3444680740 1 -- start start 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f3444680740 1 Processor -- start 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f3444680740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea4400f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f3444680740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea440630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f3444680740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea43d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f3444680740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1b1170 con 0x5643ea42cf10 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f3444680740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1b0aa0 con 0x5643ea42d830 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f3444680740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f343c1add00 con 0x5643ea4380f0 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f3444680740 10 client.0 _ll_register_callbacks cb 0x7f34240f3070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f343affd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea4400f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f343affd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea4400f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:46342/0 (socket says 172.21.15.59:46342) 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.796+0000 7f343affd640 1 -- 172.21.15.59:0/2957312656 learned_addr learned my addr 172.21.15.59:0/2957312656 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f343b7fe640 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea43d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f343affd640 1 -- 172.21.15.59:0/2957312656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 msgr2=0x5643ea43d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f343a7fc640 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea440630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f343affd640 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea43d0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f343affd640 1 -- 172.21.15.59:0/2957312656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 msgr2=0x5643ea440630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f343affd640 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea440630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f343affd640 1 -- 172.21.15.59:0/2957312656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643ea43d650 con 0x5643ea42cf10 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f343a7fc640 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea440630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f343affd640 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea4400f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f342c02a460 tx=0x7f342c005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7b7c6d1e5e05c77 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f341affd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f341affd640 1 -- 172.21.15.59:0/2957312656 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f342c051030 con 0x5643ea42cf10 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f3444680740 1 -- 172.21.15.59:0/2957312656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f342c049070 con 0x5643ea42cf10 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f341affd640 1 -- 172.21.15.59:0/2957312656 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7f342c0055f0 con 0x5643ea42cf10 2024-05-08T05:36:15.803 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f341affd640 1 -- 172.21.15.59:0/2957312656 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7f342c035050 con 0x5643ea42cf10 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f3444680740 10 client.16299 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f3444680740 1 -- 172.21.15.59:0/2957312656 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x5643ea4b2c20 con 0x5643ea42cf10 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.797+0000 7f341affd640 1 -- 172.21.15.59:0/2957312656 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f342c049070 con 0x5643ea42cf10 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f341affd640 1 -- 172.21.15.59:0/2957312656 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7f342c0035f0 con 0x5643ea42cf10 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f3444680740 10 client.16299 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f3444680740 10 client.16299 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f3444680740 20 client.16299 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f3444680740 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5643ea4b3310 0x5643ea4b5730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f3444680740 4 client.16299 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f3444680740 1 -- 172.21.15.59:0/2957312656 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643ea39a050 con 0x5643ea4b3310 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f343a7fc640 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5643ea4b3310 0x5643ea4b5730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f341affd640 10 client.16299 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:15.804 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.798+0000 7f343a7fc640 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5643ea4b3310 0x5643ea4b5730 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c0528927d4172b6 server_cookie=9913bfb5dad1f52c in_seq=0 out_seq=0 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f341affd640 1 -- 172.21.15.59:0/2957312656 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643ea39a050 con 0x5643ea4b3310 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f341affd640 10 client.16299 handle_command_reply: tid=0 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 client.16299 shutdown 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 -- 172.21.15.59:0/2957312656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 msgr2=0x5643ea4400f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea4400f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f342c02a460 tx=0x7f342c005340 comp rx=0 tx=0).stop 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 -- 172.21.15.59:0/2957312656 shutdown_connections 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5643ea4b3310 0x5643ea4b5730 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5643ea4380f0 0x5643ea43d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5643ea42d830 0x5643ea440630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f341affd640 0 client.16299 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:15.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 --2- 172.21.15.59:0/2957312656 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5643ea42cf10 0x5643ea4400f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 -- 172.21.15.59:0/2957312656 >> 172.21.15.59:0/2957312656 conn(0x5643ea429420 msgr2=0x5643ea427cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 -- 172.21.15.59:0/2957312656 shutdown_connections 2024-05-08T05:36:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.809+0000 7f3444680740 1 -- 172.21.15.59:0/2957312656 wait complete. 2024-05-08T05:36:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f3444680740 20 client.16299 trim_cache size 0 max 16384 2024-05-08T05:36:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:36:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:36:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:36:15.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:36:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 -- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f34280b2a10 msgr2=0x7f34280b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f34280b2a10 0x7f34280b4ed0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f3434007e50 tx=0x7f3434007600 comp rx=0 tx=0).stop 2024-05-08T05:36:15.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 -- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1022a0 msgr2=0x7f343c19a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1022a0 0x7f343c19a430 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7f342400d950 tx=0x7f342400de10 comp rx=0 tx=0).stop 2024-05-08T05:36:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 -- 172.21.15.59:0/172368427 shutdown_connections 2024-05-08T05:36:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f34280b2a10 0x7f34280b4ed0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f343c1078a0 0x7f343c1a4f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f343c1022a0 0x7f343c19a430 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 --2- 172.21.15.59:0/172368427 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f343c1018e0 0x7f343c199ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:15.814 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 -- 172.21.15.59:0/172368427 >> 172.21.15.59:0/172368427 conn(0x7f343c078a30 msgr2=0x7f343c1043c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 -- 172.21.15.59:0/172368427 shutdown_connections 2024-05-08T05:36:15.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:15.810+0000 7f344196a640 1 -- 172.21.15.59:0/172368427 wait complete. 2024-05-08T05:36:16.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:16 smithi059 ceph-mon[30304]: pgmap v206: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 376 KiB/s wr, 54 op/s 2024-05-08T05:36:16.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:16 smithi143 ceph-mon[34337]: pgmap v206: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 376 KiB/s wr, 54 op/s 2024-05-08T05:36:16.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:16 smithi096 ceph-mon[34253]: pgmap v206: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 376 KiB/s wr, 54 op/s 2024-05-08T05:36:19.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:18 smithi096 ceph-mon[34253]: pgmap v207: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 376 KiB/s wr, 54 op/s 2024-05-08T05:36:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:18 smithi059 ceph-mon[30304]: pgmap v207: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 376 KiB/s wr, 54 op/s 2024-05-08T05:36:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:18 smithi143 ceph-mon[34337]: pgmap v207: 129 pgs: 129 active+clean; 46 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 484 KiB/s rd, 376 KiB/s wr, 54 op/s 2024-05-08T05:36:21.282 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:21 smithi096 ceph-mon[34253]: pgmap v208: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-05-08T05:36:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:21 smithi059 ceph-mon[30304]: pgmap v208: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-05-08T05:36:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:21 smithi143 ceph-mon[34337]: pgmap v208: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.5 KiB/s rd, 59 KiB/s wr, 14 op/s 2024-05-08T05:36:23.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:23 smithi096 ceph-mon[34253]: pgmap v209: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-05-08T05:36:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:23 smithi059 ceph-mon[30304]: pgmap v209: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-05-08T05:36:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:23 smithi143 ceph-mon[34337]: pgmap v209: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 8.4 KiB/s rd, 52 KiB/s wr, 12 op/s 2024-05-08T05:36:25.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:25 smithi096 ceph-mon[34253]: pgmap v210: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 49 KiB/s wr, 11 op/s 2024-05-08T05:36:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:25 smithi059 ceph-mon[30304]: pgmap v210: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 49 KiB/s wr, 11 op/s 2024-05-08T05:36:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:25 smithi143 ceph-mon[34337]: pgmap v210: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 49 KiB/s wr, 11 op/s 2024-05-08T05:36:27.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:27 smithi096 ceph-mon[34253]: pgmap v211: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-05-08T05:36:27.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:27 smithi059 ceph-mon[30304]: pgmap v211: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-05-08T05:36:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:27 smithi143 ceph-mon[34337]: pgmap v211: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 49 KiB/s wr, 15 op/s 2024-05-08T05:36:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.912+0000 7fbe766e8640 1 -- 172.21.15.59:0/1638868904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe7010cfa0 msgr2=0x7fbe7010f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.912+0000 7fbe766e8640 1 --2- 172.21.15.59:0/1638868904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe7010cfa0 0x7fbe7010f410 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009ca0 tx=0x7fbe6802a950 comp rx=0 tx=0).stop 2024-05-08T05:36:28.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.912+0000 7fbe766e8640 1 -- 172.21.15.59:0/1638868904 shutdown_connections 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.912+0000 7fbe766e8640 1 --2- 172.21.15.59:0/1638868904 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe7010cfa0 0x7fbe7010f410 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.912+0000 7fbe766e8640 1 --2- 172.21.15.59:0/1638868904 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70104580 0x7fbe7010ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.912+0000 7fbe766e8640 1 --2- 172.21.15.59:0/1638868904 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe70103c60 0x7fbe70104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.912+0000 7fbe766e8640 1 -- 172.21.15.59:0/1638868904 >> 172.21.15.59:0/1638868904 conn(0x7fbe70078a30 msgr2=0x7fbe70078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.912+0000 7fbe766e8640 1 -- 172.21.15.59:0/1638868904 shutdown_connections 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.912+0000 7fbe766e8640 1 -- 172.21.15.59:0/1638868904 wait complete. 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe766e8640 1 Processor -- start 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe766e8640 1 -- start start 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe766e8640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70103c60 0x7fbe70199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe766e8640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe70104580 0x7fbe701995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe766e8640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7010cfa0 0x7fbe7019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe766e8640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70112c20 con 0x7fbe70104580 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe766e8640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70112aa0 con 0x7fbe70103c60 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe766e8640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe70112da0 con 0x7fbe7010cfa0 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe67fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe70104580 0x7fbe701995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe67fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe70104580 0x7fbe701995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33982/0 (socket says 172.21.15.59:33982) 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe67fff640 1 -- 172.21.15.59:0/3621275386 learned_addr learned my addr 172.21.15.59:0/3621275386 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70103c60 0x7fbe70199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.913+0000 7fbe74c85640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7010cfa0 0x7fbe7019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe67fff640 1 -- 172.21.15.59:0/3621275386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7010cfa0 msgr2=0x7fbe7019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe67fff640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7010cfa0 0x7fbe7019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe67fff640 1 -- 172.21.15.59:0/3621275386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70103c60 msgr2=0x7fbe70199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe67fff640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70103c60 0x7fbe70199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe67fff640 1 -- 172.21.15.59:0/3621275386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbe7019e000 con 0x7fbe70104580 2024-05-08T05:36:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe67fff640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe70104580 0x7fbe701995a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d950 tx=0x7fbe5800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe6dffb640 1 -- 172.21.15.59:0/3621275386 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58017030 con 0x7fbe70104580 2024-05-08T05:36:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe6dffb640 1 -- 172.21.15.59:0/3621275386 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe58004020 con 0x7fbe70104580 2024-05-08T05:36:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe766e8640 1 -- 172.21.15.59:0/3621275386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbe7019e2f0 con 0x7fbe70104580 2024-05-08T05:36:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe6dffb640 1 -- 172.21.15.59:0/3621275386 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe58005560 con 0x7fbe70104580 2024-05-08T05:36:28.918 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.914+0000 7fbe766e8640 1 -- 172.21.15.59:0/3621275386 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbe701a5b80 con 0x7fbe70104580 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe79408740 1 Processor -- start 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe79408740 1 -- start start 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe6dffb640 1 -- 172.21.15.59:0/3621275386 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fbe58009a30 con 0x7fbe70104580 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe79408740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafd51f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe79408740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd5730 0x55f4aafe3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe79408740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafe4500 0x55f4aafe68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe79408740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f4aaf3a5d0 con 0x55f4aafd5730 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe79408740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f4aaef3890 con 0x55f4aafd4e10 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe79408740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f4aaf36590 con 0x55f4aafe4500 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd5730 0x55f4aafe3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.927 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd5730 0x55f4aafe3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33996/0 (socket says 172.21.15.59:33996) 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe6ffff640 1 -- 172.21.15.59:0/718468114 learned_addr learned my addr 172.21.15.59:0/718468114 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe74c85640 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafd51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe67fff640 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafe4500 0x55f4aafe68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe6ffff640 1 -- 172.21.15.59:0/718468114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafe4500 msgr2=0x55f4aafe68f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafe4500 0x55f4aafe68f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe6ffff640 1 -- 172.21.15.59:0/718468114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 msgr2=0x55f4aafd51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.917+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafd51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.918+0000 7fbe6ffff640 1 -- 172.21.15.59:0/718468114 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4aac4c210 con 0x55f4aafd5730 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.918+0000 7fbe74c85640 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafd51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.918+0000 7fbe6dffb640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe5c0b2a10 0x7fbe5c0b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.918+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd5730 0x55f4aafe3fc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000ad00 tx=0x7fbe6000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63c6a9b88048276c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.918+0000 7fbe74c85640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe5c0b2a10 0x7fbe5c0b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.928 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.918+0000 7fbe6dffb640 1 -- 172.21.15.59:0/3621275386 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7fbe580e22d0 con 0x7fbe70104580 2024-05-08T05:36:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.918+0000 7fbe667fc640 1 -- 172.21.15.59:0/718468114 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6001d070 con 0x55f4aafd5730 2024-05-08T05:36:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.918+0000 7fbe667fc640 1 -- 172.21.15.59:0/718468114 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe60018af0 con 0x55f4aafd5730 2024-05-08T05:36:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.918+0000 7fbe667fc640 1 -- 172.21.15.59:0/718468114 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe60010b30 con 0x55f4aafd5730 2024-05-08T05:36:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 -- 172.21.15.59:0/718468114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd5730 msgr2=0x55f4aafe3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe74c85640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe5c0b2a10 0x7fbe5c0b4ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009430 tx=0x7fbe68009360 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:28.929 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd5730 0x55f4aafe3fc0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000ad00 tx=0x7fbe6000f790 comp rx=0 tx=0).stop 2024-05-08T05:36:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 -- 172.21.15.59:0/718468114 shutdown_connections 2024-05-08T05:36:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafe4500 0x55f4aafe68f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd5730 0x55f4aafe3fc0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 --2- 172.21.15.59:0/718468114 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafd51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 -- 172.21.15.59:0/718468114 >> 172.21.15.59:0/718468114 conn(0x55f4aafd1420 msgr2=0x55f4aafd1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:28.930 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 -- 172.21.15.59:0/718468114 shutdown_connections 2024-05-08T05:36:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 -- 172.21.15.59:0/718468114 wait complete. 2024-05-08T05:36:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 -- start start 2024-05-08T05:36:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.919+0000 7fbe79408740 1 Processor -- start 2024-05-08T05:36:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe79408740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd4e10 0x55f4aafe13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe79408740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd5730 0x55f4aafe1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.931 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe79408740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aafe1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe79408740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f4aafe7130 con 0x55f4aafe4500 2024-05-08T05:36:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe79408740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f4aaef7370 con 0x55f4aafd5730 2024-05-08T05:36:28.932 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe79408740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f4aafe72b0 con 0x55f4aafd4e10 2024-05-08T05:36:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe79408740 10 client.0 _ll_register_callbacks cb 0x55f4aac4ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe74c85640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aafe1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe74c85640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aafe1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33998/0 (socket says 172.21.15.59:33998) 2024-05-08T05:36:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe74c85640 1 -- 172.21.15.59:0/2597971265 learned_addr learned my addr 172.21.15.59:0/2597971265 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe67fff640 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd5730 0x55f4aafe1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd4e10 0x55f4aafe13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe74c85640 1 -- 172.21.15.59:0/2597971265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd4e10 msgr2=0x55f4aafe13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:28.933 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe74c85640 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd4e10 0x55f4aafe13d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe74c85640 1 -- 172.21.15.59:0/2597971265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd5730 msgr2=0x55f4aafe1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe74c85640 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd5730 0x55f4aafe1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe74c85640 1 -- 172.21.15.59:0/2597971265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4aad18620 con 0x55f4aafe4500 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe67fff640 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd5730 0x55f4aafe1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe74c85640 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aafe1e50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbe680096c0 tx=0x7fbe6802a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d47463def8c365e0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.920+0000 7fbe4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/2597971265 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68034040 con 0x55f4aafe4500 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/2597971265 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7fbe68004250 con 0x55f4aafe4500 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe79408740 1 -- 172.21.15.59:0/2597971265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe60018af0 con 0x55f4aafe4500 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/2597971265 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbe6802fb00 con 0x55f4aafe4500 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe79408740 10 client.16311 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe79408740 1 -- 172.21.15.59:0/2597971265 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f4aaff2fb0 con 0x55f4aafe4500 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/2597971265 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe60018af0 con 0x55f4aafe4500 2024-05-08T05:36:28.934 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/2597971265 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbe68052ba0 con 0x55f4aafe4500 2024-05-08T05:36:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe79408740 10 client.16311 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe79408740 10 client.16311 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe79408740 20 client.16311 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe79408740 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4aaff4470 0x55f4aaff68c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe79408740 4 client.16311 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:36:28.935 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.921+0000 7fbe79408740 1 -- 172.21.15.59:0/2597971265 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4aaf481e0 con 0x55f4aaff4470 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.922+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4aaff4470 0x55f4aaff68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.922+0000 7fbe4f7fe640 10 client.16311 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.922+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4aaff4470 0x55f4aaff68c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d80af90410ac5b8 server_cookie=17d31efaebe1bfa5 in_seq=0 out_seq=0 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/2597971265 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4aaf481e0 con 0x55f4aaff4470 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe4f7fe640 10 client.16311 handle_command_reply: tid=0 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 client.16311 shutdown 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 -- 172.21.15.59:0/2597971265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 msgr2=0x55f4aafe1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aafe1e50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fbe680096c0 tx=0x7fbe6802a950 comp rx=0 tx=0).stop 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 -- 172.21.15.59:0/2597971265 shutdown_connections 2024-05-08T05:36:28.936 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4aaff4470 0x55f4aaff68c0 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aafe1e50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd5730 0x55f4aafe1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 --2- 172.21.15.59:0/2597971265 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd4e10 0x55f4aafe13d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe4f7fe640 0 client.16311 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 -- 172.21.15.59:0/2597971265 >> 172.21.15.59:0/2597971265 conn(0x55f4aafd1420 msgr2=0x55f4aafd7290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 -- 172.21.15.59:0/2597971265 shutdown_connections 2024-05-08T05:36:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 1 -- 172.21.15.59:0/2597971265 wait complete. 2024-05-08T05:36:28.937 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.923+0000 7fbe79408740 20 client.16311 trim_cache size 0 max 16384 2024-05-08T05:36:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe79408740 1 Processor -- start 2024-05-08T05:36:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe79408740 1 -- start start 2024-05-08T05:36:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe79408740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafdfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe79408740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd5730 0x55f4aafe01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.944 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe79408740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aaffc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe79408740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701adb60 con 0x55f4aafe4500 2024-05-08T05:36:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe79408740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701af6a0 con 0x55f4aafd4e10 2024-05-08T05:36:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe79408740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701ae230 con 0x55f4aafd5730 2024-05-08T05:36:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe74c85640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aaffc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe74c85640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aaffc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34000/0 (socket says 172.21.15.59:34000) 2024-05-08T05:36:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe74c85640 1 -- 172.21.15.59:0/1885363 learned_addr learned my addr 172.21.15.59:0/1885363 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafdfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe67fff640 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd5730 0x55f4aafe01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.945 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe74c85640 1 -- 172.21.15.59:0/1885363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd5730 msgr2=0x55f4aafe01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe74c85640 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd5730 0x55f4aafe01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe74c85640 1 -- 172.21.15.59:0/1885363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 msgr2=0x55f4aafdfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe74c85640 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafdfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe74c85640 1 -- 172.21.15.59:0/1885363 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4aafe4330 con 0x55f4aafe4500 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafdfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.939+0000 7fbe74c85640 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aaffc1d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbe68006010 tx=0x7fbe68003e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9360caf32214aff server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe65ffb640 1 -- 172.21.15.59:0/1885363 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68034030 con 0x55f4aafe4500 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe65ffb640 1 -- 172.21.15.59:0/1885363 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe6804d070 con 0x55f4aafe4500 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe65ffb640 1 -- 172.21.15.59:0/1885363 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe68052470 con 0x55f4aafe4500 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe79408740 1 -- 172.21.15.59:0/1885363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 msgr2=0x55f4aaffc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe79408740 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aaffc1d0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fbe68006010 tx=0x7fbe68003e30 comp rx=0 tx=0).stop 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe79408740 1 -- 172.21.15.59:0/1885363 shutdown_connections 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe79408740 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafe4500 0x55f4aaffc1d0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.946 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe79408740 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd5730 0x55f4aafe01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe79408740 1 --2- 172.21.15.59:0/1885363 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafd4e10 0x55f4aafdfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe79408740 1 -- 172.21.15.59:0/1885363 >> 172.21.15.59:0/1885363 conn(0x55f4aafd1420 msgr2=0x55f4aafd1800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe79408740 1 -- 172.21.15.59:0/1885363 shutdown_connections 2024-05-08T05:36:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.940+0000 7fbe79408740 1 -- 172.21.15.59:0/1885363 wait complete. 2024-05-08T05:36:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe79408740 1 -- start start 2024-05-08T05:36:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe79408740 1 Processor -- start 2024-05-08T05:36:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe79408740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd4e10 0x55f4aafec760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe79408740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd5730 0x55f4aafddc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.947 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe79408740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafe4500 0x55f4aafde180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe79408740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701aefd0 con 0x55f4aafd4e10 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe79408740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701ae900 con 0x55f4aafe4500 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe79408740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fbe701abb60 con 0x55f4aafd5730 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe79408740 10 client.0 _ll_register_callbacks cb 0x7fbe6802f740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd4e10 0x55f4aafec760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe6ffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd4e10 0x55f4aafec760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34016/0 (socket says 172.21.15.59:34016) 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.941+0000 7fbe6ffff640 1 -- 172.21.15.59:0/3208770951 learned_addr learned my addr 172.21.15.59:0/3208770951 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe6ffff640 1 -- 172.21.15.59:0/3208770951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd5730 msgr2=0x55f4aafddc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd5730 0x55f4aafddc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe6ffff640 1 -- 172.21.15.59:0/3208770951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafe4500 msgr2=0x55f4aafde180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafe4500 0x55f4aafde180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe6ffff640 1 -- 172.21.15.59:0/3208770951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f4aafdcfd0 con 0x55f4aafd4e10 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe6ffff640 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd4e10 0x55f4aafec760 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000ad00 tx=0x7fbe6000ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=886fcaf4252dde06 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe4f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:28.948 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/3208770951 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbe6001d040 con 0x55f4aafd4e10 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/3208770951 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7fbe60018540 con 0x55f4aafd4e10 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe79408740 1 -- 172.21.15.59:0/3208770951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbe6804d070 con 0x55f4aafd4e10 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/3208770951 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fbe60039050 con 0x55f4aafd4e10 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe79408740 10 client.16317 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe79408740 1 -- 172.21.15.59:0/3208770951 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55f4aafd5c70 con 0x55f4aafd4e10 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.942+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/3208770951 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbe6804d070 con 0x55f4aafd4e10 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/3208770951 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fbe60010800 con 0x55f4aafd4e10 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe79408740 10 client.16317 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe79408740 10 client.16317 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe79408740 20 client.16317 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe79408740 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4ab05ecc0 0x55f4ab061120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe79408740 4 client.16317 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe79408740 1 -- 172.21.15.59:0/3208770951 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4aaf42050 con 0x55f4ab05ecc0 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe67fff640 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4ab05ecc0 0x55f4ab061120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:28.949 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe4f7fe640 10 client.16317 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:28.950 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.943+0000 7fbe67fff640 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4ab05ecc0 0x55f4ab061120 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3c99e48244a6503 server_cookie=928f446a2d6e6684 in_seq=0 out_seq=0 2024-05-08T05:36:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe4f7fe640 1 -- 172.21.15.59:0/3208770951 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4aaf42050 con 0x55f4ab05ecc0 2024-05-08T05:36:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe4f7fe640 10 client.16317 handle_command_reply: tid=0 2024-05-08T05:36:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe79408740 1 client.16317 shutdown 2024-05-08T05:36:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe79408740 1 -- 172.21.15.59:0/3208770951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd4e10 msgr2=0x55f4aafec760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe79408740 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd4e10 0x55f4aafec760 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fbe6000ad00 tx=0x7fbe6000ac30 comp rx=0 tx=0).stop 2024-05-08T05:36:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe79408740 1 -- 172.21.15.59:0/3208770951 shutdown_connections 2024-05-08T05:36:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe79408740 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4ab05ecc0 0x55f4ab061120 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.951 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe79408740 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f4aafe4500 0x55f4aafde180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe4f7fe640 0 client.16317 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe79408740 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4aafd5730 0x55f4aafddc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe79408740 1 --2- 172.21.15.59:0/3208770951 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4aafd4e10 0x55f4aafec760 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.950+0000 7fbe79408740 1 -- 172.21.15.59:0/3208770951 >> 172.21.15.59:0/3208770951 conn(0x55f4aafd1420 msgr2=0x55f4aafdfb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe79408740 1 -- 172.21.15.59:0/3208770951 shutdown_connections 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe79408740 1 -- 172.21.15.59:0/3208770951 wait complete. 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe79408740 20 client.16317 trim_cache size 0 max 16384 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:36:28.952 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:36:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 -- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe5c0b2a10 msgr2=0x7fbe5c0b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe5c0b2a10 0x7fbe5c0b4ed0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fbe68009430 tx=0x7fbe68009360 comp rx=0 tx=0).stop 2024-05-08T05:36:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 -- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe70104580 msgr2=0x7fbe701995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:28.953 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe70104580 0x7fbe701995a0 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fbe5800d950 tx=0x7fbe5800dd50 comp rx=0 tx=0).stop 2024-05-08T05:36:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 -- 172.21.15.59:0/3621275386 shutdown_connections 2024-05-08T05:36:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fbe5c0b2a10 0x7fbe5c0b4ed0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fbe7010cfa0 0x7fbe7019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fbe70104580 0x7fbe701995a0 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 --2- 172.21.15.59:0/3621275386 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fbe70103c60 0x7fbe70199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 -- 172.21.15.59:0/3621275386 >> 172.21.15.59:0/3621275386 conn(0x7fbe70078a30 msgr2=0x7fbe700fe8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 -- 172.21.15.59:0/3621275386 shutdown_connections 2024-05-08T05:36:28.954 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:28.951+0000 7fbe766e8640 1 -- 172.21.15.59:0/3621275386 wait complete. 2024-05-08T05:36:29.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:29 smithi096 ceph-mon[34253]: pgmap v212: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 49 KiB/s wr, 11 op/s 2024-05-08T05:36:29.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:29 smithi059 ceph-mon[30304]: pgmap v212: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 49 KiB/s wr, 11 op/s 2024-05-08T05:36:29.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:29 smithi143 ceph-mon[34337]: pgmap v212: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.9 KiB/s rd, 49 KiB/s wr, 11 op/s 2024-05-08T05:36:31.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:31 smithi096 ceph-mon[34253]: pgmap v213: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-05-08T05:36:31.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:31 smithi059 ceph-mon[30304]: pgmap v213: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-05-08T05:36:31.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:31 smithi143 ceph-mon[34337]: pgmap v213: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 53 KiB/s wr, 16 op/s 2024-05-08T05:36:33.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:33 smithi096 ceph-mon[34253]: pgmap v214: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-05-08T05:36:33.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:33 smithi059 ceph-mon[30304]: pgmap v214: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-05-08T05:36:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:33 smithi143 ceph-mon[34337]: pgmap v214: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-05-08T05:36:33.601 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:36:33.870 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:36:35.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:35 smithi096 ceph-mon[34253]: pgmap v215: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-05-08T05:36:35.284 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:35 smithi059 ceph-mon[30304]: pgmap v215: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-05-08T05:36:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:35 smithi143 ceph-mon[34337]: pgmap v215: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-05-08T05:36:35.692 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.682+0000 7ff2d542c700 1 -- 172.21.15.59:0/2699802503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d01095e0 msgr2=0x7ff2d01099c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.682+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2699802503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d01095e0 0x7ff2d01099c0 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00099c0 tx=0x7ff2c0009cd0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.682+0000 7ff2d542c700 1 -- 172.21.15.59:0/2699802503 shutdown_connections 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.682+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2699802503 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d0103f00 0x7ff2d01103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.682+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2699802503 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d01095e0 0x7ff2d01099c0 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.682+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2699802503 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2d0103540 0x7ff2d01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.682+0000 7ff2d542c700 1 -- 172.21.15.59:0/2699802503 >> 172.21.15.59:0/2699802503 conn(0x7ff2d007a650 msgr2=0x7ff2d007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.683+0000 7ff2d542c700 1 -- 172.21.15.59:0/2699802503 shutdown_connections 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.683+0000 7ff2d542c700 1 -- 172.21.15.59:0/2699802503 wait complete. 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.683+0000 7ff2d542c700 1 Processor -- start 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.683+0000 7ff2d542c700 1 -- start start 2024-05-08T05:36:35.693 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2d542c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d0103540 0x7ff2d019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2d542c700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d0103f00 0x7ff2d019fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2d542c700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2d01095e0 0x7ff2d01998e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2d542c700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d01136e0 con 0x7ff2d0103540 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2d542c700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0113560 con 0x7ff2d01095e0 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2d542c700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff2d0113860 con 0x7ff2d0103f00 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ceffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d0103540 0x7ff2d019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ceffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d0103540 0x7ff2d019f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34028/0 (socket says 172.21.15.59:34028) 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ceffd700 1 -- 172.21.15.59:0/2341317986 learned_addr learned my addr 172.21.15.59:0/2341317986 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ce7fc700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d0103f00 0x7ff2d019fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ceffd700 1 -- 172.21.15.59:0/2341317986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d0103f00 msgr2=0x7ff2d019fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ceffd700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d0103f00 0x7ff2d019fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2cf7fe700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2d01095e0 0x7ff2d01998e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ceffd700 1 -- 172.21.15.59:0/2341317986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2d01095e0 msgr2=0x7ff2d01998e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ceffd700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2d01095e0 0x7ff2d01998e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ceffd700 1 -- 172.21.15.59:0/2341317986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2d019a040 con 0x7ff2d0103540 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2ceffd700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d0103540 0x7ff2d019f720 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00098e0 tx=0x7ff2c0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2c7fff700 1 -- 172.21.15.59:0/2341317986 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0027070 con 0x7ff2d0103540 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2d019a2d0 con 0x7ff2d0103540 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2c7fff700 1 -- 172.21.15.59:0/2341317986 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2c0003fa0 con 0x7ff2d0103540 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2c7fff700 1 -- 172.21.15.59:0/2341317986 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2c0022d40 con 0x7ff2d0103540 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.684+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2d01a65f0 con 0x7ff2d0103540 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.685+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2d010ba70 con 0x7ff2d0103540 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.687+0000 7ff2c7fff700 1 -- 172.21.15.59:0/2341317986 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff2c002a070 con 0x7ff2d0103540 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.688+0000 7ff2c7fff700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff2bc0b29f0 0x7ff2bc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.688+0000 7ff2ce7fc700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff2bc0b29f0 0x7ff2bc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.688+0000 7ff2c7fff700 1 -- 172.21.15.59:0/2341317986 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7ff2c00e3b00 con 0x7ff2d0103540 2024-05-08T05:36:35.695 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.688+0000 7ff2ce7fc700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff2bc0b29f0 0x7ff2bc0b4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8005950 tx=0x7ff2b80058e0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:35.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.691+0000 7ff2c7fff700 1 -- 172.21.15.59:0/2341317986 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff2c0013070 con 0x7ff2d0103540 2024-05-08T05:36:35.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.963+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2d019ae10 con 0x7ff2d0103540 2024-05-08T05:36:35.964 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.964+0000 7ff2c7fff700 1 -- 172.21.15.59:0/2341317986 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 36 v36) ==== 94+0+11929 (secure 0 0 0) 0x7ff2c00ab1a0 con 0x7ff2d0103540 2024-05-08T05:36:35.965 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:36:35.965 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":36,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":36,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:34:26.942121+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24415":{"gid":24415,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2504523054","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2504523054},{"type":"v1","addr":"172.21.15.143:6835","nonce":2504523054}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24443,24473,24421,24433,24419]},"id":1}]} 2024-05-08T05:36:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff2bc0b29f0 msgr2=0x7ff2bc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff2bc0b29f0 0x7ff2bc0b4eb0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff2b8005950 tx=0x7ff2b80058e0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d0103540 msgr2=0x7ff2d019f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d0103540 0x7ff2d019f720 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00098e0 tx=0x7ff2c0005e80 comp rx=0 tx=0).stop 2024-05-08T05:36:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 shutdown_connections 2024-05-08T05:36:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff2bc0b29f0 0x7ff2bc0b4eb0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.967 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2d01095e0 0x7ff2d01998e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2d0103540 0x7ff2d019f720 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 --2- 172.21.15.59:0/2341317986 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff2d0103f00 0x7ff2d019fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:35.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.966+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 >> 172.21.15.59:0/2341317986 conn(0x7ff2d007a650 msgr2=0x7ff2d0102090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:35.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.967+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 shutdown_connections 2024-05-08T05:36:35.968 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:35.967+0000 7ff2d542c700 1 -- 172.21.15.59:0/2341317986 wait complete. 2024-05-08T05:36:35.968 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 36 2024-05-08T05:36:36.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:36 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2341317986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:36:36.280 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:36 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2341317986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:36:36.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:36 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2341317986' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:36:36.610 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db --set-id 64a1b109 --release --await' 2024-05-08T05:36:36.610 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db --set-id 64a1b109 --release --await 2024-05-08T05:36:36.870 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:36:37.268 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:37 smithi059 ceph-mon[30304]: pgmap v216: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-05-08T05:36:37.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:37 smithi096 ceph-mon[34253]: pgmap v216: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-05-08T05:36:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:37 smithi143 ceph-mon[34337]: pgmap v216: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 3.9 KiB/s wr, 13 op/s 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.599+0000 7fed2ac37700 1 -- 172.21.15.59:0/1280751777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 msgr2=0x7fed241087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.599+0000 7fed2ac37700 1 --2- 172.21.15.59:0/1280751777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 0x7fed241087f0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7fed1400b0a0 tx=0x7fed1400b3b0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.599+0000 7fed2ac37700 1 -- 172.21.15.59:0/1280751777 shutdown_connections 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.599+0000 7fed2ac37700 1 --2- 172.21.15.59:0/1280751777 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed24102d60 0x7fed2410f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.599+0000 7fed2ac37700 1 --2- 172.21.15.59:0/1280751777 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 0x7fed241087f0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.599+0000 7fed2ac37700 1 --2- 172.21.15.59:0/1280751777 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed241023a0 0x7fed24102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.599+0000 7fed2ac37700 1 -- 172.21.15.59:0/1280751777 >> 172.21.15.59:0/1280751777 conn(0x7fed24079350 msgr2=0x7fed24079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.599+0000 7fed2ac37700 1 -- 172.21.15.59:0/1280751777 shutdown_connections 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.599+0000 7fed2ac37700 1 -- 172.21.15.59:0/1280751777 wait complete. 2024-05-08T05:36:38.601 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.600+0000 7fed2ac37700 1 Processor -- start 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.600+0000 7fed2ac37700 1 -- start start 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.600+0000 7fed2ac37700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed241023a0 0x7fed2419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.600+0000 7fed2ac37700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed24102d60 0x7fed2419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.600+0000 7fed2ac37700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 0x7fed241999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.600+0000 7fed2ac37700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed24112560 con 0x7fed24108410 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.600+0000 7fed2ac37700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fed241123e0 con 0x7fed24102d60 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.600+0000 7fed2ac37700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fed241126e0 con 0x7fed241023a0 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.600+0000 7fed291d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 0x7fed241999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed291d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 0x7fed241999d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34046/0 (socket says 172.21.15.59:34046) 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed291d4700 1 -- 172.21.15.59:0/3817939276 learned_addr learned my addr 172.21.15.59:0/3817939276 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed289d3700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed241023a0 0x7fed2419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.602 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed23fff700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed24102d60 0x7fed2419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed291d4700 1 -- 172.21.15.59:0/3817939276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed241023a0 msgr2=0x7fed2419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed291d4700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed241023a0 0x7fed2419f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed291d4700 1 -- 172.21.15.59:0/3817939276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed24102d60 msgr2=0x7fed2419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed291d4700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed24102d60 0x7fed2419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.603 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed291d4700 1 -- 172.21.15.59:0/3817939276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed2419a130 con 0x7fed24108410 2024-05-08T05:36:38.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed291d4700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 0x7fed241999d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00d980 tx=0x7fed1c00dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:38.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed21ffb700 1 -- 172.21.15.59:0/3817939276 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c017070 con 0x7fed24108410 2024-05-08T05:36:38.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed2ac37700 1 -- 172.21.15.59:0/3817939276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed2419a420 con 0x7fed24108410 2024-05-08T05:36:38.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed21ffb700 1 -- 172.21.15.59:0/3817939276 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed1c005020 con 0x7fed24108410 2024-05-08T05:36:38.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.601+0000 7fed21ffb700 1 -- 172.21.15.59:0/3817939276 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c00fa10 con 0x7fed24108410 2024-05-08T05:36:38.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.602+0000 7fed2ac37700 1 -- 172.21.15.59:0/3817939276 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed241a66e0 con 0x7fed24108410 2024-05-08T05:36:38.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.604+0000 7fed21ffb700 1 -- 172.21.15.59:0/3817939276 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fed1c004a30 con 0x7fed24108410 2024-05-08T05:36:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.605+0000 7fed21ffb700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fed100b2a70 0x7fed100b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.605+0000 7fed289d3700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fed100b2a70 0x7fed100b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.605+0000 7fed21ffb700 1 -- 172.21.15.59:0/3817939276 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7fed1c0e2760 con 0x7fed24108410 2024-05-08T05:36:38.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed289d3700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fed100b2a70 0x7fed100b4f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fed14009e60 tx=0x7fed1400b7b0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:38.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed354cab80 1 Processor -- start 2024-05-08T05:36:38.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed354cab80 1 -- start start 2024-05-08T05:36:38.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed354cab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1f27c0 0x55b79d1f2ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed354cab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d1ecef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed354cab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 0x55b79d1f9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed354cab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b79d13ea40 con 0x55b79d1eca70 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed354cab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b79d0a3ea0 con 0x55b79d1ed430 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed354cab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b79d13aa00 con 0x55b79d1f27c0 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed23fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d1ecef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed23fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d1ecef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34060/0 (socket says 172.21.15.59:34060) 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed23fff700 1 -- 172.21.15.59:0/3554705317 learned_addr learned my addr 172.21.15.59:0/3554705317 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed289d3700 1 --2- 172.21.15.59:0/3554705317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1f27c0 0x55b79d1f2ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed291d4700 1 --2- 172.21.15.59:0/3554705317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 0x55b79d1f9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed23fff700 1 -- 172.21.15.59:0/3554705317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1f27c0 msgr2=0x55b79d1f2ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed23fff700 1 --2- 172.21.15.59:0/3554705317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1f27c0 0x55b79d1f2ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed23fff700 1 -- 172.21.15.59:0/3554705317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 msgr2=0x55b79d1f9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed23fff700 1 --2- 172.21.15.59:0/3554705317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 0x55b79d1f9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.606+0000 7fed23fff700 1 -- 172.21.15.59:0/3554705317 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b79d08b9d0 con 0x55b79d1eca70 2024-05-08T05:36:38.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.607+0000 7fed23fff700 1 --2- 172.21.15.59:0/3554705317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d1ecef0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00ac70 tx=0x7fed0c00af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d410f2fffc04e4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:38.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.607+0000 7fed1affd700 1 -- 172.21.15.59:0/3554705317 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c01d070 con 0x55b79d1eca70 2024-05-08T05:36:38.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.607+0000 7fed1affd700 1 -- 172.21.15.59:0/3554705317 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed0c00fde0 con 0x55b79d1eca70 2024-05-08T05:36:38.609 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.607+0000 7fed1affd700 1 -- 172.21.15.59:0/3554705317 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c0107b0 con 0x55b79d1eca70 2024-05-08T05:36:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.607+0000 7fed354cab80 1 -- 172.21.15.59:0/3554705317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 msgr2=0x55b79d1ecef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.607+0000 7fed354cab80 1 --2- 172.21.15.59:0/3554705317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d1ecef0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00ac70 tx=0x7fed0c00af80 comp rx=0 tx=0).stop 2024-05-08T05:36:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.608+0000 7fed354cab80 1 -- 172.21.15.59:0/3554705317 shutdown_connections 2024-05-08T05:36:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.608+0000 7fed354cab80 1 --2- 172.21.15.59:0/3554705317 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 0x55b79d1f9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.608+0000 7fed354cab80 1 --2- 172.21.15.59:0/3554705317 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1f27c0 0x55b79d1f2ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.608+0000 7fed354cab80 1 --2- 172.21.15.59:0/3554705317 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d1ecef0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.608+0000 7fed354cab80 1 -- 172.21.15.59:0/3554705317 >> 172.21.15.59:0/3554705317 conn(0x55b79d118c70 msgr2=0x55b79d1e77f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.608+0000 7fed354cab80 1 -- 172.21.15.59:0/3554705317 shutdown_connections 2024-05-08T05:36:38.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.608+0000 7fed354cab80 1 -- 172.21.15.59:0/3554705317 wait complete. 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.608+0000 7fed354cab80 1 -- start start 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.608+0000 7fed354cab80 1 Processor -- start 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed354cab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 0x55b79d1f9160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed354cab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 0x55b79d1f6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed354cab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1f27c0 0x55b79d1f6a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed354cab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55b79d1f9ac0 con 0x55b79d1f27c0 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed354cab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55b79d0f7fd0 con 0x55b79d1ed430 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed354cab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55b79d1f9c40 con 0x55b79d1eca70 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed354cab80 10 client.0 _ll_register_callbacks cb 0x55b79d0d26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed291d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1f27c0 0x55b79d1f6a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed291d4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1f27c0 0x55b79d1f6a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34062/0 (socket says 172.21.15.59:34062) 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed291d4700 1 -- 172.21.15.59:0/725337172 learned_addr learned my addr 172.21.15.59:0/725337172 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed23fff700 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 0x55b79d1f6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed289d3700 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 0x55b79d1f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed291d4700 1 -- 172.21.15.59:0/725337172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 msgr2=0x55b79d1f9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed291d4700 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 0x55b79d1f9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed291d4700 1 -- 172.21.15.59:0/725337172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 msgr2=0x55b79d1f6500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed291d4700 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 0x55b79d1f6500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed291d4700 1 -- 172.21.15.59:0/725337172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b79d0f5860 con 0x55b79d1f27c0 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed291d4700 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1f27c0 0x55b79d1f6a40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0a68e0 tx=0x7fed1c0a9df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f15c3ff9f97098a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed03fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed03fff700 1 -- 172.21.15.59:0/725337172 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed1c0b24d0 con 0x55b79d1f27c0 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed03fff700 1 -- 172.21.15.59:0/725337172 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7fed1c0a98d0 con 0x55b79d1f27c0 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.609+0000 7fed354cab80 1 -- 172.21.15.59:0/725337172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed0c00fde0 con 0x55b79d1f27c0 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed03fff700 1 -- 172.21.15.59:0/725337172 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fed1c0a9ad0 con 0x55b79d1f27c0 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed354cab80 10 client.16335 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed354cab80 1 -- 172.21.15.59:0/725337172 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b79d2042f0 con 0x55b79d1f27c0 2024-05-08T05:36:38.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed03fff700 1 -- 172.21.15.59:0/725337172 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed0c00fde0 con 0x55b79d1f27c0 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed03fff700 1 -- 172.21.15.59:0/725337172 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fed1c0e2430 con 0x55b79d1f27c0 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed354cab80 10 client.16335 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed354cab80 10 client.16335 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed354cab80 20 client.16335 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed354cab80 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b79d206930 0x55b79d208d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed354cab80 4 client.16335 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed354cab80 1 -- 172.21.15.59:0/725337172 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b79d14c420 con 0x55b79d206930 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.610+0000 7fed289d3700 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b79d206930 0x55b79d208d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.611+0000 7fed03fff700 10 client.16335 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.611+0000 7fed289d3700 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b79d206930 0x55b79d208d20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62c24c0af91549ba server_cookie=666ed4de52f8d37c in_seq=0 out_seq=0 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed03fff700 1 -- 172.21.15.59:0/725337172 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55b79d14c420 con 0x55b79d206930 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed03fff700 10 client.16335 handle_command_reply: tid=0 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 client.16335 shutdown 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 -- 172.21.15.59:0/725337172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1f27c0 msgr2=0x55b79d1f6a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:38.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1f27c0 0x55b79d1f6a40 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fed1c0a68e0 tx=0x7fed1c0a9df0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 -- 172.21.15.59:0/725337172 shutdown_connections 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b79d206930 0x55b79d208d20 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed03fff700 0 client.16335 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1f27c0 0x55b79d1f6a40 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 0x55b79d1f9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 --2- 172.21.15.59:0/725337172 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1ed430 0x55b79d1f6500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 -- 172.21.15.59:0/725337172 >> 172.21.15.59:0/725337172 conn(0x55b79d118c70 msgr2=0x55b79d1f1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 -- 172.21.15.59:0/725337172 shutdown_connections 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 1 -- 172.21.15.59:0/725337172 wait complete. 2024-05-08T05:36:38.617 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.612+0000 7fed354cab80 20 client.16335 trim_cache size 0 max 16384 2024-05-08T05:36:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.632+0000 7fed354cab80 1 Processor -- start 2024-05-08T05:36:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed354cab80 1 -- start start 2024-05-08T05:36:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed354cab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d202b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed354cab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1ed430 0x55b79d1e1720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed354cab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 0x55b79d1ec170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed354cab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed241af5c0 con 0x55b79d1eca70 2024-05-08T05:36:38.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed354cab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fed241b1100 con 0x55b79d1f27c0 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed354cab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fed241afc90 con 0x55b79d1ed430 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed289d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d202b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed289d3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d202b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34068/0 (socket says 172.21.15.59:34068) 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed289d3700 1 -- 172.21.15.59:0/3994435772 learned_addr learned my addr 172.21.15.59:0/3994435772 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed289d3700 1 -- 172.21.15.59:0/3994435772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1ed430 msgr2=0x55b79d1e1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed23fff700 1 --2- 172.21.15.59:0/3994435772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1ed430 0x55b79d1e1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed289d3700 1 --2- 172.21.15.59:0/3994435772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1ed430 0x55b79d1e1720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed289d3700 1 -- 172.21.15.59:0/3994435772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 msgr2=0x55b79d1ec170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed291d4700 1 --2- 172.21.15.59:0/3994435772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 0x55b79d1ec170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed289d3700 1 --2- 172.21.15.59:0/3994435772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 0x55b79d1ec170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed289d3700 1 -- 172.21.15.59:0/3994435772 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b79d1e94b0 con 0x55b79d1eca70 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed289d3700 1 --2- 172.21.15.59:0/3994435772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d202b40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fed1400d010 tx=0x7fed14021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e592b148ed2357b9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.633+0000 7fed1a7fc700 1 -- 172.21.15.59:0/3994435772 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14007b20 con 0x55b79d1eca70 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed1a7fc700 1 -- 172.21.15.59:0/3994435772 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed14007cc0 con 0x55b79d1eca70 2024-05-08T05:36:38.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed1a7fc700 1 -- 172.21.15.59:0/3994435772 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed14005510 con 0x55b79d1eca70 2024-05-08T05:36:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed354cab80 1 -- 172.21.15.59:0/3994435772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 msgr2=0x55b79d202b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed354cab80 1 --2- 172.21.15.59:0/3994435772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d202b40 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7fed1400d010 tx=0x7fed14021000 comp rx=0 tx=0).stop 2024-05-08T05:36:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed354cab80 1 -- 172.21.15.59:0/3994435772 shutdown_connections 2024-05-08T05:36:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed354cab80 1 --2- 172.21.15.59:0/3994435772 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 0x55b79d1ec170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.638 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed354cab80 1 --2- 172.21.15.59:0/3994435772 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1eca70 0x55b79d202b40 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed354cab80 1 --2- 172.21.15.59:0/3994435772 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1ed430 0x55b79d1e1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed354cab80 1 -- 172.21.15.59:0/3994435772 >> 172.21.15.59:0/3994435772 conn(0x55b79d118c70 msgr2=0x55b79d1eba60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed354cab80 1 -- 172.21.15.59:0/3994435772 shutdown_connections 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.634+0000 7fed354cab80 1 -- 172.21.15.59:0/3994435772 wait complete. 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed354cab80 1 -- start start 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed354cab80 1 Processor -- start 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed354cab80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 0x55b79d1fbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed354cab80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1ed430 0x55b79d1fc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed354cab80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 0x55b79d1fc910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed354cab80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fed241b0a30 con 0x55b79d1ed430 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed354cab80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fed241b0360 con 0x55b79d1f27c0 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed354cab80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fed241ad580 con 0x55b79d1eca70 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed354cab80 10 client.0 _ll_register_callbacks cb 0x55b79d0d26b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:38.639 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed23fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1ed430 0x55b79d1fc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed23fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1ed430 0x55b79d1fc3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34082/0 (socket says 172.21.15.59:34082) 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed23fff700 1 -- 172.21.15.59:0/3005921191 learned_addr learned my addr 172.21.15.59:0/3005921191 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed289d3700 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 0x55b79d1fbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.635+0000 7fed291d4700 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 0x55b79d1fc910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed23fff700 1 -- 172.21.15.59:0/3005921191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 msgr2=0x55b79d1fbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed23fff700 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 0x55b79d1fbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed23fff700 1 -- 172.21.15.59:0/3005921191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 msgr2=0x55b79d1fc910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed23fff700 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 0x55b79d1fc910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed23fff700 1 -- 172.21.15.59:0/3005921191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b79d271140 con 0x55b79d1ed430 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed23fff700 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1ed430 0x55b79d1fc3d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00ac70 tx=0x7fed0c00a6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3aec09c3d1677e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed03fff700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed03fff700 1 -- 172.21.15.59:0/3005921191 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed0c029070 con 0x55b79d1ed430 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed354cab80 1 -- 172.21.15.59:0/3005921191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed14007cc0 con 0x55b79d1ed430 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed03fff700 1 -- 172.21.15.59:0/3005921191 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7fed0c005240 con 0x55b79d1ed430 2024-05-08T05:36:38.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed03fff700 1 -- 172.21.15.59:0/3005921191 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7fed0c035050 con 0x55b79d1ed430 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed354cab80 10 client.16341 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed354cab80 1 -- 172.21.15.59:0/3005921191 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x55b79d2723c0 con 0x55b79d1ed430 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.636+0000 7fed03fff700 1 -- 172.21.15.59:0/3005921191 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed14007cc0 con 0x55b79d1ed430 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed03fff700 1 -- 172.21.15.59:0/3005921191 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7fed0c018440 con 0x55b79d1ed430 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed354cab80 10 client.16341 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed354cab80 10 client.16341 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed354cab80 20 client.16341 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed354cab80 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b79d206930 0x55b79d274800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed354cab80 4 client.16341 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "64a1b109", "release": true, "await": true}] 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed354cab80 1 -- 172.21.15.59:0/3005921191 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "64a1b109", "release": true, "await": true}) -- 0x55b79d1464c0 con 0x55b79d206930 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed289d3700 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b79d206930 0x55b79d274800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed03fff700 10 client.16341 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:38.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.637+0000 7fed289d3700 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b79d206930 0x55b79d274800 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5d987db8169f362 server_cookie=74dae4b8ca528896 in_seq=0 out_seq=0 2024-05-08T05:36:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed03fff700 1 -- 172.21.15.59:0/3005921191 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55b79d1464c0 con 0x55b79d206930 2024-05-08T05:36:38.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed03fff700 10 client.16341 handle_command_reply: tid=0 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 client.16341 shutdown 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 -- 172.21.15.59:0/3005921191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1ed430 msgr2=0x55b79d1fc3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1ed430 0x55b79d1fc3d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7fed0c00ac70 tx=0x7fed0c00a6a0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 -- 172.21.15.59:0/3005921191 shutdown_connections 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55b79d206930 0x55b79d274800 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55b79d1f27c0 0x55b79d1fc910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55b79d1eca70 0x55b79d1fbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed03fff700 0 client.16341 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 --2- 172.21.15.59:0/3005921191 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55b79d1ed430 0x55b79d1fc3d0 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 -- 172.21.15.59:0/3005921191 >> 172.21.15.59:0/3005921191 conn(0x55b79d118c70 msgr2=0x55b79d1ec2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.842+0000 7fed354cab80 1 -- 172.21.15.59:0/3005921191 shutdown_connections 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.843+0000 7fed354cab80 1 -- 172.21.15.59:0/3005921191 wait complete. 2024-05-08T05:36:38.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.843+0000 7fed354cab80 20 client.16341 trim_cache size 0 max 16384 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 36, 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 24, 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "64a1b109": { 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 24, 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:36:38.845 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 85.7, 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:38.846 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:38.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:38.847 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:36:38.847 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:36:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.843+0000 7fed2ac37700 1 -- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fed100b2a70 msgr2=0x7fed100b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.843+0000 7fed2ac37700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fed100b2a70 0x7fed100b4f30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fed14009e60 tx=0x7fed1400b7b0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.843+0000 7fed2ac37700 1 -- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 msgr2=0x7fed241999d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:38.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.843+0000 7fed2ac37700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 0x7fed241999d0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7fed1c00d980 tx=0x7fed1c00dc90 comp rx=0 tx=0).stop 2024-05-08T05:36:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.844+0000 7fed2ac37700 1 -- 172.21.15.59:0/3817939276 shutdown_connections 2024-05-08T05:36:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.844+0000 7fed2ac37700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fed100b2a70 0x7fed100b4f30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.844+0000 7fed2ac37700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fed24108410 0x7fed241999d0 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.844+0000 7fed2ac37700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fed241023a0 0x7fed2419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.844+0000 7fed2ac37700 1 --2- 172.21.15.59:0/3817939276 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fed24102d60 0x7fed2419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.844+0000 7fed2ac37700 1 -- 172.21.15.59:0/3817939276 >> 172.21.15.59:0/3817939276 conn(0x7fed24079350 msgr2=0x7fed24101750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.844+0000 7fed2ac37700 1 -- 172.21.15.59:0/3817939276 shutdown_connections 2024-05-08T05:36:38.848 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:36:38.844+0000 7fed2ac37700 1 -- 172.21.15.59:0/3817939276 wait complete. 2024-05-08T05:36:39.182 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:39 smithi059 ceph-mon[30304]: pgmap v217: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-05-08T05:36:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:39 smithi143 ceph-mon[34337]: pgmap v217: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-05-08T05:36:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:39 smithi096 ceph-mon[34253]: pgmap v217: 129 pgs: 129 active+clean; 47 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 7.6 KiB/s rd, 3.9 KiB/s wr, 8 op/s 2024-05-08T05:36:39.621 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set '64a1b109', seconds elapsed: 40.3 2024-05-08T05:36:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:41 smithi059 ceph-mon[30304]: pgmap v218: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 207 KiB/s wr, 46 op/s 2024-05-08T05:36:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:41 smithi143 ceph-mon[34337]: pgmap v218: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 207 KiB/s wr, 46 op/s 2024-05-08T05:36:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:41 smithi096 ceph-mon[34253]: pgmap v218: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 676 KiB/s rd, 207 KiB/s wr, 46 op/s 2024-05-08T05:36:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.072+0000 7ff9a38e3640 1 -- 172.21.15.59:0/3543105436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff99c1089e0 msgr2=0x7ff99c108dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.072+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/3543105436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff99c1089e0 0x7ff99c108dc0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7ff998009890 tx=0x7ff99802a380 comp rx=0 tx=0).stop 2024-05-08T05:36:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.072+0000 7ff9a38e3640 1 -- 172.21.15.59:0/3543105436 shutdown_connections 2024-05-08T05:36:42.073 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.072+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/3543105436 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c103390 0x7ff99c10f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.072+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/3543105436 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99c1029d0 0x7ff99c102e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.072+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/3543105436 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff99c1089e0 0x7ff99c108dc0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.072+0000 7ff9a38e3640 1 -- 172.21.15.59:0/3543105436 >> 172.21.15.59:0/3543105436 conn(0x7ff99c0fe460 msgr2=0x7ff99c100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.073+0000 7ff9a38e3640 1 -- 172.21.15.59:0/3543105436 shutdown_connections 2024-05-08T05:36:42.074 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.073+0000 7ff9a38e3640 1 -- 172.21.15.59:0/3543105436 wait complete. 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.073+0000 7ff9a38e3640 1 Processor -- start 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a38e3640 1 -- start start 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a38e3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99c1029d0 0x7ff99c116f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a38e3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff99c103390 0x7ff99c117490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a38e3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c1089e0 0x7ff99c112060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a38e3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c076270 con 0x7ff99c1089e0 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a38e3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c0760f0 con 0x7ff99c1029d0 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a38e3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c0763f0 con 0x7ff99c103390 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a167f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99c1029d0 0x7ff99c116f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.076 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a1e80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c1089e0 0x7ff99c112060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a0e7e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff99c103390 0x7ff99c117490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a167f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99c1029d0 0x7ff99c116f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:42966/0 (socket says 172.21.15.59:42966) 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a1e80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c1089e0 0x7ff99c112060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45880/0 (socket says 172.21.15.59:45880) 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a167f640 1 -- 172.21.15.59:0/2685463582 learned_addr learned my addr 172.21.15.59:0/2685463582 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/2685463582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99c1029d0 msgr2=0x7ff99c116f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99c1029d0 0x7ff99c116f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c1089e0 msgr2=0x7ff99c112060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c1089e0 0x7ff99c112060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.074+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/2685463582 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff99c112920 con 0x7ff99c103390 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.075+0000 7ff9a167f640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99c1029d0 0x7ff99c116f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.075+0000 7ff9a1e80640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c1089e0 0x7ff99c112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.075+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff99c103390 0x7ff99c117490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00e9f0 tx=0x7ff98c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.075+0000 7ff9927fc640 1 -- 172.21.15.59:0/2685463582 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c00ccc0 con 0x7ff99c103390 2024-05-08T05:36:42.077 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.075+0000 7ff9927fc640 1 -- 172.21.15.59:0/2685463582 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98c005020 con 0x7ff99c103390 2024-05-08T05:36:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.075+0000 7ff9927fc640 1 -- 172.21.15.59:0/2685463582 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c004a00 con 0x7ff99c103390 2024-05-08T05:36:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.075+0000 7ff9a38e3640 1 -- 172.21.15.59:0/2685463582 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff99c112c10 con 0x7ff99c103390 2024-05-08T05:36:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.075+0000 7ff9a38e3640 1 -- 172.21.15.59:0/2685463582 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff99c11ce40 con 0x7ff99c103390 2024-05-08T05:36:42.078 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.078+0000 7ff9927fc640 1 -- 172.21.15.59:0/2685463582 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff98c004ba0 con 0x7ff99c103390 2024-05-08T05:36:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.078+0000 7ff9a65fd740 1 Processor -- start 2024-05-08T05:36:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a65fd740 1 -- start start 2024-05-08T05:36:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d612f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.080 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d6bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 0x558311d6e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558311cc65d0 con 0x558311d60f10 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558311c7f890 con 0x558311d6c0f0 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558311cc2590 con 0x558311d61830 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a1e80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d612f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a167f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d6bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a167f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d6bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:60688/0 (socket says 172.21.15.59:60688) 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a167f640 1 -- 172.21.15.59:0/2973713588 learned_addr learned my addr 172.21.15.59:0/2973713588 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9927fc640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff9880b2a10 0x7ff9880b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/2973713588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 0x558311d6e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a1e80640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff9880b2a10 0x7ff9880b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a167f640 1 -- 172.21.15.59:0/2973713588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 msgr2=0x558311d6e4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:42.081 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a167f640 1 --2- 172.21.15.59:0/2973713588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 0x558311d6e4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a167f640 1 -- 172.21.15.59:0/2973713588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 msgr2=0x558311d612f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a167f640 1 --2- 172.21.15.59:0/2973713588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d612f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.079+0000 7ff9a167f640 1 -- 172.21.15.59:0/2973713588 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583119d8210 con 0x558311d61830 2024-05-08T05:36:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.080+0000 7ff9927fc640 1 -- 172.21.15.59:0/2685463582 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7ff98c0e2950 con 0x7ff99c103390 2024-05-08T05:36:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.080+0000 7ff9a167f640 1 --2- 172.21.15.59:0/2973713588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d6bbb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9980099b0 tx=0x7ff998005dc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2743531accfd40bb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.080+0000 7ff9877fe640 1 -- 172.21.15.59:0/2973713588 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998041020 con 0x558311d61830 2024-05-08T05:36:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.080+0000 7ff9a1e80640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff9880b2a10 0x7ff9880b4ed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff994009590 tx=0x7ff994004ab0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.080+0000 7ff9877fe640 1 -- 172.21.15.59:0/2973713588 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff998030dd0 con 0x558311d61830 2024-05-08T05:36:42.082 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.080+0000 7ff9877fe640 1 -- 172.21.15.59:0/2973713588 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff998037040 con 0x558311d61830 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2973713588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 msgr2=0x558311d6bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/2973713588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d6bbb0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7ff9980099b0 tx=0x7ff998005dc0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2973713588 shutdown_connections 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/2973713588 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 0x558311d6e4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/2973713588 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d6bbb0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/2973713588 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d612f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2973713588 >> 172.21.15.59:0/2973713588 conn(0x558311d5d420 msgr2=0x558311d5d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2973713588 shutdown_connections 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2973713588 wait complete. 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 -- start start 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.081+0000 7ff9a65fd740 1 Processor -- start 2024-05-08T05:36:42.087 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d68ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 0x558311d69530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d6c0f0 0x558311d69a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x558311d6ecc0 con 0x558311d60f10 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x558311c83370 con 0x558311d61830 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x558311d6ee40 con 0x558311d6c0f0 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a65fd740 10 client.0 _ll_register_callbacks cb 0x5583119d7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a167f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d68ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a0e7e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 0x558311d69530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a167f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d68ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45900/0 (socket says 172.21.15.59:45900) 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a167f640 1 -- 172.21.15.59:0/3145697936 learned_addr learned my addr 172.21.15.59:0/3145697936 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a1e80640 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d6c0f0 0x558311d69a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/3145697936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d6c0f0 msgr2=0x558311d69a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d6c0f0 0x558311d69a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/3145697936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 msgr2=0x558311d68ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:42.088 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d68ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/3145697936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558311aa4620 con 0x558311d61830 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a167f640 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d68ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 0x558311d69530 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0dfcd0 tx=0x7ff98c004f50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c321cd60b3e1180a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.082+0000 7ff984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff984ff9640 1 -- 172.21.15.59:0/3145697936 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c01c070 con 0x558311d61830 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a65fd740 1 -- 172.21.15.59:0/3145697936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff998030dd0 con 0x558311d61830 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff984ff9640 1 -- 172.21.15.59:0/3145697936 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7ff98c0dff50 con 0x558311d61830 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff984ff9640 1 -- 172.21.15.59:0/3145697936 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff98c0ade70 con 0x558311d61830 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a65fd740 10 client.25901 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a65fd740 1 -- 172.21.15.59:0/3145697936 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558311d7aaf0 con 0x558311d61830 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff984ff9640 1 -- 172.21.15.59:0/3145697936 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff998030dd0 con 0x558311d61830 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff984ff9640 1 -- 172.21.15.59:0/3145697936 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7ff98c0a5af0 con 0x558311d61830 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a65fd740 10 client.25901 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:42.089 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a65fd740 10 client.25901 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a65fd740 20 client.25901 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558311d7bf60 0x558311d7e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a65fd740 4 client.25901 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a65fd740 1 -- 172.21.15.59:0/3145697936 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558311cd41e0 con 0x558311d7bf60 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.083+0000 7ff9a167f640 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558311d7bf60 0x558311d7e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.084+0000 7ff9a167f640 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558311d7bf60 0x558311d7e390 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be9c2ce88422c56a server_cookie=498e3739895fb52b in_seq=0 out_seq=0 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.084+0000 7ff984ff9640 10 client.25901 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff984ff9640 1 -- 172.21.15.59:0/3145697936 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x558311cd41e0 con 0x558311d7bf60 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff984ff9640 10 client.25901 handle_command_reply: tid=0 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 client.25901 shutdown 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 -- 172.21.15.59:0/3145697936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 msgr2=0x558311d69530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 0x558311d69530 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7ff98c0dfcd0 tx=0x7ff98c004f50 comp rx=0 tx=0).stop 2024-05-08T05:36:42.090 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 -- 172.21.15.59:0/3145697936 shutdown_connections 2024-05-08T05:36:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558311d7bf60 0x558311d7e390 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d6c0f0 0x558311d69a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 0x558311d69530 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff984ff9640 0 client.25901 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/3145697936 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d68ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 -- 172.21.15.59:0/3145697936 >> 172.21.15.59:0/3145697936 conn(0x558311d5d420 msgr2=0x558311d631a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 -- 172.21.15.59:0/3145697936 shutdown_connections 2024-05-08T05:36:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 1 -- 172.21.15.59:0/3145697936 wait complete. 2024-05-08T05:36:42.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.085+0000 7ff9a65fd740 20 client.25901 trim_cache size 0 max 16384 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a65fd740 1 Processor -- start 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a65fd740 1 -- start start 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d58450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d83c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 0x558311d841b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1beb90 con 0x558311d60f10 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1c06d0 con 0x558311d6c0f0 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1bf260 con 0x558311d61830 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a167f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d58450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a0e7e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d83c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a0e7e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d83c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:60704/0 (socket says 172.21.15.59:60704) 2024-05-08T05:36:42.105 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/2561336066 learned_addr learned my addr 172.21.15.59:0/2561336066 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.104+0000 7ff9a1e80640 1 --2- 172.21.15.59:0/2561336066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 0x558311d841b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.105+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/2561336066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 msgr2=0x558311d841b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.105+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/2561336066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 0x558311d841b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.105+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/2561336066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 msgr2=0x558311d58450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.105+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/2561336066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d58450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.105+0000 7ff9a0e7e640 1 -- 172.21.15.59:0/2561336066 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558311d67540 con 0x558311d61830 2024-05-08T05:36:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.105+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/2561336066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d83c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c6a0 tx=0x7ff98c009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6505bbe822d1b8ee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:42.109 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.105+0000 7ff986ffd640 1 -- 172.21.15.59:0/2561336066 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c01c070 con 0x558311d61830 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.105+0000 7ff986ffd640 1 -- 172.21.15.59:0/2561336066 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98c0a9370 con 0x558311d61830 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.105+0000 7ff986ffd640 1 -- 172.21.15.59:0/2561336066 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff98c0a5af0 con 0x558311d61830 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.106+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2561336066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 msgr2=0x558311d83c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.106+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/2561336066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d83c70 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00c6a0 tx=0x7ff98c009f60 comp rx=0 tx=0).stop 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.106+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2561336066 shutdown_connections 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.106+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/2561336066 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d6c0f0 0x558311d841b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.106+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/2561336066 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d61830 0x558311d83c70 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.106+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/2561336066 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d60f10 0x558311d58450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.106+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2561336066 >> 172.21.15.59:0/2561336066 conn(0x558311d5d420 msgr2=0x558311d5d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.106+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2561336066 shutdown_connections 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.106+0000 7ff9a65fd740 1 -- 172.21.15.59:0/2561336066 wait complete. 2024-05-08T05:36:42.110 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a65fd740 1 -- start start 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a65fd740 1 Processor -- start 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d60f10 0x558311d74680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 0x558311d6f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a65fd740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d6c0f0 0x558311d6ff70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1c0000 con 0x558311d6c0f0 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1bf930 con 0x558311d61830 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a65fd740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff99c1bcb90 con 0x558311d60f10 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a65fd740 10 client.0 _ll_register_callbacks cb 0x7ff98c0046c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a1e80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d6c0f0 0x558311d6ff70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a1e80640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d6c0f0 0x558311d6ff70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45920/0 (socket says 172.21.15.59:45920) 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a0e7e640 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 0x558311d6f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a1e80640 1 -- 172.21.15.59:0/1811693813 learned_addr learned my addr 172.21.15.59:0/1811693813 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.107+0000 7ff9a167f640 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d60f10 0x558311d74680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff9a1e80640 1 -- 172.21.15.59:0/1811693813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d60f10 msgr2=0x558311d74680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:42.111 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff9a1e80640 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d60f10 0x558311d74680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff9a1e80640 1 -- 172.21.15.59:0/1811693813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 msgr2=0x558311d6f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff9a1e80640 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 0x558311d6f9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff9a1e80640 1 -- 172.21.15.59:0/1811693813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558311d707a0 con 0x558311d6c0f0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff9a1e80640 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d6c0f0 0x558311d6ff70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff994003bd0 tx=0x7ff994004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20a4ff0e5e3688d4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff984ff9640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff984ff9640 1 -- 172.21.15.59:0/1811693813 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff994030060 con 0x558311d6c0f0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff984ff9640 1 -- 172.21.15.59:0/1811693813 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9496+0+0 (secure 0 0 0) 0x7ff9940047d0 con 0x558311d6c0f0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff9a65fd740 1 -- 172.21.15.59:0/1811693813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff98c0a9370 con 0x558311d6c0f0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff984ff9640 1 -- 172.21.15.59:0/1811693813 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=36) ==== 24+0+0 (secure 0 0 0) 0x7ff994037050 con 0x558311d6c0f0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff9a65fd740 10 client.16359 fetch_fsmap learned FSMap version 36 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.108+0000 7ff9a65fd740 1 -- 172.21.15.59:0/1811693813 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=36,monmap=4+}) -- 0x558311de6930 con 0x558311d6c0f0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff984ff9640 1 -- 172.21.15.59:0/1811693813 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff98c0a9370 con 0x558311d6c0f0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff984ff9640 1 -- 172.21.15.59:0/1811693813 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 36) ==== 8362+0+0 (secure 0 0 0) 0x7ff99400b040 con 0x558311d6c0f0 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff9a65fd740 10 client.16359 fetch_fsmap finished waiting for FSMap version 36 2024-05-08T05:36:42.112 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff9a65fd740 10 client.16359 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff9a65fd740 20 client.16359 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558311de70e0 0x558311de9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff9a65fd740 4 client.16359 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:36:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff9a65fd740 1 -- 172.21.15.59:0/1811693813 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558311cce050 con 0x558311de70e0 2024-05-08T05:36:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff9a167f640 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558311de70e0 0x558311de9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff984ff9640 10 client.16359 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:42.113 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.109+0000 7ff9a167f640 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558311de70e0 0x558311de9500 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2817be72bd525105 server_cookie=a9c5817e245da3c1 in_seq=0 out_seq=0 2024-05-08T05:36:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff984ff9640 1 -- 172.21.15.59:0/1811693813 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558311cce050 con 0x558311de70e0 2024-05-08T05:36:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff984ff9640 10 client.16359 handle_command_reply: tid=0 2024-05-08T05:36:42.119 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff9a65fd740 1 client.16359 shutdown 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff9a65fd740 1 -- 172.21.15.59:0/1811693813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d6c0f0 msgr2=0x558311d6ff70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d6c0f0 0x558311d6ff70 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7ff994003bd0 tx=0x7ff994004a40 comp rx=0 tx=0).stop 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff9a65fd740 1 -- 172.21.15.59:0/1811693813 shutdown_connections 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x558311de70e0 0x558311de9500 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x558311d6c0f0 0x558311d6ff70 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x558311d61830 0x558311d6f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff9a65fd740 1 --2- 172.21.15.59:0/1811693813 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x558311d60f10 0x558311d74680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.118+0000 7ff9a65fd740 1 -- 172.21.15.59:0/1811693813 >> 172.21.15.59:0/1811693813 conn(0x558311d5d420 msgr2=0x558311d58450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.119+0000 7ff984ff9640 0 client.16359 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.119+0000 7ff9a65fd740 1 -- 172.21.15.59:0/1811693813 shutdown_connections 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.119+0000 7ff9a65fd740 1 -- 172.21.15.59:0/1811693813 wait complete. 2024-05-08T05:36:42.120 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.119+0000 7ff9a65fd740 20 client.16359 trim_cache size 0 max 16384 2024-05-08T05:36:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:36:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:36:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:36:42.121 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 -- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff9880b2a10 msgr2=0x7ff9880b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff9880b2a10 0x7ff9880b4ed0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ff994009590 tx=0x7ff994004ab0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 -- 172.21.15.59:0/2685463582 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff99c103390 msgr2=0x7ff99c117490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff99c103390 0x7ff99c117490 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7ff98c00e9f0 tx=0x7ff98c00eeb0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 -- 172.21.15.59:0/2685463582 shutdown_connections 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff9880b2a10 0x7ff9880b4ed0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff99c1089e0 0x7ff99c112060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff99c103390 0x7ff99c117490 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 --2- 172.21.15.59:0/2685463582 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff99c1029d0 0x7ff99c116f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 -- 172.21.15.59:0/2685463582 >> 172.21.15.59:0/2685463582 conn(0x7ff99c0fe460 msgr2=0x7ff99c0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:42.122 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 -- 172.21.15.59:0/2685463582 shutdown_connections 2024-05-08T05:36:42.123 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:42.120+0000 7ff9a38e3640 1 -- 172.21.15.59:0/2685463582 wait complete. 2024-05-08T05:36:42.993 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:36:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f[64685]: ignoring --setuser ceph since I am not root 2024-05-08T05:36:42.993 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:36:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f[64685]: ignoring --setgroup ceph since I am not root 2024-05-08T05:36:42.993 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:36:42 smithi143 ceph-mds[64689]: -- 172.21.15.143:0/738320258 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55dddd799a40 con 0x55dddd76c800 2024-05-08T05:36:42.993 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:36:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f[64685]: starting mds.f at 2024-05-08T05:36:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:43 smithi059 ceph-mon[30304]: pgmap v219: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 203 KiB/s wr, 41 op/s 2024-05-08T05:36:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:43 smithi059 ceph-mon[30304]: daemon mds.f restarted 2024-05-08T05:36:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:43 smithi143 ceph-mon[34337]: pgmap v219: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 203 KiB/s wr, 41 op/s 2024-05-08T05:36:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:43 smithi143 ceph-mon[34337]: daemon mds.f restarted 2024-05-08T05:36:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:43 smithi096 ceph-mon[34253]: pgmap v219: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 671 KiB/s rd, 203 KiB/s wr, 41 op/s 2024-05-08T05:36:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:43 smithi096 ceph-mon[34253]: daemon mds.f restarted 2024-05-08T05:36:43.592 INFO:tasks.workunit.client.0.smithi059.stdout:50000 write 0x5304f4 thru 0x53fa11 (0xf51e bytes) 2024-05-08T05:36:43.592 INFO:tasks.workunit.client.0.smithi059.stdout:All 50000 operations completed A-OK! 2024-05-08T05:36:43.594 INFO:tasks.workunit.client.0.smithi059.stderr:+ ./fsx -z 100MB -N 50000 -p 10000 -l 104857600 2024-05-08T05:36:43.597 INFO:tasks.workunit.client.0.smithi059.stdout:Seed set to 1 2024-05-08T05:36:44.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:44 smithi059 ceph-mon[30304]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T05:36:44.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:44 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6834/2351774265,v1:172.21.15.143:6835/2351774265] up:boot 2024-05-08T05:36:44.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:44 smithi059 ceph-mon[30304]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-08T05:36:44.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:44 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:36:44.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:44 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:36:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:44 smithi143 ceph-mon[34337]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T05:36:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:44 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6834/2351774265,v1:172.21.15.143:6835/2351774265] up:boot 2024-05-08T05:36:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:44 smithi143 ceph-mon[34337]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-08T05:36:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:44 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:36:44.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:44 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:36:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:44 smithi096 ceph-mon[34253]: osdmap e94: 12 total, 12 up, 12 in 2024-05-08T05:36:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:44 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6834/2351774265,v1:172.21.15.143:6835/2351774265] up:boot 2024-05-08T05:36:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:44 smithi096 ceph-mon[34253]: fsmap cephfs:5 4 up:standby-replay 3 up:standby 5 up:active 2024-05-08T05:36:44.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:44 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:36:44.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:44 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:36:44.806 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode 0, disabling! 2024-05-08T05:36:44.806 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode FALLOC_FL_KEEP_SIZE, disabling! 2024-05-08T05:36:44.808 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode FALLOC_FL_COLLAPSE_RANGE, disabling! 2024-05-08T05:36:44.808 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support fallocate mode FALLOC_FL_INSERT_RANGE, disabling! 2024-05-08T05:36:44.808 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support clone range, disabling! 2024-05-08T05:36:44.810 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support dedupe range, disabling! 2024-05-08T05:36:44.813 INFO:tasks.workunit.client.0.smithi059.stderr:main: filesystem does not support exchange range, disabling! 2024-05-08T05:36:44.813 INFO:tasks.workunit.client.0.smithi059.stdout:skipping zero size read 2024-05-08T05:36:44.824 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0x1c37b90 2024-05-08T05:36:44.831 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0x5bd330d 2024-05-08T05:36:44.852 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0x637bf99 2024-05-08T05:36:45.120 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0x63d9b9c 2024-05-08T05:36:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:45 smithi059 ceph-mon[30304]: pgmap v221: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 244 KiB/s wr, 49 op/s 2024-05-08T05:36:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:45 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:36:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:45 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:36:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:45 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:36:45.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:45 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:36:45.393 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:45 smithi143 ceph-mon[34337]: pgmap v221: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 244 KiB/s wr, 49 op/s 2024-05-08T05:36:45.393 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:45 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:36:45.394 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:45 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:36:45.394 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:45 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:36:45.394 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:45 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:36:45.430 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:45 smithi096 ceph-mon[34253]: pgmap v221: 129 pgs: 129 active+clean; 45 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 805 KiB/s rd, 244 KiB/s wr, 49 op/s 2024-05-08T05:36:45.431 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:45 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:36:45.431 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:45 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:36:45.431 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:45 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:36:45.431 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:45 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:36:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:47 smithi059 ceph-mon[30304]: pgmap v222: 129 pgs: 129 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 195 op/s 2024-05-08T05:36:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:47 smithi143 ceph-mon[34337]: pgmap v222: 129 pgs: 129 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 195 op/s 2024-05-08T05:36:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:47 smithi096 ceph-mon[34253]: pgmap v222: 129 pgs: 129 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 195 op/s 2024-05-08T05:36:49.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:49 smithi059 ceph-mon[30304]: pgmap v223: 129 pgs: 129 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 195 op/s 2024-05-08T05:36:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:49 smithi143 ceph-mon[34337]: pgmap v223: 129 pgs: 129 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 195 op/s 2024-05-08T05:36:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:49 smithi096 ceph-mon[34253]: pgmap v223: 129 pgs: 129 active+clean; 50 MiB data, 1.5 GiB used, 1.0 TiB / 1.0 TiB avail; 3.0 MiB/s rd, 1.1 MiB/s wr, 195 op/s 2024-05-08T05:36:51.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:51 smithi059 ceph-mon[30304]: pgmap v224: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.8 MiB/s wr, 470 op/s 2024-05-08T05:36:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:51 smithi143 ceph-mon[34337]: pgmap v224: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.8 MiB/s wr, 470 op/s 2024-05-08T05:36:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:51 smithi096 ceph-mon[34253]: pgmap v224: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.8 MiB/s wr, 470 op/s 2024-05-08T05:36:53.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:53 smithi059 ceph-mon[30304]: pgmap v225: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.8 MiB/s wr, 470 op/s 2024-05-08T05:36:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:53 smithi143 ceph-mon[34337]: pgmap v225: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.8 MiB/s wr, 470 op/s 2024-05-08T05:36:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:53 smithi096 ceph-mon[34253]: pgmap v225: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 6.3 MiB/s rd, 1.8 MiB/s wr, 470 op/s 2024-05-08T05:36:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.241+0000 7f52b28d3640 1 -- 172.21.15.59:0/1013431004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac1079e0 msgr2=0x7f52ac107dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.241+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1013431004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac1079e0 0x7f52ac107dc0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f529c009900 tx=0x7f529c02a410 comp rx=0 tx=0).stop 2024-05-08T05:36:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.241+0000 7f52b28d3640 1 -- 172.21.15.59:0/1013431004 shutdown_connections 2024-05-08T05:36:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.241+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1013431004 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ac06bb50 0x7f52ac10f870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.241+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1013431004 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52ac06b190 0x7f52ac06b610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.241+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1013431004 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac1079e0 0x7f52ac107dc0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.241+0000 7f52b28d3640 1 -- 172.21.15.59:0/1013431004 >> 172.21.15.59:0/1013431004 conn(0x7f52ac078690 msgr2=0x7f52ac078aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.241+0000 7f52b28d3640 1 -- 172.21.15.59:0/1013431004 shutdown_connections 2024-05-08T05:36:55.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.241+0000 7f52b28d3640 1 -- 172.21.15.59:0/1013431004 wait complete. 2024-05-08T05:36:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.242+0000 7f52b28d3640 1 Processor -- start 2024-05-08T05:36:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.245+0000 7f52b28d3640 1 -- start start 2024-05-08T05:36:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.245+0000 7f52b28d3640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ac06b190 0x7f52ac19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.245+0000 7f52b28d3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52ac06bb50 0x7f52ac1a01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.245+0000 7f52b28d3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac1079e0 0x7f52ac102980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.245+0000 7f52b28d3640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac112c20 con 0x7f52ac1079e0 2024-05-08T05:36:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.245+0000 7f52b28d3640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac112aa0 con 0x7f52ac06b190 2024-05-08T05:36:55.247 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.246+0000 7f52b28d3640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f52ac112da0 con 0x7f52ac06bb50 2024-05-08T05:36:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.246+0000 7f52abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ac06b190 0x7f52ac19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.246+0000 7f52abfff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ac06b190 0x7f52ac19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:38972/0 (socket says 172.21.15.59:38972) 2024-05-08T05:36:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.246+0000 7f52abfff640 1 -- 172.21.15.59:0/1549154968 learned_addr learned my addr 172.21.15.59:0/1549154968 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.246+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52ac06bb50 0x7f52ac1a01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.247+0000 7f52abfff640 1 -- 172.21.15.59:0/1549154968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52ac06bb50 msgr2=0x7f52ac1a01e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.247+0000 7f52abfff640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52ac06bb50 0x7f52ac1a01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.247+0000 7f52abfff640 1 -- 172.21.15.59:0/1549154968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac1079e0 msgr2=0x7f52ac102980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:55.248 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.247+0000 7f52abfff640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac1079e0 0x7f52ac102980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.247+0000 7f52abfff640 1 -- 172.21.15.59:0/1549154968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52ac103240 con 0x7f52ac06b190 2024-05-08T05:36:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.248+0000 7f52abfff640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ac06b190 0x7f52ac19fca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f529c006010 tx=0x7f529c0040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.248+0000 7f52a97fa640 1 -- 172.21.15.59:0/1549154968 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c03f030 con 0x7f52ac06b190 2024-05-08T05:36:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.248+0000 7f52b28d3640 1 -- 172.21.15.59:0/1549154968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52ac1034d0 con 0x7f52ac06b190 2024-05-08T05:36:55.249 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.248+0000 7f52b28d3640 1 -- 172.21.15.59:0/1549154968 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52ac19b8c0 con 0x7f52ac06b190 2024-05-08T05:36:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.248+0000 7f52a97fa640 1 -- 172.21.15.59:0/1549154968 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f529c0044c0 con 0x7f52ac06b190 2024-05-08T05:36:55.250 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.248+0000 7f52a97fa640 1 -- 172.21.15.59:0/1549154968 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f529c035070 con 0x7f52ac06b190 2024-05-08T05:36:55.254 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.252+0000 7f52b55f2740 1 Processor -- start 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.252+0000 7f52b55f2740 1 -- start start 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2572f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d261bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d2620f0 0x555f8d2644e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b55f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f8d1bc5d0 con 0x555f8d2620f0 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b55f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f8d175890 con 0x555f8d257830 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b55f2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555f8d1b8590 con 0x555f8d256f10 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52a97fa640 1 -- 172.21.15.59:0/1549154968 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f529c042030 con 0x7f52ac06b190 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b0e70640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2572f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b0e70640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2572f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:57952/0 (socket says 172.21.15.59:57952) 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b0e70640 1 -- 172.21.15.59:0/547794622 learned_addr learned my addr 172.21.15.59:0/547794622 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d2620f0 0x555f8d2644e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.255 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52abfff640 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d261bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b0e70640 1 -- 172.21.15.59:0/547794622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 msgr2=0x555f8d261bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b0e70640 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d261bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b0e70640 1 -- 172.21.15.59:0/547794622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d2620f0 msgr2=0x555f8d2644e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b0e70640 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d2620f0 0x555f8d2644e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52b0e70640 1 -- 172.21.15.59:0/547794622 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f8cece210 con 0x555f8d256f10 2024-05-08T05:36:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.253+0000 7f52abfff640 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d261bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:36:55.256 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.254+0000 7f52b0e70640 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2572f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009e00 tx=0x7f52a400e720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=35fbedc5057f449e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.254+0000 7f52927fc640 1 -- 172.21.15.59:0/547794622 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a4004d20 con 0x555f8d256f10 2024-05-08T05:36:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.254+0000 7f52927fc640 1 -- 172.21.15.59:0/547794622 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a400ee70 con 0x555f8d256f10 2024-05-08T05:36:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.254+0000 7f52927fc640 1 -- 172.21.15.59:0/547794622 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40047a0 con 0x555f8d256f10 2024-05-08T05:36:55.259 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.254+0000 7f52a97fa640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f52940b2a90 0x7f52940b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.254+0000 7f52a97fa640 1 -- 172.21.15.59:0/1549154968 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f529c1028e0 con 0x7f52ac06b190 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.254+0000 7f52abfff640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f52940b2a90 0x7f52940b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 -- 172.21.15.59:0/547794622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 msgr2=0x555f8d2572f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2572f0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f52a4009e00 tx=0x7f52a400e720 comp rx=0 tx=0).stop 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 -- 172.21.15.59:0/547794622 shutdown_connections 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d2620f0 0x555f8d2644e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d261bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 --2- 172.21.15.59:0/547794622 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2572f0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 -- 172.21.15.59:0/547794622 >> 172.21.15.59:0/547794622 conn(0x555f8d253420 msgr2=0x555f8d253800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:55.260 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 -- 172.21.15.59:0/547794622 shutdown_connections 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 -- 172.21.15.59:0/547794622 wait complete. 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52abfff640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f52940b2a90 0x7f52940b4f50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x555f8d2584b0 tx=0x7f529c02a410 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 -- start start 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.255+0000 7f52b55f2740 1 Processor -- start 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d25ef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d257830 0x555f8d25f4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d2620f0 0x555f8d25f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52b55f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x555f8d264d20 con 0x555f8d256f10 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52b0e70640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d25ef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52ab7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d257830 0x555f8d25f4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52ab7fe640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d257830 0x555f8d25f4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:57968/0 (socket says 172.21.15.59:57968) 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52ab7fe640 1 -- 172.21.15.59:0/734022352 learned_addr learned my addr 172.21.15.59:0/734022352 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x555f8d179370 con 0x555f8d2620f0 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x555f8d264ea0 con 0x555f8d257830 2024-05-08T05:36:55.261 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52b55f2740 10 client.0 _ll_register_callbacks cb 0x555f8cecda00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52ab7fe640 1 -- 172.21.15.59:0/734022352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d2620f0 msgr2=0x555f8d25f9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.256+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d2620f0 0x555f8d25f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f52ab7fe640 1 -- 172.21.15.59:0/734022352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 msgr2=0x555f8d25ef70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d25ef70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f52ab7fe640 1 -- 172.21.15.59:0/734022352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f8cf9a620 con 0x555f8d257830 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d257830 0x555f8d25f4b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52a00134e0 tx=0x7f52a00139a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5d8319ed1e2baf6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f528b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f528b7fe640 1 -- 172.21.15.59:0/734022352 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0004020 con 0x555f8d257830 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f528b7fe640 1 -- 172.21.15.59:0/734022352 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f52a00041c0 con 0x555f8d257830 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a400ee70 con 0x555f8d257830 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f528b7fe640 1 -- 172.21.15.59:0/734022352 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f52a0025050 con 0x555f8d257830 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f52b55f2740 10 client.25885 fetch_fsmap learned FSMap version 38 2024-05-08T05:36:55.262 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.257+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555f8d270c10 con 0x555f8d257830 2024-05-08T05:36:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.258+0000 7f528b7fe640 1 -- 172.21.15.59:0/734022352 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a400ee70 con 0x555f8d257830 2024-05-08T05:36:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.258+0000 7f528b7fe640 1 -- 172.21.15.59:0/734022352 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f52a000f040 con 0x555f8d257830 2024-05-08T05:36:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.258+0000 7f52b55f2740 10 client.25885 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:36:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.258+0000 7f52b55f2740 10 client.25885 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.258+0000 7f52b55f2740 20 client.25885 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.258+0000 7f52b55f2740 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555f8d272140 0x555f8d274590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.258+0000 7f52b55f2740 4 client.25885 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:36:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.258+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f8d1ca1e0 con 0x555f8d272140 2024-05-08T05:36:55.263 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.258+0000 7f52b0e70640 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555f8d272140 0x555f8d274590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.259+0000 7f528b7fe640 10 client.25885 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.259+0000 7f52b0e70640 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555f8d272140 0x555f8d274590 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4615bd216211dd06 server_cookie=2fa3ddf39ff2770d in_seq=0 out_seq=0 2024-05-08T05:36:55.264 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.259+0000 7f528b7fe640 1 -- 172.21.15.59:0/734022352 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x555f8d1ca1e0 con 0x555f8d272140 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.259+0000 7f528b7fe640 10 client.25885 handle_command_reply: tid=0 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.259+0000 7f52b55f2740 1 client.25885 shutdown 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d257830 msgr2=0x555f8d25f4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d257830 0x555f8d25f4b0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f52a00134e0 tx=0x7f52a00139a0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 shutdown_connections 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555f8d272140 0x555f8d274590 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d2620f0 0x555f8d25f9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d257830 0x555f8d25f4b0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f528b7fe640 0 client.25885 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 --2- 172.21.15.59:0/734022352 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d25ef70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 >> 172.21.15.59:0/734022352 conn(0x555f8d253420 msgr2=0x555f8d259150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 shutdown_connections 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 1 -- 172.21.15.59:0/734022352 wait complete. 2024-05-08T05:36:55.265 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.260+0000 7f52b55f2740 20 client.25885 trim_cache size 0 max 16384 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52b55f2740 1 Processor -- start 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52b55f2740 1 -- start start 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2598d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d257830 0x555f8d279ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d2620f0 0x555f8d27a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52b55f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f529800fe80 con 0x555f8d257830 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52b55f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f52980119c0 con 0x555f8d2620f0 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52b55f2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5298010550 con 0x555f8d256f10 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d257830 0x555f8d279ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52ab7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d257830 0x555f8d279ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45626/0 (socket says 172.21.15.59:45626) 2024-05-08T05:36:55.283 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.280+0000 7f52ab7fe640 1 -- 172.21.15.59:0/580047034 learned_addr learned my addr 172.21.15.59:0/580047034 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f52b0e70640 1 --2- 172.21.15.59:0/580047034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2598d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f52b0e70640 1 --2- 172.21.15.59:0/580047034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d2620f0 0x555f8d27a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f52ab7fe640 1 -- 172.21.15.59:0/580047034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 msgr2=0x555f8d2598d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/580047034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2598d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f52ab7fe640 1 -- 172.21.15.59:0/580047034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d2620f0 msgr2=0x555f8d27a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/580047034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d2620f0 0x555f8d27a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f52ab7fe640 1 -- 172.21.15.59:0/580047034 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f8d264850 con 0x555f8d257830 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/580047034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d257830 0x555f8d279ef0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f52a0005580 tx=0x7f52a0009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a29c530affbd220 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f5291ffb640 1 -- 172.21.15.59:0/580047034 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0011ab0 con 0x555f8d257830 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f5291ffb640 1 -- 172.21.15.59:0/580047034 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a0005600 con 0x555f8d257830 2024-05-08T05:36:55.284 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.281+0000 7f5291ffb640 1 -- 172.21.15.59:0/580047034 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a0029d70 con 0x555f8d257830 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.282+0000 7f52b55f2740 1 -- 172.21.15.59:0/580047034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d257830 msgr2=0x555f8d279ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.282+0000 7f52b55f2740 1 --2- 172.21.15.59:0/580047034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d257830 0x555f8d279ef0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f52a0005580 tx=0x7f52a0009b70 comp rx=0 tx=0).stop 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.282+0000 7f52b55f2740 1 -- 172.21.15.59:0/580047034 shutdown_connections 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.282+0000 7f52b55f2740 1 --2- 172.21.15.59:0/580047034 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d2620f0 0x555f8d27a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.282+0000 7f52b55f2740 1 --2- 172.21.15.59:0/580047034 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d257830 0x555f8d279ef0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.282+0000 7f52b55f2740 1 --2- 172.21.15.59:0/580047034 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d256f10 0x555f8d2598d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.282+0000 7f52b55f2740 1 -- 172.21.15.59:0/580047034 >> 172.21.15.59:0/580047034 conn(0x555f8d253420 msgr2=0x555f8d252580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 -- 172.21.15.59:0/580047034 shutdown_connections 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 -- 172.21.15.59:0/580047034 wait complete. 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 -- start start 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 Processor -- start 2024-05-08T05:36:55.288 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d269f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d26a480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d2620f0 0x555f8d266410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f52980112f0 con 0x555f8d256f10 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5298010c20 con 0x555f8d257830 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f529800de40 con 0x555f8d2620f0 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.283+0000 7f52b55f2740 10 client.0 _ll_register_callbacks cb 0x7f52a0009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d269f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d26a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d269f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45642/0 (socket says 172.21.15.59:45642) 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52ab7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d26a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:39008/0 (socket says 172.21.15.59:39008) 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 -- 172.21.15.59:0/1079395618 learned_addr learned my addr 172.21.15.59:0/1079395618 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d2620f0 0x555f8d266410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.289 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 -- 172.21.15.59:0/1079395618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d2620f0 msgr2=0x555f8d266410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d2620f0 0x555f8d266410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 -- 172.21.15.59:0/1079395618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 msgr2=0x555f8d26a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d26a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 -- 172.21.15.59:0/1079395618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f8d266b50 con 0x555f8d256f10 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d26a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b0e70640 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d269f40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f52a400f010 tx=0x7f52a0009be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8923dbb3fdccf63f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f528b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f528b7fe640 1 -- 172.21.15.59:0/1079395618 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52a40047a0 con 0x555f8d256f10 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f528b7fe640 1 -- 172.21.15.59:0/1079395618 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f52a4027430 con 0x555f8d256f10 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.284+0000 7f52b55f2740 1 -- 172.21.15.59:0/1079395618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f52a0005600 con 0x555f8d256f10 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f528b7fe640 1 -- 172.21.15.59:0/1079395618 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f52a403a050 con 0x555f8d256f10 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f52b55f2740 10 client.16380 fetch_fsmap learned FSMap version 38 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f52b55f2740 1 -- 172.21.15.59:0/1079395618 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x555f8d2dd4c0 con 0x555f8d256f10 2024-05-08T05:36:55.290 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f528b7fe640 1 -- 172.21.15.59:0/1079395618 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52a0005600 con 0x555f8d256f10 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f528b7fe640 1 -- 172.21.15.59:0/1079395618 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f52a4003d50 con 0x555f8d256f10 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f52b55f2740 10 client.16380 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f52b55f2740 10 client.16380 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f52b55f2740 20 client.16380 populate_metadata read hostname 'smithi059' 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f52b55f2740 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555f8d272140 0x555f8d2dfa50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f52b55f2740 4 client.16380 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f52b55f2740 1 -- 172.21.15.59:0/1079395618 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f8d1c4050 con 0x555f8d272140 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.285+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555f8d272140 0x555f8d2dfa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.286+0000 7f528b7fe640 10 client.16380 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:55.291 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.286+0000 7f52ab7fe640 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555f8d272140 0x555f8d2dfa50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6968d5e71dc813a server_cookie=78ce2d50eccb5093 in_seq=0 out_seq=0 2024-05-08T05:36:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f528b7fe640 1 -- 172.21.15.59:0/1079395618 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f8d1c4050 con 0x555f8d272140 2024-05-08T05:36:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f528b7fe640 10 client.16380 handle_command_reply: tid=0 2024-05-08T05:36:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 client.16380 shutdown 2024-05-08T05:36:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 -- 172.21.15.59:0/1079395618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 msgr2=0x555f8d269f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d269f40 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f52a400f010 tx=0x7f52a0009be0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 -- 172.21.15.59:0/1079395618 shutdown_connections 2024-05-08T05:36:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x555f8d272140 0x555f8d2dfa50 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x555f8d2620f0 0x555f8d266410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.302 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x555f8d257830 0x555f8d26a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f528b7fe640 0 client.16380 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 --2- 172.21.15.59:0/1079395618 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x555f8d256f10 0x555f8d269f40 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 -- 172.21.15.59:0/1079395618 >> 172.21.15.59:0/1079395618 conn(0x555f8d253420 msgr2=0x555f8d251cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 -- 172.21.15.59:0/1079395618 shutdown_connections 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.300+0000 7f52b55f2740 1 -- 172.21.15.59:0/1079395618 wait complete. 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b55f2740 20 client.16380 trim_cache size 0 max 16384 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:36:55.303 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:36:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 -- 172.21.15.59:0/1549154968 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f52940b2a90 msgr2=0x7f52940b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f52940b2a90 0x7f52940b4f50 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x555f8d2584b0 tx=0x7f529c02a410 comp rx=0 tx=0).stop 2024-05-08T05:36:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 -- 172.21.15.59:0/1549154968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ac06b190 msgr2=0x7f52ac19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:36:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ac06b190 0x7f52ac19fca0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f529c006010 tx=0x7f529c0040c0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.304 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 -- 172.21.15.59:0/1549154968 shutdown_connections 2024-05-08T05:36:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f52940b2a90 0x7f52940b4f50 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f52ac1079e0 0x7f52ac102980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f52ac06bb50 0x7f52ac1a01e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 --2- 172.21.15.59:0/1549154968 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f52ac06b190 0x7f52ac19fca0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:36:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 -- 172.21.15.59:0/1549154968 >> 172.21.15.59:0/1549154968 conn(0x7f52ac078690 msgr2=0x7f52ac0feb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:36:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 -- 172.21.15.59:0/1549154968 shutdown_connections 2024-05-08T05:36:55.305 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:36:55.301+0000 7f52b28d3640 1 -- 172.21.15.59:0/1549154968 wait complete. 2024-05-08T05:36:55.339 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:55 smithi059 ceph-mon[30304]: pgmap v226: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.7 MiB/s wr, 430 op/s 2024-05-08T05:36:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:55 smithi143 ceph-mon[34337]: pgmap v226: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.7 MiB/s wr, 430 op/s 2024-05-08T05:36:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:55 smithi096 ceph-mon[34253]: pgmap v226: 129 pgs: 129 active+clean; 64 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 1.7 MiB/s wr, 430 op/s 2024-05-08T05:36:57.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:57 smithi059 ceph-mon[30304]: pgmap v227: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.2 MiB/s wr, 687 op/s 2024-05-08T05:36:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:57 smithi143 ceph-mon[34337]: pgmap v227: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.2 MiB/s wr, 687 op/s 2024-05-08T05:36:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:57 smithi096 ceph-mon[34253]: pgmap v227: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 2.2 MiB/s wr, 687 op/s 2024-05-08T05:36:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:36:59 smithi059 ceph-mon[30304]: pgmap v228: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.5 MiB/s wr, 561 op/s 2024-05-08T05:36:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:36:59 smithi143 ceph-mon[34337]: pgmap v228: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.5 MiB/s wr, 561 op/s 2024-05-08T05:36:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:36:59 smithi096 ceph-mon[34253]: pgmap v228: 129 pgs: 129 active+clean; 59 MiB data, 1.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 MiB/s rd, 1.5 MiB/s wr, 561 op/s 2024-05-08T05:37:01.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:01 smithi059 ceph-mon[30304]: pgmap v229: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 835 op/s 2024-05-08T05:37:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:01 smithi143 ceph-mon[34337]: pgmap v229: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 835 op/s 2024-05-08T05:37:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:01 smithi096 ceph-mon[34253]: pgmap v229: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 16 MiB/s rd, 2.4 MiB/s wr, 835 op/s 2024-05-08T05:37:01.897 INFO:tasks.workunit.client.0.smithi059.stdout:10000 mapread 0x33cc4e3 thru 0x33cec16 (0x2734 bytes) 2024-05-08T05:37:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:03 smithi059 ceph-mon[30304]: pgmap v230: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 569 op/s 2024-05-08T05:37:03.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:03 smithi059 ceph-mon[30304]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T05:37:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:03 smithi143 ceph-mon[34337]: pgmap v230: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 569 op/s 2024-05-08T05:37:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:03 smithi143 ceph-mon[34337]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T05:37:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:03 smithi096 ceph-mon[34253]: pgmap v230: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.6 MiB/s wr, 569 op/s 2024-05-08T05:37:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:03 smithi096 ceph-mon[34253]: osdmap e95: 12 total, 12 up, 12 in 2024-05-08T05:37:05.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:05 smithi059 ceph-mon[30304]: pgmap v232: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.9 MiB/s wr, 683 op/s 2024-05-08T05:37:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:05 smithi143 ceph-mon[34337]: pgmap v232: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.9 MiB/s wr, 683 op/s 2024-05-08T05:37:05.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:05 smithi096 ceph-mon[34253]: pgmap v232: 129 pgs: 129 active+clean; 56 MiB data, 1.7 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.9 MiB/s wr, 683 op/s 2024-05-08T05:37:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:06 smithi143 ceph-mon[34337]: pgmap v233: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.8 MiB/s wr, 573 op/s 2024-05-08T05:37:07.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:06 smithi096 ceph-mon[34253]: pgmap v233: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.8 MiB/s wr, 573 op/s 2024-05-08T05:37:07.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:06 smithi059 ceph-mon[30304]: pgmap v233: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.8 MiB/s wr, 573 op/s 2024-05-08T05:37:08.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.437+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/3299652454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41016c0 msgr2=0x7fa1a4101b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.437+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/3299652454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41016c0 0x7fa1a4101b40 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7fa198009890 tx=0x7fa19802a380 comp rx=0 tx=0).stop 2024-05-08T05:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.438+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/3299652454 shutdown_connections 2024-05-08T05:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.438+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/3299652454 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a4102080 0x7fa1a410e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.438+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/3299652454 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a41016c0 0x7fa1a4101b40 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.438+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/3299652454 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41076d0 0x7fa1a4107ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.438+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/3299652454 >> 172.21.15.59:0/3299652454 conn(0x7fa1a40fd1d0 msgr2=0x7fa1a40ff5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.438+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/3299652454 shutdown_connections 2024-05-08T05:37:08.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.438+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/3299652454 wait complete. 2024-05-08T05:37:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a9ac2640 1 Processor -- start 2024-05-08T05:37:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a9ac2640 1 -- start start 2024-05-08T05:37:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a9ac2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41016c0 0x7fa1a4196e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a9ac2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4102080 0x7fa1a41973d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a9ac2640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41076d0 0x7fa1a419b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a9ac2640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41108d0 con 0x7fa1a4102080 2024-05-08T05:37:08.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a9ac2640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4110750 con 0x7fa1a41076d0 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a9ac2640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a4110a50 con 0x7fa1a41016c0 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4102080 0x7fa1a41973d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4102080 0x7fa1a41973d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37616/0 (socket says 172.21.15.59:37616) 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a92c1640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41076d0 0x7fa1a419b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a3fff640 1 -- 172.21.15.59:0/2940948994 learned_addr learned my addr 172.21.15.59:0/2940948994 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.439+0000 7fa1a8ac0640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41016c0 0x7fa1a4196e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a3fff640 1 -- 172.21.15.59:0/2940948994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41016c0 msgr2=0x7fa1a4196e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41016c0 0x7fa1a4196e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a3fff640 1 -- 172.21.15.59:0/2940948994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41076d0 msgr2=0x7fa1a419b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41076d0 0x7fa1a419b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a3fff640 1 -- 172.21.15.59:0/2940948994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1a419be80 con 0x7fa1a4102080 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4102080 0x7fa1a41973d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa19802a8f0 tx=0x7fa198032d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a1ffb640 1 -- 172.21.15.59:0/2940948994 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198042070 con 0x7fa1a4102080 2024-05-08T05:37:08.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a1ffb640 1 -- 172.21.15.59:0/2940948994 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa198030d50 con 0x7fa1a4102080 2024-05-08T05:37:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/2940948994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1a419c110 con 0x7fa1a4102080 2024-05-08T05:37:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a1ffb640 1 -- 172.21.15.59:0/2940948994 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa198037040 con 0x7fa1a4102080 2024-05-08T05:37:08.443 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.440+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/2940948994 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1a41a3a00 con 0x7fa1a4102080 2024-05-08T05:37:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1a1ffb640 1 -- 172.21.15.59:0/2940948994 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa198045070 con 0x7fa1a4102080 2024-05-08T05:37:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1ada41740 1 Processor -- start 2024-05-08T05:37:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1ada41740 1 -- start start 2024-05-08T05:37:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779f0e0 0x564b7779f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779fa00 0x564b777a9d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b777aa2c0 0x564b777ac6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1ada41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564b777045d0 con 0x564b7779f0e0 2024-05-08T05:37:08.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1ada41740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564b776bd890 con 0x564b7779fa00 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1ada41740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564b77700590 con 0x564b777aa2c0 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.443+0000 7fa1a8ac0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779fa00 0x564b777a9d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b777aa2c0 0x564b777ac6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a8ac0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779fa00 0x564b777a9d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:43966/0 (socket says 172.21.15.59:43966) 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a8ac0640 1 -- 172.21.15.59:0/1164461580 learned_addr learned my addr 172.21.15.59:0/1164461580 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a1ffb640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa18c0b6db0 0x7fa18c0b9270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a92c1640 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779f0e0 0x564b7779f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a92c1640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa18c0b6db0 0x7fa18c0b9270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a1ffb640 1 -- 172.21.15.59:0/2940948994 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(95..95 src has 1..95) ==== 9584+0+0 (secure 0 0 0) 0x7fa198102100 con 0x7fa1a4102080 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a3fff640 1 -- 172.21.15.59:0/1164461580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779fa00 msgr2=0x564b777a9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779fa00 0x564b777a9d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a3fff640 1 -- 172.21.15.59:0/1164461580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779f0e0 msgr2=0x564b7779f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779f0e0 0x564b7779f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a3fff640 1 -- 172.21.15.59:0/1164461580 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b77416210 con 0x564b777aa2c0 2024-05-08T05:37:08.446 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a92c1640 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779f0e0 0x564b7779f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:37:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a8ac0640 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779fa00 0x564b777a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b777aa2c0 0x564b777ac6b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1980ff540 tx=0x7fa19802a380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc6fd708d36125cb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa1a92c1640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa18c0b6db0 0x7fa18c0b9270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa19c005c40 tx=0x7fa19c004ab0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa18affd640 1 -- 172.21.15.59:0/1164461580 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1980c8a20 con 0x564b777aa2c0 2024-05-08T05:37:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa18affd640 1 -- 172.21.15.59:0/1164461580 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1980c8be0 con 0x564b777aa2c0 2024-05-08T05:37:08.447 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.444+0000 7fa18affd640 1 -- 172.21.15.59:0/1164461580 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1980c8840 con 0x564b777aa2c0 2024-05-08T05:37:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.445+0000 7fa1ada41740 1 -- 172.21.15.59:0/1164461580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b777aa2c0 msgr2=0x564b777ac6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:08.451 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.445+0000 7fa1ada41740 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b777aa2c0 0x564b777ac6b0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7fa1980ff540 tx=0x7fa19802a380 comp rx=0 tx=0).stop 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.445+0000 7fa1ada41740 1 -- 172.21.15.59:0/1164461580 shutdown_connections 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.445+0000 7fa1ada41740 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b777aa2c0 0x564b777ac6b0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.445+0000 7fa1ada41740 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779fa00 0x564b777a9d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.445+0000 7fa1ada41740 1 --2- 172.21.15.59:0/1164461580 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779f0e0 0x564b7779f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.445+0000 7fa1ada41740 1 -- 172.21.15.59:0/1164461580 >> 172.21.15.59:0/1164461580 conn(0x564b7779b5f0 msgr2=0x564b7779b9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.445+0000 7fa1ada41740 1 -- 172.21.15.59:0/1164461580 shutdown_connections 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.445+0000 7fa1ada41740 1 -- 172.21.15.59:0/1164461580 wait complete. 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1ada41740 1 -- start start 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1ada41740 1 Processor -- start 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 0x564b777a7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.452 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1ada41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564b777accb0 con 0x564b7779fa00 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1ada41740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564b776c1370 con 0x564b777aa2c0 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1ada41740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564b777ace30 con 0x564b7779f0e0 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1ada41740 10 client.0 _ll_register_callbacks cb 0x564b77415a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1a8ac0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.446+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a8ac0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:49842/0 (socket says 172.21.15.59:49842) 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a8ac0640 1 -- 172.21.15.59:0/11934923 learned_addr learned my addr 172.21.15.59:0/11934923 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a92c1640 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 0x564b777a7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a8ac0640 1 -- 172.21.15.59:0/11934923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 msgr2=0x564b777a7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a8ac0640 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 0x564b777a7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a8ac0640 1 -- 172.21.15.59:0/11934923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 msgr2=0x564b777a7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a8ac0640 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a8ac0640 1 -- 172.21.15.59:0/11934923 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b774e2620 con 0x564b7779f0e0 2024-05-08T05:37:08.453 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1a8ac0640 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a72c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa194011730 tx=0x7fa194011bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d570ec69fba20011 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa183fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa183fff640 1 -- 172.21.15.59:0/11934923 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa19401f060 con 0x564b7779f0e0 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1ada41740 1 -- 172.21.15.59:0/11934923 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1980c8be0 con 0x564b7779f0e0 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa183fff640 1 -- 172.21.15.59:0/11934923 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9584+0+0 (secure 0 0 0) 0x7fa194004320 con 0x564b7779f0e0 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa183fff640 1 -- 172.21.15.59:0/11934923 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa194026050 con 0x564b7779f0e0 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1ada41740 10 client.25903 fetch_fsmap learned FSMap version 38 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.447+0000 7fa1ada41740 1 -- 172.21.15.59:0/11934923 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564b777b8c00 con 0x564b7779f0e0 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.448+0000 7fa183fff640 1 -- 172.21.15.59:0/11934923 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1980c8be0 con 0x564b7779f0e0 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.448+0000 7fa183fff640 1 -- 172.21.15.59:0/11934923 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fa194003c70 con 0x564b7779f0e0 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.448+0000 7fa1ada41740 10 client.25903 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.448+0000 7fa1ada41740 10 client.25903 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.448+0000 7fa1ada41740 20 client.25903 populate_metadata read hostname 'smithi059' 2024-05-08T05:37:08.454 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.448+0000 7fa1ada41740 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564b777ba070 0x564b777bc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.448+0000 7fa1ada41740 4 client.25903 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.448+0000 7fa1ada41740 1 -- 172.21.15.59:0/11934923 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564b777121e0 con 0x564b777ba070 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.448+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564b777ba070 0x564b777bc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564b777ba070 0x564b777bc4a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=739958a3e2b14a3a server_cookie=bf3101a7d80643de in_seq=0 out_seq=0 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa183fff640 10 client.25903 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa183fff640 1 -- 172.21.15.59:0/11934923 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564b777121e0 con 0x564b777ba070 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa183fff640 10 client.25903 handle_command_reply: tid=0 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa1ada41740 1 client.25903 shutdown 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa1ada41740 1 -- 172.21.15.59:0/11934923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 msgr2=0x564b777a72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa1ada41740 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a72c0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fa194011730 tx=0x7fa194011bf0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa1ada41740 1 -- 172.21.15.59:0/11934923 shutdown_connections 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa1ada41740 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564b777ba070 0x564b777bc4a0 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.449+0000 7fa1ada41740 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 0x564b777a7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.450+0000 7fa1ada41740 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.455 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.450+0000 7fa1ada41740 1 --2- 172.21.15.59:0/11934923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a72c0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.450+0000 7fa183fff640 0 client.25903 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.450+0000 7fa1ada41740 1 -- 172.21.15.59:0/11934923 >> 172.21.15.59:0/11934923 conn(0x564b7779b5f0 msgr2=0x564b777a1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.450+0000 7fa1ada41740 1 -- 172.21.15.59:0/11934923 shutdown_connections 2024-05-08T05:37:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.450+0000 7fa1ada41740 1 -- 172.21.15.59:0/11934923 wait complete. 2024-05-08T05:37:08.456 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.450+0000 7fa1ada41740 20 client.25903 trim_cache size 0 max 16384 2024-05-08T05:37:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1ada41740 1 Processor -- start 2024-05-08T05:37:08.467 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1ada41740 1 -- start start 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a1730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 0x564b777a7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1ada41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41adbf0 con 0x564b7779fa00 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1ada41740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41af730 con 0x564b777aa2c0 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1ada41740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41ae2c0 con 0x564b7779f0e0 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1a8ac0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1a8ac0640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a11f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:49848/0 (socket says 172.21.15.59:49848) 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1a8ac0640 1 -- 172.21.15.59:0/3828393605 learned_addr learned my addr 172.21.15.59:0/3828393605 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.466+0000 7fa1a92c1640 1 --2- 172.21.15.59:0/3828393605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 0x564b777a7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa1a3fff640 1 -- 172.21.15.59:0/3828393605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 msgr2=0x564b777a11f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/3828393605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a11f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.468 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa1a3fff640 1 -- 172.21.15.59:0/3828393605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 msgr2=0x564b777a7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/3828393605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 0x564b777a7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa1a3fff640 1 -- 172.21.15.59:0/3828393605 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564b7779bf70 con 0x564b7779fa00 2024-05-08T05:37:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa1a8ac0640 1 --2- 172.21.15.59:0/3828393605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/3828393605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a1730 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa1980ff540 tx=0x7fa1980cd480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=881c73dde789c4d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa18a7fc640 1 -- 172.21.15.59:0/3828393605 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1980c8a20 con 0x564b7779fa00 2024-05-08T05:37:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa18a7fc640 1 -- 172.21.15.59:0/3828393605 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1980c8be0 con 0x564b7779fa00 2024-05-08T05:37:08.469 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.467+0000 7fa18a7fc640 1 -- 172.21.15.59:0/3828393605 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1980d1f30 con 0x564b7779fa00 2024-05-08T05:37:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 -- 172.21.15.59:0/3828393605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 msgr2=0x564b777a1730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3828393605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a1730 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7fa1980ff540 tx=0x7fa1980cd480 comp rx=0 tx=0).stop 2024-05-08T05:37:08.472 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 -- 172.21.15.59:0/3828393605 shutdown_connections 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3828393605 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b777aa2c0 0x564b777a7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3828393605 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b7779fa00 0x564b777a1730 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3828393605 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779f0e0 0x564b777a11f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 -- 172.21.15.59:0/3828393605 >> 172.21.15.59:0/3828393605 conn(0x564b7779b5f0 msgr2=0x564b777a0ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 -- 172.21.15.59:0/3828393605 shutdown_connections 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 -- 172.21.15.59:0/3828393605 wait complete. 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 -- start start 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.468+0000 7fa1ada41740 1 Processor -- start 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779f0e0 0x564b777ade50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779fa00 0x564b777ae390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1ada41740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b777aa2c0 0x564b777ae8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1ada41740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41af060 con 0x564b777aa2c0 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1ada41740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41ae990 con 0x564b7779f0e0 2024-05-08T05:37:08.473 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1ada41740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa1a41abbd0 con 0x564b7779fa00 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1ada41740 10 client.0 _ll_register_callbacks cb 0x7fa194009950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a92c1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b777aa2c0 0x564b777ae8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a8ac0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779f0e0 0x564b777ade50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779fa00 0x564b777ae390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a92c1640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b777aa2c0 0x564b777ae8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37662/0 (socket says 172.21.15.59:37662) 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a8ac0640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779f0e0 0x564b777ade50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:43978/0 (socket says 172.21.15.59:43978) 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a92c1640 1 -- 172.21.15.59:0/3950031064 learned_addr learned my addr 172.21.15.59:0/3950031064 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a3fff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779fa00 0x564b777ae390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:49860/0 (socket says 172.21.15.59:49860) 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a3fff640 1 -- 172.21.15.59:0/3950031064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779f0e0 msgr2=0x564b777ade50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779f0e0 0x564b777ade50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a3fff640 1 -- 172.21.15.59:0/3950031064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b777aa2c0 msgr2=0x564b777ae8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b777aa2c0 0x564b777ae8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a3fff640 1 -- 172.21.15.59:0/3950031064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564b77823c00 con 0x564b7779fa00 2024-05-08T05:37:08.474 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a92c1640 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b777aa2c0 0x564b777ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa1a3fff640 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779fa00 0x564b777ae390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x564b777bc740 tx=0x7fa1980fec90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c62ba05d9bc8f28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa183fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.469+0000 7fa183fff640 1 -- 172.21.15.59:0/3950031064 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1980c8a20 con 0x564b7779fa00 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa183fff640 1 -- 172.21.15.59:0/3950031064 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(95..95 src has 1..95) ==== 9584+0+0 (secure 0 0 0) 0x7fa1980fed80 con 0x564b7779fa00 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1ada41740 1 -- 172.21.15.59:0/3950031064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa1980c8be0 con 0x564b7779fa00 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa183fff640 1 -- 172.21.15.59:0/3950031064 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fa1980c90d0 con 0x564b7779fa00 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1ada41740 10 client.25909 fetch_fsmap learned FSMap version 38 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1ada41740 1 -- 172.21.15.59:0/3950031064 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564b77824f10 con 0x564b7779fa00 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa183fff640 1 -- 172.21.15.59:0/3950031064 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1980c8be0 con 0x564b7779fa00 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa183fff640 1 -- 172.21.15.59:0/3950031064 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fa198030450 con 0x564b7779fa00 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1ada41740 10 client.25909 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1ada41740 10 client.25909 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1ada41740 20 client.25909 populate_metadata read hostname 'smithi059' 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564b778252b0 0x564b778276a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:08.475 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1ada41740 4 client.25909 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:37:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1ada41740 1 -- 172.21.15.59:0/3950031064 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564b7770c050 con 0x564b778252b0 2024-05-08T05:37:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.470+0000 7fa1a8ac0640 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564b778252b0 0x564b778276a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.471+0000 7fa1a8ac0640 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564b778252b0 0x564b778276a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f610d172bc0e32c8 server_cookie=4015164a508197b8 in_seq=0 out_seq=0 2024-05-08T05:37:08.476 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.471+0000 7fa183fff640 10 client.25909 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:08.487 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa183fff640 1 -- 172.21.15.59:0/3950031064 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564b7770c050 con 0x564b778252b0 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa183fff640 10 client.25909 handle_command_reply: tid=0 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 client.25909 shutdown 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 -- 172.21.15.59:0/3950031064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779fa00 msgr2=0x564b777ae390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779fa00 0x564b777ae390 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x564b777bc740 tx=0x7fa1980fec90 comp rx=0 tx=0).stop 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 -- 172.21.15.59:0/3950031064 shutdown_connections 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564b778252b0 0x564b778276a0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564b777aa2c0 0x564b777ae8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564b7779fa00 0x564b777ae390 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 --2- 172.21.15.59:0/3950031064 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564b7779f0e0 0x564b777ade50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa183fff640 0 client.25909 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 -- 172.21.15.59:0/3950031064 >> 172.21.15.59:0/3950031064 conn(0x564b7779b5f0 msgr2=0x564b777a1370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 -- 172.21.15.59:0/3950031064 shutdown_connections 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 1 -- 172.21.15.59:0/3950031064 wait complete. 2024-05-08T05:37:08.488 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.487+0000 7fa1ada41740 20 client.25909 trim_cache size 0 max 16384 2024-05-08T05:37:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:37:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:37:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:37:08.489 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:37:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa18c0b6db0 msgr2=0x7fa18c0b9270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa18c0b6db0 0x7fa18c0b9270 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa19c005c40 tx=0x7fa19c004ab0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4102080 msgr2=0x7fa1a41973d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4102080 0x7fa1a41973d0 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7fa19802a8f0 tx=0x7fa198032d40 comp rx=0 tx=0).stop 2024-05-08T05:37:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/2940948994 shutdown_connections 2024-05-08T05:37:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa18c0b6db0 0x7fa18c0b9270 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa1a41076d0 0x7fa1a419b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.490 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa1a4102080 0x7fa1a41973d0 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 --2- 172.21.15.59:0/2940948994 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa1a41016c0 0x7fa1a4196e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/2940948994 >> 172.21.15.59:0/2940948994 conn(0x7fa1a40fd1d0 msgr2=0x7fa1a40fdb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/2940948994 shutdown_connections 2024-05-08T05:37:08.491 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:08.488+0000 7fa1a9ac2640 1 -- 172.21.15.59:0/2940948994 wait complete. 2024-05-08T05:37:09.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:09 smithi096 ceph-mon[34253]: pgmap v234: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.8 MiB/s wr, 573 op/s 2024-05-08T05:37:09.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:09 smithi059 ceph-mon[30304]: pgmap v234: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.8 MiB/s wr, 573 op/s 2024-05-08T05:37:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:09 smithi143 ceph-mon[34337]: pgmap v234: 129 pgs: 1 active+clean+snaptrim, 128 active+clean; 72 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.8 MiB/s wr, 573 op/s 2024-05-08T05:37:11.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:11 smithi096 ceph-mon[34253]: pgmap v235: 129 pgs: 129 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 576 op/s 2024-05-08T05:37:11.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:11 smithi096 ceph-mon[34253]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T05:37:11.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:11 smithi059 ceph-mon[30304]: pgmap v235: 129 pgs: 129 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 576 op/s 2024-05-08T05:37:11.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:11 smithi059 ceph-mon[30304]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T05:37:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:11 smithi143 ceph-mon[34337]: pgmap v235: 129 pgs: 129 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 576 op/s 2024-05-08T05:37:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:11 smithi143 ceph-mon[34337]: osdmap e96: 12 total, 12 up, 12 in 2024-05-08T05:37:13.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:13 smithi096 ceph-mon[34253]: pgmap v237: 129 pgs: 129 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 583 op/s 2024-05-08T05:37:13.338 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:13 smithi059 ceph-mon[30304]: pgmap v237: 129 pgs: 129 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 583 op/s 2024-05-08T05:37:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:13 smithi143 ceph-mon[34337]: pgmap v237: 129 pgs: 129 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 583 op/s 2024-05-08T05:37:14.027 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0x63e5d2a 2024-05-08T05:37:15.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:15 smithi096 ceph-mon[34253]: pgmap v238: 129 pgs: 129 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 576 op/s 2024-05-08T05:37:15.294 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:15 smithi143 ceph-mon[34337]: pgmap v238: 129 pgs: 129 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 576 op/s 2024-05-08T05:37:15.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:15 smithi059 ceph-mon[30304]: pgmap v238: 129 pgs: 129 active+clean; 75 MiB data, 1.8 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.8 MiB/s wr, 576 op/s 2024-05-08T05:37:17.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:17 smithi096 ceph-mon[34253]: pgmap v239: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 653 op/s 2024-05-08T05:37:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:17 smithi059 ceph-mon[30304]: pgmap v239: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 653 op/s 2024-05-08T05:37:17.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:17 smithi143 ceph-mon[34337]: pgmap v239: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 653 op/s 2024-05-08T05:37:18.282 INFO:tasks.workunit.client.0.smithi059.stdout:truncating to largest ever: 0x63e9b52 2024-05-08T05:37:19.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:19 smithi096 ceph-mon[34253]: pgmap v240: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 653 op/s 2024-05-08T05:37:19.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:19 smithi059 ceph-mon[30304]: pgmap v240: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 653 op/s 2024-05-08T05:37:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:19 smithi143 ceph-mon[34337]: pgmap v240: 129 pgs: 129 active+clean; 70 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.0 MiB/s wr, 653 op/s 2024-05-08T05:37:21.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:21 smithi096 ceph-mon[34253]: pgmap v241: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.9 MiB/s wr, 619 op/s 2024-05-08T05:37:21.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:21 smithi059 ceph-mon[30304]: pgmap v241: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.9 MiB/s wr, 619 op/s 2024-05-08T05:37:21.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:21 smithi143 ceph-mon[34337]: pgmap v241: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 1.9 MiB/s wr, 619 op/s 2024-05-08T05:37:21.605 INFO:tasks.workunit.client.0.smithi059.stdout:20000 read 0xa6cbf1 thru 0xa6e358 (0x1768 bytes) 2024-05-08T05:37:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.606+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3599705325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c10cf80 msgr2=0x7f5a6c10f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:21.607 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.606+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3599705325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c10cf80 0x7f5a6c10f420 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009ca0 tx=0x7f5a6402a9f0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.607+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3599705325 shutdown_connections 2024-05-08T05:37:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.607+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3599705325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c10cf80 0x7f5a6c10f420 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.607+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3599705325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a6c104560 0x7f5a6c10ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.607+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3599705325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c103c40 0x7f5a6c104020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.608 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.607+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3599705325 >> 172.21.15.59:0/3599705325 conn(0x7f5a6c078a30 msgr2=0x7f5a6c078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:21.611 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.611+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3599705325 shutdown_connections 2024-05-08T05:37:21.612 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.611+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3599705325 wait complete. 2024-05-08T05:37:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.613+0000 7f5a70d9d640 1 Processor -- start 2024-05-08T05:37:21.613 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.613+0000 7f5a70d9d640 1 -- start start 2024-05-08T05:37:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a70d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c103c40 0x7f5a6c196de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a70d9d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c104560 0x7f5a6c197320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a70d9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a6c10cf80 0x7f5a6c19b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.614 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a70d9d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c110ae0 con 0x7f5a6c103c40 2024-05-08T05:37:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a69d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c104560 0x7f5a6c197320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a69d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c104560 0x7f5a6c197320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:58426/0 (socket says 172.21.15.59:58426) 2024-05-08T05:37:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a69d9b640 1 -- 172.21.15.59:0/3943935187 learned_addr learned my addr 172.21.15.59:0/3943935187 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a6c10cf80 0x7f5a6c19b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c110960 con 0x7f5a6c104560 2024-05-08T05:37:21.615 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.614+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c110c60 con 0x7f5a6c10cf80 2024-05-08T05:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.615+0000 7f5a69d9b640 1 -- 172.21.15.59:0/3943935187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a6c10cf80 msgr2=0x7f5a6c19b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.615+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a6c10cf80 0x7f5a6c19b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.615+0000 7f5a69d9b640 1 -- 172.21.15.59:0/3943935187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c103c40 msgr2=0x7f5a6c196de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-05-08T05:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.615+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c103c40 0x7f5a6c196de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.615+0000 7f5a69d9b640 1 -- 172.21.15.59:0/3943935187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a6c19bdd0 con 0x7f5a6c104560 2024-05-08T05:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.615+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c104560 0x7f5a6c197320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000d950 tx=0x7f5a6000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.615+0000 7f5a5b7fe640 1 -- 172.21.15.59:0/3943935187 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60017070 con 0x7f5a6c104560 2024-05-08T05:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.616+0000 7f5a5b7fe640 1 -- 172.21.15.59:0/3943935187 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a60009c30 con 0x7f5a6c104560 2024-05-08T05:37:21.617 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.616+0000 7f5a5b7fe640 1 -- 172.21.15.59:0/3943935187 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a6000f620 con 0x7f5a6c104560 2024-05-08T05:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.616+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a6c19c0c0 con 0x7f5a6c104560 2024-05-08T05:37:21.618 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.616+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a6c1a3950 con 0x7f5a6c104560 2024-05-08T05:37:21.619 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.619+0000 7f5a5b7fe640 1 -- 172.21.15.59:0/3943935187 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f5a60010460 con 0x7f5a6c104560 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.619+0000 7f5a73b92740 1 Processor -- start 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a73b92740 1 -- start start 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9512f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab95bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab95e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a73b92740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5620ab8b65d0 con 0x5620ab951830 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a73b92740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5620ab86f890 con 0x5620ab950f10 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a73b92740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5620ab8b2590 con 0x5620ab95c0f0 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a69d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab95e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a5b7fe640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5a540b6ea0 0x7f5a540b9360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.622 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a69d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab95e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:57398/0 (socket says 172.21.15.59:57398) 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a69d9b640 1 -- 172.21.15.59:0/2257975504 learned_addr learned my addr 172.21.15.59:0/2257975504 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a6a59c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab95bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/2257975504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9512f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.620+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5a540b6ea0 0x7f5a540b9360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a69d9b640 1 -- 172.21.15.59:0/2257975504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 msgr2=0x5620ab9512f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/2257975504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9512f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a69d9b640 1 -- 172.21.15.59:0/2257975504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 msgr2=0x5620ab95bbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/2257975504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab95bbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a5b7fe640 1 -- 172.21.15.59:0/3943935187 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f5a600e2530 con 0x7f5a6c104560 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a69d9b640 1 -- 172.21.15.59:0/2257975504 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ab5c8210 con 0x5620ab95c0f0 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/2257975504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab95e4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ad370 tx=0x7f5a600567b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b20ba4676ca4622 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5a540b6ea0 0x7f5a540b9360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a640098d0 tx=0x7f5a6402a9f0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:21.623 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a58ff9640 1 -- 172.21.15.59:0/2257975504 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a600a9b80 con 0x5620ab95c0f0 2024-05-08T05:37:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a58ff9640 1 -- 172.21.15.59:0/2257975504 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a60009de0 con 0x5620ab95c0f0 2024-05-08T05:37:21.624 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.621+0000 7f5a58ff9640 1 -- 172.21.15.59:0/2257975504 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60063190 con 0x5620ab95c0f0 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 -- 172.21.15.59:0/2257975504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 msgr2=0x5620ab95e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 --2- 172.21.15.59:0/2257975504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab95e4e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5a600ad370 tx=0x7f5a600567b0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 -- 172.21.15.59:0/2257975504 shutdown_connections 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 --2- 172.21.15.59:0/2257975504 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab95e4e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 --2- 172.21.15.59:0/2257975504 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab95bbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 --2- 172.21.15.59:0/2257975504 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9512f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 -- 172.21.15.59:0/2257975504 >> 172.21.15.59:0/2257975504 conn(0x5620ab94d420 msgr2=0x5620ab94d800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 -- 172.21.15.59:0/2257975504 shutdown_connections 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 -- 172.21.15.59:0/2257975504 wait complete. 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.622+0000 7f5a73b92740 1 -- start start 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a73b92740 1 Processor -- start 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab959030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.628 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab959570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab959ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a73b92740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5620ab95ed20 con 0x5620ab95c0f0 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a73b92740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5620ab873370 con 0x5620ab950f10 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a73b92740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5620ab95eea0 con 0x5620ab951830 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a73b92740 10 client.0 _ll_register_callbacks cb 0x5620ab5c7a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a6a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab959030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a6ad9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab959ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a69d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab959570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a69d9b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab959570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:57412/0 (socket says 172.21.15.59:57412) 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a69d9b640 1 -- 172.21.15.59:0/899627922 learned_addr learned my addr 172.21.15.59:0/899627922 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a6ad9d640 1 -- 172.21.15.59:0/899627922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 msgr2=0x5620ab959570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab959570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a6ad9d640 1 -- 172.21.15.59:0/899627922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 msgr2=0x5620ab959030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:21.629 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab959030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.623+0000 7f5a6ad9d640 1 -- 172.21.15.59:0/899627922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ab694620 con 0x5620ab95c0f0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.624+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab959ab0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009bf0 tx=0x7f5a64005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e98b7e37e41007c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.624+0000 7f5a51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.624+0000 7f5a51ffb640 1 -- 172.21.15.59:0/899627922 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64033070 con 0x5620ab95c0f0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.624+0000 7f5a51ffb640 1 -- 172.21.15.59:0/899627922 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f5a64041050 con 0x5620ab95c0f0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.624+0000 7f5a73b92740 1 -- 172.21.15.59:0/899627922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a60009de0 con 0x5620ab95c0f0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.624+0000 7f5a51ffb640 1 -- 172.21.15.59:0/899627922 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5a6402e8f0 con 0x5620ab95c0f0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.624+0000 7f5a73b92740 10 client.16404 fetch_fsmap learned FSMap version 38 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.624+0000 7f5a73b92740 1 -- 172.21.15.59:0/899627922 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5620ab96abe0 con 0x5620ab95c0f0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.624+0000 7f5a51ffb640 1 -- 172.21.15.59:0/899627922 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a64004040 con 0x5620ab95c0f0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a51ffb640 1 -- 172.21.15.59:0/899627922 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f5a64044070 con 0x5620ab95c0f0 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a73b92740 10 client.16404 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:37:21.630 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a73b92740 10 client.16404 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:37:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a73b92740 20 client.16404 populate_metadata read hostname 'smithi059' 2024-05-08T05:37:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a73b92740 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5620ab96c0a0 0x5620ab96e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a73b92740 4 client.16404 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:37:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a73b92740 1 -- 172.21.15.59:0/899627922 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f5a60009de0 con 0x5620ab96c0a0 2024-05-08T05:37:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a6a59c640 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5620ab96c0a0 0x5620ab96e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a6a59c640 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5620ab96c0a0 0x5620ab96e490 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19f3017eda02c7fb server_cookie=1d9abbd33cd7e5a8 in_seq=0 out_seq=0 2024-05-08T05:37:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.625+0000 7f5a51ffb640 10 client.16404 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a51ffb640 1 -- 172.21.15.59:0/899627922 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f5a60009de0 con 0x5620ab96c0a0 2024-05-08T05:37:21.631 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a51ffb640 10 client.16404 handle_command_reply: tid=0 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 client.16404 shutdown 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 -- 172.21.15.59:0/899627922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 msgr2=0x5620ab959ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab959ab0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f5a64009bf0 tx=0x7f5a64005fd0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 -- 172.21.15.59:0/899627922 shutdown_connections 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5620ab96c0a0 0x5620ab96e490 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab959ab0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab959570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a51ffb640 0 client.16404 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 --2- 172.21.15.59:0/899627922 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab959030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 -- 172.21.15.59:0/899627922 >> 172.21.15.59:0/899627922 conn(0x5620ab94d420 msgr2=0x5620ab953150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 -- 172.21.15.59:0/899627922 shutdown_connections 2024-05-08T05:37:21.632 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 1 -- 172.21.15.59:0/899627922 wait complete. 2024-05-08T05:37:21.633 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.626+0000 7f5a73b92740 20 client.16404 trim_cache size 0 max 16384 2024-05-08T05:37:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.645+0000 7f5a73b92740 1 Processor -- start 2024-05-08T05:37:21.646 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a73b92740 1 -- start start 2024-05-08T05:37:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9538d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab973d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab9742a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.652 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a73b92740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1ad9c0 con 0x5620ab95c0f0 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a73b92740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1af500 con 0x5620ab950f10 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a73b92740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1ae090 con 0x5620ab951830 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a6a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9538d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a6a59c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9538d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:58448/0 (socket says 172.21.15.59:58448) 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a6a59c640 1 -- 172.21.15.59:0/2223910527 learned_addr learned my addr 172.21.15.59:0/2223910527 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab9742a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.646+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab973d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a6ad9d640 1 -- 172.21.15.59:0/2223910527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 msgr2=0x5620ab973d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab973d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a6ad9d640 1 -- 172.21.15.59:0/2223910527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 msgr2=0x5620ab9538d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9538d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a6ad9d640 1 -- 172.21.15.59:0/2223910527 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620ab951660 con 0x5620ab95c0f0 2024-05-08T05:37:21.653 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab973d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a6a59c640 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab9742a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5a64004af0 tx=0x7f5a64005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0e48ebf7738d40 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a53fff640 1 -- 172.21.15.59:0/2223910527 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64005bd0 con 0x5620ab95c0f0 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a53fff640 1 -- 172.21.15.59:0/2223910527 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a64003c80 con 0x5620ab95c0f0 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.647+0000 7f5a53fff640 1 -- 172.21.15.59:0/2223910527 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a64004850 con 0x5620ab95c0f0 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 -- 172.21.15.59:0/2223910527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 msgr2=0x5620ab9742a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab9742a0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f5a64004af0 tx=0x7f5a64005ed0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 -- 172.21.15.59:0/2223910527 shutdown_connections 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab95c0f0 0x5620ab9742a0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab951830 0x5620ab973d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.654 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 --2- 172.21.15.59:0/2223910527 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9538d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 -- 172.21.15.59:0/2223910527 >> 172.21.15.59:0/2223910527 conn(0x5620ab94d420 msgr2=0x5620ab94c580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 -- 172.21.15.59:0/2223910527 shutdown_connections 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 -- 172.21.15.59:0/2223910527 wait complete. 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 -- start start 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.648+0000 7f5a73b92740 1 Processor -- start 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9604b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab9609f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a73b92740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab9cf8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a73b92740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1aee30 con 0x5620ab951830 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a73b92740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1ae760 con 0x5620ab950f10 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a73b92740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f5a6c1ab9c0 con 0x5620ab95c0f0 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a73b92740 10 client.0 _ll_register_callbacks cb 0x7f5a6402ef30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a69d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab9609f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.655 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a6ad9d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab9cf8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a69d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab9609f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:37498/0 (socket says 172.21.15.59:37498) 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a69d9b640 1 -- 172.21.15.59:0/4193351189 learned_addr learned my addr 172.21.15.59:0/4193351189 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a6a59c640 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9604b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a69d9b640 1 -- 172.21.15.59:0/4193351189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 msgr2=0x5620ab9cf8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab9cf8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a69d9b640 1 -- 172.21.15.59:0/4193351189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 msgr2=0x5620ab9604b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9604b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a69d9b640 1 -- 172.21.15.59:0/4193351189 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620ab9cfea0 con 0x5620ab951830 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a6a59c640 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a6ad9d640 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab9cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a69d9b640 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab9609f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5a600a5500 tx=0x7f5a600a5940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a31be1786e41c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a51ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a51ffb640 1 -- 172.21.15.59:0/4193351189 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a60020040 con 0x5620ab951830 2024-05-08T05:37:21.656 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a51ffb640 1 -- 172.21.15.59:0/4193351189 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f5a600a3ba0 con 0x5620ab951830 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.649+0000 7f5a73b92740 1 -- 172.21.15.59:0/4193351189 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5620ab8c41e0 con 0x5620ab951830 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a51ffb640 1 -- 172.21.15.59:0/4193351189 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f5a600a9b80 con 0x5620ab951830 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a73b92740 10 client.16410 fetch_fsmap learned FSMap version 38 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a73b92740 1 -- 172.21.15.59:0/4193351189 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5620ab9d7270 con 0x5620ab951830 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a51ffb640 1 -- 172.21.15.59:0/4193351189 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5620ab8c41e0 con 0x5620ab951830 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a51ffb640 1 -- 172.21.15.59:0/4193351189 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f5a600a86e0 con 0x5620ab951830 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a73b92740 10 client.16410 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a73b92740 10 client.16410 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a73b92740 20 client.16410 populate_metadata read hostname 'smithi059' 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a73b92740 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5620ab9d7640 0x5620ab9d9a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a73b92740 4 client.16410 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a73b92740 1 -- 172.21.15.59:0/4193351189 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620ab8be050 con 0x5620ab9d7640 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.650+0000 7f5a6a59c640 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5620ab9d7640 0x5620ab9d9a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:21.657 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.651+0000 7f5a51ffb640 10 client.16410 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:21.658 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.651+0000 7f5a6a59c640 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5620ab9d7640 0x5620ab9d9a30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3845ddbe84ab5196 server_cookie=33469cc174a10c54 in_seq=0 out_seq=0 2024-05-08T05:37:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.680+0000 7f5a51ffb640 1 -- 172.21.15.59:0/4193351189 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620ab8be050 con 0x5620ab9d7640 2024-05-08T05:37:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.680+0000 7f5a51ffb640 10 client.16410 handle_command_reply: tid=0 2024-05-08T05:37:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.680+0000 7f5a73b92740 1 client.16410 shutdown 2024-05-08T05:37:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 -- 172.21.15.59:0/4193351189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 msgr2=0x5620ab9609f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab9609f0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5a600a5500 tx=0x7f5a600a5940 comp rx=0 tx=0).stop 2024-05-08T05:37:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 -- 172.21.15.59:0/4193351189 shutdown_connections 2024-05-08T05:37:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5620ab9d7640 0x5620ab9d9a30 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a51ffb640 0 client.16410 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:21.682 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5620ab95c0f0 0x5620ab9cf8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5620ab951830 0x5620ab9609f0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 --2- 172.21.15.59:0/4193351189 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5620ab950f10 0x5620ab9604b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 -- 172.21.15.59:0/4193351189 >> 172.21.15.59:0/4193351189 conn(0x5620ab94d420 msgr2=0x5620ab958fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 -- 172.21.15.59:0/4193351189 shutdown_connections 2024-05-08T05:37:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 1 -- 172.21.15.59:0/4193351189 wait complete. 2024-05-08T05:37:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.681+0000 7f5a73b92740 20 client.16410 trim_cache size 0 max 16384 2024-05-08T05:37:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:37:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:37:21.683 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:37:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:37:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5a540b6ea0 msgr2=0x7f5a540b9360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5a540b6ea0 0x7f5a540b9360 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f5a640098d0 tx=0x7f5a6402a9f0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c104560 msgr2=0x7f5a6c197320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:21.684 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c104560 0x7f5a6c197320 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f5a6000d950 tx=0x7f5a6000de10 comp rx=0 tx=0).stop 2024-05-08T05:37:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 shutdown_connections 2024-05-08T05:37:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f5a540b6ea0 0x7f5a540b9360 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5a6c10cf80 0x7f5a6c19b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5a6c104560 0x7f5a6c197320 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 --2- 172.21.15.59:0/3943935187 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5a6c103c40 0x7f5a6c196de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 >> 172.21.15.59:0/3943935187 conn(0x7f5a6c078a30 msgr2=0x7f5a6c0fe7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 shutdown_connections 2024-05-08T05:37:21.685 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:21.682+0000 7f5a70d9d640 1 -- 172.21.15.59:0/3943935187 wait complete. 2024-05-08T05:37:23.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:23 smithi059 ceph-mon[30304]: pgmap v242: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 549 op/s 2024-05-08T05:37:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:23 smithi143 ceph-mon[34337]: pgmap v242: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 549 op/s 2024-05-08T05:37:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:23 smithi096 ceph-mon[34253]: pgmap v242: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 549 op/s 2024-05-08T05:37:24.648 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0x63ed6b9 2024-05-08T05:37:25.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:25 smithi059 ceph-mon[30304]: pgmap v243: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 516 op/s 2024-05-08T05:37:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:25 smithi143 ceph-mon[34337]: pgmap v243: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 516 op/s 2024-05-08T05:37:25.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:25 smithi096 ceph-mon[34253]: pgmap v243: 129 pgs: 129 active+clean; 87 MiB data, 1.9 GiB used, 1.0 TiB / 1.0 TiB avail; 12 MiB/s rd, 1.6 MiB/s wr, 516 op/s 2024-05-08T05:37:27.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:27 smithi059 ceph-mon[30304]: pgmap v244: 129 pgs: 129 active+clean; 68 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.4 MiB/s wr, 776 op/s 2024-05-08T05:37:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:27 smithi143 ceph-mon[34337]: pgmap v244: 129 pgs: 129 active+clean; 68 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.4 MiB/s wr, 776 op/s 2024-05-08T05:37:27.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:27 smithi096 ceph-mon[34253]: pgmap v244: 129 pgs: 129 active+clean; 68 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 17 MiB/s rd, 2.4 MiB/s wr, 776 op/s 2024-05-08T05:37:29.337 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:29 smithi059 ceph-mon[30304]: pgmap v245: 129 pgs: 129 active+clean; 68 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 508 op/s 2024-05-08T05:37:29.496 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:29 smithi143 ceph-mon[34337]: pgmap v245: 129 pgs: 129 active+clean; 68 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 508 op/s 2024-05-08T05:37:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:29 smithi096 ceph-mon[34253]: pgmap v245: 129 pgs: 129 active+clean; 68 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 508 op/s 2024-05-08T05:37:31.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:31 smithi059 ceph-mon[30304]: pgmap v246: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 752 op/s 2024-05-08T05:37:31.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:31 smithi143 ceph-mon[34337]: pgmap v246: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 752 op/s 2024-05-08T05:37:31.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:31 smithi096 ceph-mon[34253]: pgmap v246: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.4 MiB/s wr, 752 op/s 2024-05-08T05:37:33.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:33 smithi059 ceph-mon[30304]: pgmap v247: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.6 MiB/s wr, 504 op/s 2024-05-08T05:37:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:33 smithi143 ceph-mon[34337]: pgmap v247: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.6 MiB/s wr, 504 op/s 2024-05-08T05:37:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:33 smithi096 ceph-mon[34253]: pgmap v247: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.6 MiB/s wr, 504 op/s 2024-05-08T05:37:34.796 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.795+0000 7f1cc359e640 1 -- 172.21.15.59:0/2049416319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc41027c0 msgr2=0x7f1cc4102c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.795+0000 7f1cc359e640 1 --2- 172.21.15.59:0/2049416319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc41027c0 0x7f1cc4102c40 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4009930 tx=0x7f1cb402a420 comp rx=0 tx=0).stop 2024-05-08T05:37:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.796+0000 7f1cc359e640 1 -- 172.21.15.59:0/2049416319 shutdown_connections 2024-05-08T05:37:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.796+0000 7f1cc359e640 1 --2- 172.21.15.59:0/2049416319 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc4103180 0x7f1cc410f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.796+0000 7f1cc359e640 1 --2- 172.21.15.59:0/2049416319 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc41027c0 0x7f1cc4102c40 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.796+0000 7f1cc359e640 1 --2- 172.21.15.59:0/2049416319 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc41087d0 0x7f1cc4108bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.796+0000 7f1cc359e640 1 -- 172.21.15.59:0/2049416319 >> 172.21.15.59:0/2049416319 conn(0x7f1cc40fe2f0 msgr2=0x7f1cc4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.796+0000 7f1cc359e640 1 -- 172.21.15.59:0/2049416319 shutdown_connections 2024-05-08T05:37:34.797 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.796+0000 7f1cc359e640 1 -- 172.21.15.59:0/2049416319 wait complete. 2024-05-08T05:37:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.796+0000 7f1cc359e640 1 Processor -- start 2024-05-08T05:37:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc359e640 1 -- start start 2024-05-08T05:37:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc359e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc41027c0 0x7f1cc4074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc359e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc4103180 0x7f1cc40751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc359e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc41087d0 0x7f1cc4075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.799 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc359e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4112a90 con 0x7f1cc41087d0 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc359e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4112910 con 0x7f1cc4103180 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc359e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc4112c10 con 0x7f1cc41027c0 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc4103180 0x7f1cc40751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc2d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc41087d0 0x7f1cc4075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc41027c0 0x7f1cc4074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc4103180 0x7f1cc40751f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:36946/0 (socket says 172.21.15.59:36946) 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/1113563371 learned_addr learned my addr 172.21.15.59:0/1113563371 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/1113563371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc41027c0 msgr2=0x7f1cc4074cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc41027c0 0x7f1cc4074cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc41087d0 msgr2=0x7f1cc4075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc41087d0 0x7f1cc4075730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/1113563371 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1cc41a7580 con 0x7f1cc4103180 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc259c640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc41027c0 0x7f1cc4074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:34.800 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.797+0000 7f1cc2d9d640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc41087d0 0x7f1cc4075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.798+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc4103180 0x7f1cc40751f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4005940 tx=0x7f1cb40042f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.798+0000 7f1cb37fe640 1 -- 172.21.15.59:0/1113563371 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb403f030 con 0x7f1cc4103180 2024-05-08T05:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.798+0000 7f1cb37fe640 1 -- 172.21.15.59:0/1113563371 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb4003cb0 con 0x7f1cc4103180 2024-05-08T05:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.798+0000 7f1cc359e640 1 -- 172.21.15.59:0/1113563371 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1cc41a7810 con 0x7f1cc4103180 2024-05-08T05:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.798+0000 7f1cb37fe640 1 -- 172.21.15.59:0/1113563371 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4035070 con 0x7f1cc4103180 2024-05-08T05:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.798+0000 7f1cc359e640 1 -- 172.21.15.59:0/1113563371 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1cc41a7d00 con 0x7f1cc4103180 2024-05-08T05:37:34.801 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.801+0000 7f1cb37fe640 1 -- 172.21.15.59:0/1113563371 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1cb4042030 con 0x7f1cc4103180 2024-05-08T05:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.801+0000 7f1ccb794740 1 Processor -- start 2024-05-08T05:37:34.802 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1ccb794740 1 -- start start 2024-05-08T05:37:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab80e0 0x5600f7ab84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ac32c0 0x5600f7ac56b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1ccb794740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5600f7a1d5d0 con 0x5600f7ab80e0 2024-05-08T05:37:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1ccb794740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5600f79d6890 con 0x5600f7ab8a00 2024-05-08T05:37:34.807 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1ccb794740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5600f7a19590 con 0x5600f7ac32c0 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc2d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab80e0 0x5600f7ab84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc2d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab80e0 0x5600f7ab84c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52174/0 (socket says 172.21.15.59:52174) 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc259c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc2d9d640 1 -- 172.21.15.59:0/1326069423 learned_addr learned my addr 172.21.15.59:0/1326069423 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cb37fe640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1cac0b2950 0x7f1cac0b4e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/1326069423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ac32c0 0x5600f7ac56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc2d9d640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1cac0b2950 0x7f1cac0b4e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc259c640 1 -- 172.21.15.59:0/1326069423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ac32c0 msgr2=0x5600f7ac56b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc259c640 1 --2- 172.21.15.59:0/1326069423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ac32c0 0x5600f7ac56b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc259c640 1 -- 172.21.15.59:0/1326069423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab80e0 msgr2=0x5600f7ab84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc259c640 1 --2- 172.21.15.59:0/1326069423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab80e0 0x5600f7ab84c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc259c640 1 -- 172.21.15.59:0/1326069423 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600f772f210 con 0x5600f7ab8a00 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cb37fe640 1 -- 172.21.15.59:0/1113563371 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f1cb4103170 con 0x7f1cc4103180 2024-05-08T05:37:34.808 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.802+0000 7f1cc2d9d640 1 --2- 172.21.15.59:0/1326069423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab80e0 0x5600f7ab84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.803+0000 7f1cc259c640 1 --2- 172.21.15.59:0/1326069423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac2d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009fd0 tx=0x7f1cb80118d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d039181ae7da2ee1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.803+0000 7f1cc2d9d640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1cac0b2950 0x7f1cac0b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009440 tx=0x7f1cbc004a40 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.803+0000 7f1cb0ff9640 1 -- 172.21.15.59:0/1326069423 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8004030 con 0x5600f7ab8a00 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.803+0000 7f1cb0ff9640 1 -- 172.21.15.59:0/1326069423 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb80054d0 con 0x5600f7ab8a00 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.803+0000 7f1cb0ff9640 1 -- 172.21.15.59:0/1326069423 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb8004ce0 con 0x5600f7ab8a00 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 -- 172.21.15.59:0/1326069423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 msgr2=0x5600f7ac2d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1326069423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac2d80 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1cb8009fd0 tx=0x7f1cb80118d0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 -- 172.21.15.59:0/1326069423 shutdown_connections 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1326069423 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ac32c0 0x5600f7ac56b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.809 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1326069423 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac2d80 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1326069423 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab80e0 0x5600f7ab84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 -- 172.21.15.59:0/1326069423 >> 172.21.15.59:0/1326069423 conn(0x5600f7ab45f0 msgr2=0x5600f7ab49d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 -- 172.21.15.59:0/1326069423 shutdown_connections 2024-05-08T05:37:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 -- 172.21.15.59:0/1326069423 wait complete. 2024-05-08T05:37:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 -- start start 2024-05-08T05:37:34.810 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 Processor -- start 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7ac0200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5600f7ac5ef0 con 0x5600f7ac32c0 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5600f79da370 con 0x5600f7ab8a00 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.804+0000 7f1ccb794740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5600f7ac6070 con 0x5600f7ab80e0 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1ccb794740 10 client.0 _ll_register_callbacks cb 0x5600f772ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc2d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc2d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52186/0 (socket says 172.21.15.59:52186) 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc2d9d640 1 -- 172.21.15.59:0/611639804 learned_addr learned my addr 172.21.15.59:0/611639804 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc259c640 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7ac0200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc2d9d640 1 -- 172.21.15.59:0/611639804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 msgr2=0x5600f7ac0200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:34.811 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc2d9d640 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7ac0200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc2d9d640 1 -- 172.21.15.59:0/611639804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 msgr2=0x5600f7ac0740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc2d9d640 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac0740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc2d9d640 1 -- 172.21.15.59:0/611639804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600f77fb620 con 0x5600f7ac32c0 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1cc2d9d640 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac0c80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc01aa00 tx=0x7f1cbc00d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b608f60335c630 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1ca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/611639804 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc025410 con 0x5600f7ac32c0 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/611639804 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f1cbc0255b0 con 0x5600f7ac32c0 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1ccb794740 1 -- 172.21.15.59:0/611639804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb80054d0 con 0x5600f7ac32c0 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/611639804 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1cbc030050 con 0x5600f7ac32c0 2024-05-08T05:37:34.812 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1ccb794740 10 client.16422 fetch_fsmap learned FSMap version 38 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.805+0000 7f1ccb794740 1 -- 172.21.15.59:0/611639804 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5600f7ad1dd0 con 0x5600f7ac32c0 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/611639804 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb80054d0 con 0x5600f7ac32c0 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/611639804 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1cbc024920 con 0x5600f7ac32c0 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1ccb794740 10 client.16422 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1ccb794740 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1ccb794740 20 client.16422 populate_metadata read hostname 'smithi059' 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1ccb794740 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5600f7ad32e0 0x5600f7ad56f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1ccb794740 4 client.16422 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1ccb794740 1 -- 172.21.15.59:0/611639804 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5600f7a2b1e0 con 0x5600f7ad32e0 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1cc259c640 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5600f7ad32e0 0x5600f7ad56f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1cc259c640 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5600f7ad32e0 0x5600f7ad56f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1a4c557129ca23f0 server_cookie=cfbe30695fc36094 in_seq=0 out_seq=0 2024-05-08T05:37:34.813 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.806+0000 7f1ca9ffb640 10 client.16422 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/611639804 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5600f7a2b1e0 con 0x5600f7ad32e0 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ca9ffb640 10 client.16422 handle_command_reply: tid=0 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 client.16422 shutdown 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 -- 172.21.15.59:0/611639804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 msgr2=0x5600f7ac0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac0c80 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc01aa00 tx=0x7f1cbc00d040 comp rx=0 tx=0).stop 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 -- 172.21.15.59:0/611639804 shutdown_connections 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5600f7ad32e0 0x5600f7ad56f0 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ca9ffb640 0 client.16422 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac0c80 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac0740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 --2- 172.21.15.59:0/611639804 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7ac0200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.815 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 -- 172.21.15.59:0/611639804 >> 172.21.15.59:0/611639804 conn(0x5600f7ab45f0 msgr2=0x5600f7aba370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 -- 172.21.15.59:0/611639804 shutdown_connections 2024-05-08T05:37:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 1 -- 172.21.15.59:0/611639804 wait complete. 2024-05-08T05:37:34.816 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.808+0000 7f1ccb794740 20 client.16422 trim_cache size 0 max 16384 2024-05-08T05:37:34.837 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1ccb794740 1 Processor -- start 2024-05-08T05:37:34.838 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1ccb794740 1 -- start start 2024-05-08T05:37:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7ab3ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab8a00 0x5600f7adb000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ac32c0 0x5600f7adb540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1ccb794740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41b1ee0 con 0x5600f7ab8a00 2024-05-08T05:37:34.844 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1ccb794740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41b3a20 con 0x5600f7ac32c0 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1ccb794740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41b25b0 con 0x5600f7ab80e0 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1cc1d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab8a00 0x5600f7adb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1cc1d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab8a00 0x5600f7adb000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:52200/0 (socket says 172.21.15.59:52200) 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/3415041164 learned_addr learned my addr 172.21.15.59:0/3415041164 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.837+0000 7f1cc259c640 1 --2- 172.21.15.59:0/3415041164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7ab3ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.838+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/3415041164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 msgr2=0x5600f7ab3ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.838+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/3415041164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7ab3ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.838+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/3415041164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ac32c0 msgr2=0x5600f7adb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.838+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/3415041164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ac32c0 0x5600f7adb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.838+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/3415041164 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5600f7abe7b0 con 0x5600f7ab8a00 2024-05-08T05:37:34.845 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.838+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/3415041164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab8a00 0x5600f7adb000 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40c68f0 tx=0x7f1cb40dff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=726323c7cbf96d64 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.838+0000 7f1cabfff640 1 -- 172.21.15.59:0/3415041164 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb4049040 con 0x5600f7ab8a00 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.838+0000 7f1cabfff640 1 -- 172.21.15.59:0/3415041164 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb40c9c40 con 0x5600f7ab8a00 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.838+0000 7f1cabfff640 1 -- 172.21.15.59:0/3415041164 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40ca390 con 0x5600f7ab8a00 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.839+0000 7f1ccb794740 1 -- 172.21.15.59:0/3415041164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab8a00 msgr2=0x5600f7adb000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.839+0000 7f1ccb794740 1 --2- 172.21.15.59:0/3415041164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab8a00 0x5600f7adb000 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f1cb40c68f0 tx=0x7f1cb40dff90 comp rx=0 tx=0).stop 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.839+0000 7f1ccb794740 1 -- 172.21.15.59:0/3415041164 shutdown_connections 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.839+0000 7f1ccb794740 1 --2- 172.21.15.59:0/3415041164 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ac32c0 0x5600f7adb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.839+0000 7f1ccb794740 1 --2- 172.21.15.59:0/3415041164 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ab8a00 0x5600f7adb000 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.839+0000 7f1ccb794740 1 --2- 172.21.15.59:0/3415041164 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7ab3ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.839+0000 7f1ccb794740 1 -- 172.21.15.59:0/3415041164 >> 172.21.15.59:0/3415041164 conn(0x5600f7ab45f0 msgr2=0x5600f7acdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 -- 172.21.15.59:0/3415041164 shutdown_connections 2024-05-08T05:37:34.846 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 -- 172.21.15.59:0/3415041164 wait complete. 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 -- start start 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 Processor -- start 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7acb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41b3350 con 0x5600f7ac32c0 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41b2c80 con 0x5600f7ab8a00 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.840+0000 7f1ccb794740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1cc41afee0 con 0x5600f7ab80e0 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1ccb794740 10 client.0 _ll_register_callbacks cb 0x7f1cbc01a680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7acb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.847 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc1d9b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc259c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7acb4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:56230/0 (socket says 172.21.15.59:56230) 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc259c640 1 -- 172.21.15.59:0/1598337844 learned_addr learned my addr 172.21.15.59:0/1598337844 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc2d9d640 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/1598337844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 msgr2=0x5600f7acb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7acb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/1598337844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 msgr2=0x5600f7ac87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc1d9b640 1 -- 172.21.15.59:0/1598337844 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5600f7ac8d10 con 0x5600f7ab8a00 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc259c640 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7acb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1cc1d9b640 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac8290 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4000c00 tx=0x7f1cb40c67f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb9372ef2c8695f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:34.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.841+0000 7f1ca9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.842+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/1598337844 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cb40044e0 con 0x5600f7ab8a00 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.842+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/1598337844 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f1cb40ca390 con 0x5600f7ab8a00 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.842+0000 7f1ccb794740 1 -- 172.21.15.59:0/1598337844 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cb40c9c40 con 0x5600f7ab8a00 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.842+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/1598337844 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f1cb40ce430 con 0x5600f7ab8a00 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.842+0000 7f1ccb794740 10 client.25979 fetch_fsmap learned FSMap version 38 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.842+0000 7f1ccb794740 1 -- 172.21.15.59:0/1598337844 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5600f7b3e830 con 0x5600f7ab8a00 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.842+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/1598337844 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cb40c9c40 con 0x5600f7ab8a00 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/1598337844 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1cb40d2a50 con 0x5600f7ab8a00 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1ccb794740 10 client.25979 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1ccb794740 10 client.25979 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1ccb794740 20 client.25979 populate_metadata read hostname 'smithi059' 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5600f7ad32e0 0x5600f7b40e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:34.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1ccb794740 4 client.25979 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:37:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1ccb794740 1 -- 172.21.15.59:0/1598337844 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5600f7a25050 con 0x5600f7ad32e0 2024-05-08T05:37:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1cc259c640 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5600f7ad32e0 0x5600f7b40e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1cc259c640 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5600f7ad32e0 0x5600f7b40e50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ad3f043953857f5 server_cookie=ccb78a6918e51fd4 in_seq=0 out_seq=0 2024-05-08T05:37:34.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.843+0000 7f1ca9ffb640 10 client.25979 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ca9ffb640 1 -- 172.21.15.59:0/1598337844 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5600f7a25050 con 0x5600f7ad32e0 2024-05-08T05:37:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ca9ffb640 10 client.25979 handle_command_reply: tid=0 2024-05-08T05:37:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 client.25979 shutdown 2024-05-08T05:37:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 -- 172.21.15.59:0/1598337844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 msgr2=0x5600f7ac8290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac8290 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4000c00 tx=0x7f1cb40c67f0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 -- 172.21.15.59:0/1598337844 shutdown_connections 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5600f7ad32e0 0x5600f7b40e50 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5600f7ac32c0 0x5600f7ac87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ca9ffb640 0 client.25979 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5600f7ab8a00 0x5600f7ac8290 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 --2- 172.21.15.59:0/1598337844 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5600f7ab80e0 0x5600f7acb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 -- 172.21.15.59:0/1598337844 >> 172.21.15.59:0/1598337844 conn(0x5600f7ab45f0 msgr2=0x5600f7ace7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 -- 172.21.15.59:0/1598337844 shutdown_connections 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 1 -- 172.21.15.59:0/1598337844 wait complete. 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.863+0000 7f1ccb794740 20 client.25979 trim_cache size 0 max 16384 2024-05-08T05:37:34.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:37:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:37:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:37:34.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:37:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 -- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1cac0b2950 msgr2=0x7f1cac0b4e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1cac0b2950 0x7f1cac0b4e10 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc009440 tx=0x7f1cbc004a40 comp rx=0 tx=0).stop 2024-05-08T05:37:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 -- 172.21.15.59:0/1113563371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc4103180 msgr2=0x7f1cc40751f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc4103180 0x7f1cc40751f0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1cb4005940 tx=0x7f1cb40042f0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 -- 172.21.15.59:0/1113563371 shutdown_connections 2024-05-08T05:37:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1cac0b2950 0x7f1cac0b4e10 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1cc41087d0 0x7f1cc4075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1cc4103180 0x7f1cc40751f0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.867 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 --2- 172.21.15.59:0/1113563371 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1cc41027c0 0x7f1cc4074cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 -- 172.21.15.59:0/1113563371 >> 172.21.15.59:0/1113563371 conn(0x7f1cc40fe2f0 msgr2=0x7f1cc40febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 -- 172.21.15.59:0/1113563371 shutdown_connections 2024-05-08T05:37:34.868 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:34.864+0000 7f1cc359e640 1 -- 172.21.15.59:0/1113563371 wait complete. 2024-05-08T05:37:35.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:35 smithi059 ceph-mon[30304]: pgmap v248: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.6 MiB/s wr, 504 op/s 2024-05-08T05:37:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:35 smithi143 ceph-mon[34337]: pgmap v248: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.6 MiB/s wr, 504 op/s 2024-05-08T05:37:35.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:35 smithi096 ceph-mon[34253]: pgmap v248: 129 pgs: 129 active+clean; 88 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.6 MiB/s wr, 504 op/s 2024-05-08T05:37:37.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:37 smithi059 ceph-mon[30304]: pgmap v249: 129 pgs: 129 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 765 op/s 2024-05-08T05:37:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:37 smithi143 ceph-mon[34337]: pgmap v249: 129 pgs: 129 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 765 op/s 2024-05-08T05:37:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:37 smithi096 ceph-mon[34253]: pgmap v249: 129 pgs: 129 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.3 MiB/s wr, 765 op/s 2024-05-08T05:37:39.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:39 smithi059 ceph-mon[30304]: pgmap v250: 129 pgs: 129 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.5 MiB/s wr, 505 op/s 2024-05-08T05:37:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:39 smithi143 ceph-mon[34337]: pgmap v250: 129 pgs: 129 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.5 MiB/s wr, 505 op/s 2024-05-08T05:37:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:39 smithi096 ceph-mon[34253]: pgmap v250: 129 pgs: 129 active+clean; 87 MiB data, 2.0 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.5 MiB/s wr, 505 op/s 2024-05-08T05:37:41.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:41 smithi059 ceph-mon[30304]: pgmap v251: 129 pgs: 129 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.2 MiB/s wr, 740 op/s 2024-05-08T05:37:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:41 smithi143 ceph-mon[34337]: pgmap v251: 129 pgs: 129 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.2 MiB/s wr, 740 op/s 2024-05-08T05:37:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:41 smithi096 ceph-mon[34253]: pgmap v251: 129 pgs: 129 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.2 MiB/s wr, 740 op/s 2024-05-08T05:37:42.081 INFO:tasks.workunit.client.0.smithi059.stdout:copying to largest ever: 0x63ef9ee 2024-05-08T05:37:43.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:43 smithi059 ceph-mon[30304]: pgmap v252: 129 pgs: 129 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.4 MiB/s wr, 496 op/s 2024-05-08T05:37:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:43 smithi143 ceph-mon[34337]: pgmap v252: 129 pgs: 129 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.4 MiB/s wr, 496 op/s 2024-05-08T05:37:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:43 smithi096 ceph-mon[34253]: pgmap v252: 129 pgs: 129 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.4 MiB/s wr, 496 op/s 2024-05-08T05:37:45.330 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:45 smithi096 ceph-mon[34253]: pgmap v253: 129 pgs: 129 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.4 MiB/s wr, 496 op/s 2024-05-08T05:37:45.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:45 smithi059 ceph-mon[30304]: pgmap v253: 129 pgs: 129 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.4 MiB/s wr, 496 op/s 2024-05-08T05:37:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:45 smithi143 ceph-mon[34337]: pgmap v253: 129 pgs: 129 active+clean; 87 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.5 MiB/s rd, 1.4 MiB/s wr, 496 op/s 2024-05-08T05:37:46.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:46 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:37:46.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:46 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:37:46.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:46 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:37:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:47 smithi059 ceph-mon[30304]: pgmap v254: 129 pgs: 129 active+clean; 79 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 738 op/s 2024-05-08T05:37:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:47 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:37:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:47 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:37:47.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:47 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:37:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:47 smithi143 ceph-mon[34337]: pgmap v254: 129 pgs: 129 active+clean; 79 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 738 op/s 2024-05-08T05:37:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:47 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:37:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:47 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:37:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:47 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:37:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:47 smithi096 ceph-mon[34253]: pgmap v254: 129 pgs: 129 active+clean; 79 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 738 op/s 2024-05-08T05:37:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:47 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:37:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:47 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:37:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:47 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:37:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 -- 172.21.15.59:0/1721575740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f169410cfb0 msgr2=0x7f169410f420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 --2- 172.21.15.59:0/1721575740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f169410cfb0 0x7f169410f420 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f168c009ca0 tx=0x7f168c02a950 comp rx=0 tx=0).stop 2024-05-08T05:37:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 -- 172.21.15.59:0/1721575740 shutdown_connections 2024-05-08T05:37:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 --2- 172.21.15.59:0/1721575740 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f169410cfb0 0x7f169410f420 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 --2- 172.21.15.59:0/1721575740 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1694104500 0x7f169410c9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:47.991 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 --2- 172.21.15.59:0/1721575740 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1694103be0 0x7f1694103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 -- 172.21.15.59:0/1721575740 >> 172.21.15.59:0/1721575740 conn(0x7f16940fd380 msgr2=0x7f16940ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 -- 172.21.15.59:0/1721575740 shutdown_connections 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 -- 172.21.15.59:0/1721575740 wait complete. 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 Processor -- start 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.990+0000 7f169c519640 1 -- start start 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f169c519640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1694103be0 0x7f1694199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f169c519640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1694104500 0x7f16941996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f169c519640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f169410cfb0 0x7f169419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f169c519640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1694112c70 con 0x7f1694104500 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f169c519640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1694112af0 con 0x7f169410cfb0 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f169c519640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1694112df0 con 0x7f1694103be0 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f1699ab4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1694104500 0x7f16941996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f1699ab4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1694104500 0x7f16941996d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50262/0 (socket says 172.21.15.59:50262) 2024-05-08T05:37:47.992 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f1699ab4640 1 -- 172.21.15.59:0/417990749 learned_addr learned my addr 172.21.15.59:0/417990749 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f169aab6640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f169410cfb0 0x7f169419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f1699ab4640 1 -- 172.21.15.59:0/417990749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1694103be0 msgr2=0x7f1694199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f169a2b5640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1694103be0 0x7f1694199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f1699ab4640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1694103be0 0x7f1694199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f1699ab4640 1 -- 172.21.15.59:0/417990749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f169410cfb0 msgr2=0x7f169419da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f1699ab4640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f169410cfb0 0x7f169419da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f1699ab4640 1 -- 172.21.15.59:0/417990749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f169419e130 con 0x7f1694104500 2024-05-08T05:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f169a2b5640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1694103be0 0x7f1694199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:37:47.994 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f1699ab4640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1694104500 0x7f16941996d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f168400ea60 tx=0x7f168400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.991+0000 7f168b7fe640 1 -- 172.21.15.59:0/417990749 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168400cde0 con 0x7f1694104500 2024-05-08T05:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.992+0000 7f169c519640 1 -- 172.21.15.59:0/417990749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f169419e420 con 0x7f1694104500 2024-05-08T05:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.992+0000 7f168b7fe640 1 -- 172.21.15.59:0/417990749 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16840035f0 con 0x7f1694104500 2024-05-08T05:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.992+0000 7f168b7fe640 1 -- 172.21.15.59:0/417990749 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684010690 con 0x7f1694104500 2024-05-08T05:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.992+0000 7f169c519640 1 -- 172.21.15.59:0/417990749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16941a5cb0 con 0x7f1694104500 2024-05-08T05:37:47.995 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.995+0000 7f168b7fe640 1 -- 172.21.15.59:0/417990749 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f16840037a0 con 0x7f1694104500 2024-05-08T05:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.996+0000 7f168b7fe640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f16800b2a60 0x7f16800b4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.996+0000 7f168b7fe640 1 -- 172.21.15.59:0/417990749 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f16840e2e30 con 0x7f1694104500 2024-05-08T05:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.996+0000 7f169a2b5640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f16800b2a60 0x7f16800b4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:47.996 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.996+0000 7f169f237740 1 Processor -- start 2024-05-08T05:37:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169f237740 1 -- start start 2024-05-08T05:37:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169a2b5640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f16800b2a60 0x7f16800b4f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1690006fd0 tx=0x7f1690008040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169f237740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f14202f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:47.998 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169f237740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f1420830 0x5653f142abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169f237740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f142b0f0 0x5653f142d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169f237740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5653f13855d0 con 0x5653f142b0f0 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169f237740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653f133e890 con 0x5653f1420830 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169f237740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5653f1381590 con 0x5653f141ff10 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f1699ab4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f1420830 0x5653f142abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169aab6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f142b0f0 0x5653f142d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169aab6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f142b0f0 0x5653f142d4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50278/0 (socket says 172.21.15.59:50278) 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f1699ab4640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f1420830 0x5653f142abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:40586/0 (socket says 172.21.15.59:40586) 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169aab6640 1 -- 172.21.15.59:0/3749320348 learned_addr learned my addr 172.21.15.59:0/3749320348 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f169a2b5640 1 --2- 172.21.15.59:0/3749320348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f14202f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f1699ab4640 1 -- 172.21.15.59:0/3749320348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 msgr2=0x5653f14202f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f1699ab4640 1 --2- 172.21.15.59:0/3749320348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f14202f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f1699ab4640 1 -- 172.21.15.59:0/3749320348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f142b0f0 msgr2=0x5653f142d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:47.999 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f1699ab4640 1 --2- 172.21.15.59:0/3749320348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f142b0f0 0x5653f142d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.997+0000 7f1699ab4640 1 -- 172.21.15.59:0/3749320348 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f1097210 con 0x5653f1420830 2024-05-08T05:37:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.998+0000 7f1699ab4640 1 --2- 172.21.15.59:0/3749320348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f1420830 0x5653f142abb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f16840e0150 tx=0x7f16840a9450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fad1a959ca5880a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.998+0000 7f1688bfe640 1 -- 172.21.15.59:0/3749320348 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840a9ce0 con 0x5653f1420830 2024-05-08T05:37:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.998+0000 7f1688bfe640 1 -- 172.21.15.59:0/3749320348 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16840ae090 con 0x5653f1420830 2024-05-08T05:37:48.000 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.998+0000 7f1688bfe640 1 -- 172.21.15.59:0/3749320348 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840b2910 con 0x5653f1420830 2024-05-08T05:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 -- 172.21.15.59:0/3749320348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f1420830 msgr2=0x5653f142abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 --2- 172.21.15.59:0/3749320348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f1420830 0x5653f142abb0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f16840e0150 tx=0x7f16840a9450 comp rx=0 tx=0).stop 2024-05-08T05:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 -- 172.21.15.59:0/3749320348 shutdown_connections 2024-05-08T05:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 --2- 172.21.15.59:0/3749320348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f142b0f0 0x5653f142d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 --2- 172.21.15.59:0/3749320348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f1420830 0x5653f142abb0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 --2- 172.21.15.59:0/3749320348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f14202f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 -- 172.21.15.59:0/3749320348 >> 172.21.15.59:0/3749320348 conn(0x5653f141c420 msgr2=0x5653f141c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:48.005 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 -- 172.21.15.59:0/3749320348 shutdown_connections 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 -- 172.21.15.59:0/3749320348 wait complete. 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 -- start start 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:47.999+0000 7f169f237740 1 Processor -- start 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169f237740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f1428140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169f237740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f1428680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169f237740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f142b0f0 0x5653f1428bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169f237740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5653f1432130 con 0x5653f1420830 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169f237740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5653f1342370 con 0x5653f142b0f0 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169f237740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5653f14322b0 con 0x5653f141ff10 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169a2b5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f1428140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169f237740 10 client.0 _ll_register_callbacks cb 0x5653f1096a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f1699ab4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f1428680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f1699ab4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f1428680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50286/0 (socket says 172.21.15.59:50286) 2024-05-08T05:37:48.006 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f1699ab4640 1 -- 172.21.15.59:0/1112593081 learned_addr learned my addr 172.21.15.59:0/1112593081 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169aab6640 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f142b0f0 0x5653f1428bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169a2b5640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f1428140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:41336/0 (socket says 172.21.15.59:41336) 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f1699ab4640 1 -- 172.21.15.59:0/1112593081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 msgr2=0x5653f1428140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f1699ab4640 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f1428140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f1699ab4640 1 -- 172.21.15.59:0/1112593081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f142b0f0 msgr2=0x5653f1428bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f1699ab4640 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f142b0f0 0x5653f1428bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f1699ab4640 1 -- 172.21.15.59:0/1112593081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f1163620 con 0x5653f1420830 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.000+0000 7f169a2b5640 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f1428140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f1699ab4640 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f1428680 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f168400c760 tx=0x7f1684003e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=853c3ffcd82b08b2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f167dffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f167dffb640 1 -- 172.21.15.59:0/1112593081 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168406e060 con 0x5653f1420830 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f167dffb640 1 -- 172.21.15.59:0/1112593081 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f16840a62d0 con 0x5653f1420830 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f169f237740 1 -- 172.21.15.59:0/1112593081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16840ae090 con 0x5653f1420830 2024-05-08T05:37:48.007 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f167dffb640 1 -- 172.21.15.59:0/1112593081 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f16840a5f70 con 0x5653f1420830 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f169f237740 10 client.16440 fetch_fsmap learned FSMap version 38 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f169f237740 1 -- 172.21.15.59:0/1112593081 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5653f143e0c0 con 0x5653f1420830 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f167dffb640 1 -- 172.21.15.59:0/1112593081 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16840ae090 con 0x5653f1420830 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f167dffb640 1 -- 172.21.15.59:0/1112593081 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1684004000 con 0x5653f1420830 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f169f237740 10 client.16440 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f169f237740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.001+0000 7f169f237740 20 client.16440 populate_metadata read hostname 'smithi059' 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.002+0000 7f169f237740 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5653f143f410 0x5653f1441800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.002+0000 7f169f237740 4 client.16440 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.002+0000 7f169f237740 1 -- 172.21.15.59:0/1112593081 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653f13931e0 con 0x5653f143f410 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.002+0000 7f169a2b5640 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5653f143f410 0x5653f1441800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.002+0000 7f169a2b5640 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5653f143f410 0x5653f1441800 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e30dd079b46951 server_cookie=27e7a9ea14ea87df in_seq=0 out_seq=0 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.002+0000 7f167dffb640 10 client.16440 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f167dffb640 1 -- 172.21.15.59:0/1112593081 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5653f13931e0 con 0x5653f143f410 2024-05-08T05:37:48.008 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f167dffb640 10 client.16440 handle_command_reply: tid=0 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 client.16440 shutdown 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 -- 172.21.15.59:0/1112593081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 msgr2=0x5653f1428680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f1428680 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f168400c760 tx=0x7f1684003e00 comp rx=0 tx=0).stop 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 -- 172.21.15.59:0/1112593081 shutdown_connections 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5653f143f410 0x5653f1441800 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f142b0f0 0x5653f1428bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f167dffb640 0 client.16440 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f1428680 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 --2- 172.21.15.59:0/1112593081 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f141ff10 0x5653f1428140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 -- 172.21.15.59:0/1112593081 >> 172.21.15.59:0/1112593081 conn(0x5653f141c420 msgr2=0x5653f1422150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 -- 172.21.15.59:0/1112593081 shutdown_connections 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.003+0000 7f169f237740 1 -- 172.21.15.59:0/1112593081 wait complete. 2024-05-08T05:37:48.009 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.004+0000 7f169f237740 20 client.16440 trim_cache size 0 max 16384 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.025+0000 7f169f237740 1 Processor -- start 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.025+0000 7f169f237740 1 -- start start 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.025+0000 7f169f237740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f141ff10 0x5653f1422ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.025+0000 7f169f237740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f14253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.025+0000 7f169f237740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f142b0f0 0x5653f1425900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169a2b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f141ff10 0x5653f1422ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f1699ab4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f14253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169a2b5640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f141ff10 0x5653f1422ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:40604/0 (socket says 172.21.15.59:40604) 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169a2b5640 1 -- 172.21.15.59:0/1996642372 learned_addr learned my addr 172.21.15.59:0/1996642372 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169f237740 1 -- 172.21.15.59:0/1996642372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16941adca0 con 0x5653f1420830 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169f237740 1 -- 172.21.15.59:0/1996642372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16941af7e0 con 0x5653f141ff10 2024-05-08T05:37:48.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169f237740 1 -- 172.21.15.59:0/1996642372 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f16941ae370 con 0x5653f142b0f0 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169aab6640 1 --2- 172.21.15.59:0/1996642372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f142b0f0 0x5653f1425900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169a2b5640 1 -- 172.21.15.59:0/1996642372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f142b0f0 msgr2=0x5653f1425900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169a2b5640 1 --2- 172.21.15.59:0/1996642372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f142b0f0 0x5653f1425900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169a2b5640 1 -- 172.21.15.59:0/1996642372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 msgr2=0x5653f14253c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169a2b5640 1 --2- 172.21.15.59:0/1996642372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f14253c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169a2b5640 1 -- 172.21.15.59:0/1996642372 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653f1425e40 con 0x5653f141ff10 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169aab6640 1 --2- 172.21.15.59:0/1996642372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f142b0f0 0x5653f1425900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.026+0000 7f169a2b5640 1 --2- 172.21.15.59:0/1996642372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f141ff10 0x5653f1422ac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f169000d010 tx=0x7f1690012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=73a4b2b83511d1b3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f167ffff640 1 -- 172.21.15.59:0/1996642372 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f169002e070 con 0x5653f141ff10 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f167ffff640 1 -- 172.21.15.59:0/1996642372 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16900282d0 con 0x5653f141ff10 2024-05-08T05:37:48.028 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f167ffff640 1 -- 172.21.15.59:0/1996642372 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690013420 con 0x5653f141ff10 2024-05-08T05:37:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f169f237740 1 -- 172.21.15.59:0/1996642372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f141ff10 msgr2=0x5653f1422ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f169f237740 1 --2- 172.21.15.59:0/1996642372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f141ff10 0x5653f1422ac0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f169000d010 tx=0x7f1690012000 comp rx=0 tx=0).stop 2024-05-08T05:37:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f169f237740 1 -- 172.21.15.59:0/1996642372 shutdown_connections 2024-05-08T05:37:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f169f237740 1 --2- 172.21.15.59:0/1996642372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f142b0f0 0x5653f1425900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f169f237740 1 --2- 172.21.15.59:0/1996642372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f1420830 0x5653f14253c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f169f237740 1 --2- 172.21.15.59:0/1996642372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f141ff10 0x5653f1422ac0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.027+0000 7f169f237740 1 -- 172.21.15.59:0/1996642372 >> 172.21.15.59:0/1996642372 conn(0x5653f141c420 msgr2=0x5653f1417450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.028+0000 7f169f237740 1 -- 172.21.15.59:0/1996642372 shutdown_connections 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.028+0000 7f169f237740 1 -- 172.21.15.59:0/1996642372 wait complete. 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.028+0000 7f169f237740 1 -- start start 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.028+0000 7f169f237740 1 Processor -- start 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.028+0000 7f169f237740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f141ff10 0x5653f14427a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.028+0000 7f169f237740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f1420830 0x5653f1442ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.028+0000 7f169f237740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f142b0f0 0x5653f14a8930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169f237740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f16941af110 con 0x5653f141ff10 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169f237740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f16941aea40 con 0x5653f142b0f0 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169a2b5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f141ff10 0x5653f14427a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169a2b5640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f141ff10 0x5653f14427a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:50308/0 (socket says 172.21.15.59:50308) 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169a2b5640 1 -- 172.21.15.59:0/2774773820 learned_addr learned my addr 172.21.15.59:0/2774773820 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:37:48.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169f237740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f16941abca0 con 0x5653f1420830 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169f237740 10 client.0 _ll_register_callbacks cb 0x7f1684005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169a2b5640 1 -- 172.21.15.59:0/2774773820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f1420830 msgr2=0x5653f1442ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169a2b5640 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f1420830 0x5653f1442ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169a2b5640 1 -- 172.21.15.59:0/2774773820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f142b0f0 msgr2=0x5653f14a8930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169a2b5640 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f142b0f0 0x5653f14a8930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169a2b5640 1 -- 172.21.15.59:0/2774773820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653f14a8e70 con 0x5653f141ff10 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169a2b5640 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f141ff10 0x5653f14427a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f169000cd20 tx=0x7f169000cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5693188fcc182907 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f167dffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f167dffb640 1 -- 172.21.15.59:0/2774773820 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1690037060 con 0x5653f141ff10 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f167dffb640 1 -- 172.21.15.59:0/2774773820 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f1690027040 con 0x5653f141ff10 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f169f237740 1 -- 172.21.15.59:0/2774773820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f16900282d0 con 0x5653f141ff10 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.029+0000 7f167dffb640 1 -- 172.21.15.59:0/2774773820 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f169002e030 con 0x5653f141ff10 2024-05-08T05:37:48.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f169f237740 10 client.16446 fetch_fsmap learned FSMap version 38 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f169f237740 1 -- 172.21.15.59:0/2774773820 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5653f14a9fc0 con 0x5653f141ff10 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f167dffb640 1 -- 172.21.15.59:0/2774773820 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16900282d0 con 0x5653f141ff10 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f167dffb640 1 -- 172.21.15.59:0/2774773820 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f1690028ce0 con 0x5653f141ff10 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f169f237740 10 client.16446 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f169f237740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f169f237740 20 client.16446 populate_metadata read hostname 'smithi059' 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f169f237740 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5653f143f410 0x5653f14ac690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f169f237740 4 client.16446 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f169f237740 1 -- 172.21.15.59:0/2774773820 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653f138d050 con 0x5653f143f410 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.030+0000 7f1699ab4640 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5653f143f410 0x5653f14ac690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.031+0000 7f167dffb640 10 client.16446 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:48.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.031+0000 7f1699ab4640 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5653f143f410 0x5653f14ac690 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2be508dae3bf5de server_cookie=6eaceeb7fc26e2f5 in_seq=0 out_seq=0 2024-05-08T05:37:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.054+0000 7f167dffb640 1 -- 172.21.15.59:0/2774773820 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653f138d050 con 0x5653f143f410 2024-05-08T05:37:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.054+0000 7f167dffb640 10 client.16446 handle_command_reply: tid=0 2024-05-08T05:37:48.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.054+0000 7f169f237740 1 client.16446 shutdown 2024-05-08T05:37:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.057+0000 7f169f237740 1 -- 172.21.15.59:0/2774773820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f141ff10 msgr2=0x5653f14427a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.057+0000 7f169f237740 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f141ff10 0x5653f14427a0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f169000cd20 tx=0x7f169000cef0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.057+0000 7f169f237740 1 -- 172.21.15.59:0/2774773820 shutdown_connections 2024-05-08T05:37:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.057+0000 7f169f237740 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5653f143f410 0x5653f14ac690 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.057+0000 7f169f237740 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5653f142b0f0 0x5653f14a8930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.057+0000 7f169f237740 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5653f1420830 0x5653f1442ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.057+0000 7f169f237740 1 --2- 172.21.15.59:0/2774773820 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5653f141ff10 0x5653f14427a0 secure :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f169000cd20 tx=0x7f169000cef0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.057+0000 7f169f237740 1 -- 172.21.15.59:0/2774773820 >> 172.21.15.59:0/2774773820 conn(0x5653f141c420 msgr2=0x5653f141acf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:48.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.058+0000 7f167dffb640 0 client.16446 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:37:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.058+0000 7f169f237740 1 -- 172.21.15.59:0/2774773820 shutdown_connections 2024-05-08T05:37:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.058+0000 7f169f237740 1 -- 172.21.15.59:0/2774773820 wait complete. 2024-05-08T05:37:48.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.059+0000 7f169f237740 20 client.16446 trim_cache size 0 max 16384 2024-05-08T05:37:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:37:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:37:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:37:48.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:37:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.062+0000 7f169c519640 1 -- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f16800b2a60 msgr2=0x7f16800b4f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.062+0000 7f169c519640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f16800b2a60 0x7f16800b4f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1690006fd0 tx=0x7f1690008040 comp rx=0 tx=0).stop 2024-05-08T05:37:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.062+0000 7f169c519640 1 -- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1694104500 msgr2=0x7f16941996d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:37:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.062+0000 7f169c519640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1694104500 0x7f16941996d0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f168400ea60 tx=0x7f168400ef20 comp rx=0 tx=0).stop 2024-05-08T05:37:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.063+0000 7f169c519640 1 -- 172.21.15.59:0/417990749 shutdown_connections 2024-05-08T05:37:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.063+0000 7f169c519640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f16800b2a60 0x7f16800b4f20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.063+0000 7f169c519640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f169410cfb0 0x7f169419da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.063 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.063+0000 7f169c519640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1694104500 0x7f16941996d0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.063+0000 7f169c519640 1 --2- 172.21.15.59:0/417990749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1694103be0 0x7f1694199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:37:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.063+0000 7f169c519640 1 -- 172.21.15.59:0/417990749 >> 172.21.15.59:0/417990749 conn(0x7f16940fd380 msgr2=0x7f16940feaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:37:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.063+0000 7f169c519640 1 -- 172.21.15.59:0/417990749 shutdown_connections 2024-05-08T05:37:48.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:37:48.064+0000 7f169c519640 1 -- 172.21.15.59:0/417990749 wait complete. 2024-05-08T05:37:49.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:49 smithi059 ceph-mon[30304]: pgmap v255: 129 pgs: 129 active+clean; 79 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.4 MiB/s wr, 477 op/s 2024-05-08T05:37:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:49 smithi143 ceph-mon[34337]: pgmap v255: 129 pgs: 129 active+clean; 79 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.4 MiB/s wr, 477 op/s 2024-05-08T05:37:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:49 smithi096 ceph-mon[34253]: pgmap v255: 129 pgs: 129 active+clean; 79 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 9.0 MiB/s rd, 1.4 MiB/s wr, 477 op/s 2024-05-08T05:37:51.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:51 smithi059 ceph-mon[30304]: pgmap v256: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 714 op/s 2024-05-08T05:37:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:51 smithi143 ceph-mon[34337]: pgmap v256: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 714 op/s 2024-05-08T05:37:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:51 smithi096 ceph-mon[34253]: pgmap v256: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.1 MiB/s wr, 714 op/s 2024-05-08T05:37:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:53 smithi143 ceph-mon[34337]: pgmap v257: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 479 op/s 2024-05-08T05:37:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:53 smithi096 ceph-mon[34253]: pgmap v257: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 479 op/s 2024-05-08T05:37:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:53 smithi059 ceph-mon[30304]: pgmap v257: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 479 op/s 2024-05-08T05:37:55.388 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:55 smithi059 ceph-mon[30304]: pgmap v258: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 479 op/s 2024-05-08T05:37:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:55 smithi143 ceph-mon[34337]: pgmap v258: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 479 op/s 2024-05-08T05:37:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:55 smithi096 ceph-mon[34253]: pgmap v258: 129 pgs: 129 active+clean; 64 MiB data, 2.1 GiB used, 1.0 TiB / 1.0 TiB avail; 8.6 MiB/s rd, 1.4 MiB/s wr, 479 op/s 2024-05-08T05:37:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:57 smithi143 ceph-mon[34337]: pgmap v259: 129 pgs: 129 active+clean; 74 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.2 MiB/s wr, 723 op/s 2024-05-08T05:37:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:57 smithi096 ceph-mon[34253]: pgmap v259: 129 pgs: 129 active+clean; 74 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.2 MiB/s wr, 723 op/s 2024-05-08T05:37:57.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:57 smithi059 ceph-mon[30304]: pgmap v259: 129 pgs: 129 active+clean; 74 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 2.2 MiB/s wr, 723 op/s 2024-05-08T05:37:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:37:59 smithi143 ceph-mon[34337]: pgmap v260: 129 pgs: 129 active+clean; 74 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.4 MiB/s wr, 480 op/s 2024-05-08T05:37:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:37:59 smithi096 ceph-mon[34253]: pgmap v260: 129 pgs: 129 active+clean; 74 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.4 MiB/s wr, 480 op/s 2024-05-08T05:37:59.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:37:59 smithi059 ceph-mon[30304]: pgmap v260: 129 pgs: 129 active+clean; 74 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.4 MiB/s wr, 480 op/s 2024-05-08T05:38:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.184+0000 7f44c0037640 1 -- 172.21.15.59:0/2534060961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8104480 msgr2=0x7f44b810c960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.184+0000 7f44c0037640 1 --2- 172.21.15.59:0/2534060961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8104480 0x7f44b810c960 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009900 tx=0x7f44a802a480 comp rx=0 tx=0).stop 2024-05-08T05:38:01.187 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.184+0000 7f44c0037640 1 -- 172.21.15.59:0/2534060961 shutdown_connections 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.184+0000 7f44c0037640 1 --2- 172.21.15.59:0/2534060961 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44b810cf80 0x7f44b810f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.184+0000 7f44c0037640 1 --2- 172.21.15.59:0/2534060961 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8104480 0x7f44b810c960 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.184+0000 7f44c0037640 1 --2- 172.21.15.59:0/2534060961 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8103b60 0x7f44b8103f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.184+0000 7f44c0037640 1 -- 172.21.15.59:0/2534060961 >> 172.21.15.59:0/2534060961 conn(0x7f44b80fd380 msgr2=0x7f44b80ff7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 -- 172.21.15.59:0/2534060961 shutdown_connections 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 -- 172.21.15.59:0/2534060961 wait complete. 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 Processor -- start 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 -- start start 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8103b60 0x7f44b819b410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8104480 0x7f44b819b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44b810cf80 0x7f44b819fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8112c30 con 0x7f44b8103b60 2024-05-08T05:38:01.188 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8112ab0 con 0x7f44b8104480 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.185+0000 7f44c0037640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f44b8112db0 con 0x7f44b810cf80 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bddd3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8103b60 0x7f44b819b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bddd3640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8103b60 0x7f44b819b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35658/0 (socket says 172.21.15.59:35658) 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bddd3640 1 -- 172.21.15.59:0/1736402404 learned_addr learned my addr 172.21.15.59:0/1736402404 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44be5d4640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44b810cf80 0x7f44b819fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8104480 0x7f44b819b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bddd3640 1 -- 172.21.15.59:0/1736402404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44b810cf80 msgr2=0x7f44b819fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bddd3640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44b810cf80 0x7f44b819fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bddd3640 1 -- 172.21.15.59:0/1736402404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8104480 msgr2=0x7f44b819b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bddd3640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8104480 0x7f44b819b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bddd3640 1 -- 172.21.15.59:0/1736402404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b81a03b0 con 0x7f44b8103b60 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44be5d4640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44b810cf80 0x7f44b819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:38:01.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44bddd3640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8103b60 0x7f44b819b410 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f44b400ea60 tx=0x7f44b400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44aeffd640 1 -- 172.21.15.59:0/1736402404 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b400cde0 con 0x7f44b8103b60 2024-05-08T05:38:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44c0037640 1 -- 172.21.15.59:0/1736402404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b81a06a0 con 0x7f44b8103b60 2024-05-08T05:38:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44aeffd640 1 -- 172.21.15.59:0/1736402404 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b40035f0 con 0x7f44b8103b60 2024-05-08T05:38:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.186+0000 7f44aeffd640 1 -- 172.21.15.59:0/1736402404 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b4010690 con 0x7f44b8103b60 2024-05-08T05:38:01.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.187+0000 7f44c0037640 1 -- 172.21.15.59:0/1736402404 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b81a7f30 con 0x7f44b8103b60 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.189+0000 7f44c2e2e740 1 Processor -- start 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.189+0000 7f44c2e2e740 1 -- start start 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.189+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 0x5604e4c052f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.189+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 0x5604e4c0fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.189+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c100f0 0x5604e4c124e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5604e4b6a5d0 con 0x5604e4c05830 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5604e4b23890 con 0x5604e4c04f10 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5604e4b66590 con 0x5604e4c100f0 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44bd5d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c100f0 0x5604e4c124e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44bd5d2640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c100f0 0x5604e4c124e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:51620/0 (socket says 172.21.15.59:51620) 2024-05-08T05:38:01.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44bd5d2640 1 -- 172.21.15.59:0/853326362 learned_addr learned my addr 172.21.15.59:0/853326362 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44be5d4640 1 --2- 172.21.15.59:0/853326362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 0x5604e4c052f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44bd5d2640 1 -- 172.21.15.59:0/853326362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 msgr2=0x5604e4c052f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/853326362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 0x5604e4c052f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44bd5d2640 1 -- 172.21.15.59:0/853326362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 msgr2=0x5604e4c0fbb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/853326362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 0x5604e4c0fbb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44bd5d2640 1 -- 172.21.15.59:0/853326362 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e487c210 con 0x5604e4c100f0 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44aeffd640 1 -- 172.21.15.59:0/1736402404 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f44b40037a0 con 0x7f44b8103b60 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.190+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/853326362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c100f0 0x5604e4c124e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009900 tx=0x7f44a8032b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38262f64c54b1f17 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.191+0000 7f44a3fff640 1 -- 172.21.15.59:0/853326362 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8040070 con 0x5604e4c100f0 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.191+0000 7f44a3fff640 1 -- 172.21.15.59:0/853326362 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a8004030 con 0x5604e4c100f0 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.191+0000 7f44aeffd640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f44a40b2a10 0x7f44a40b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.191+0000 7f44a3fff640 1 -- 172.21.15.59:0/853326362 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8035040 con 0x5604e4c100f0 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.191+0000 7f44be5d4640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f44a40b2a10 0x7f44a40b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.194 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44aeffd640 1 -- 172.21.15.59:0/1736402404 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f44b40e2c90 con 0x7f44b8103b60 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 -- 172.21.15.59:0/853326362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c100f0 msgr2=0x5604e4c124e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/853326362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c100f0 0x5604e4c124e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f44a8009900 tx=0x7f44a8032b20 comp rx=0 tx=0).stop 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 -- 172.21.15.59:0/853326362 shutdown_connections 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/853326362 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c100f0 0x5604e4c124e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/853326362 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 0x5604e4c0fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/853326362 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 0x5604e4c052f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 -- 172.21.15.59:0/853326362 >> 172.21.15.59:0/853326362 conn(0x5604e4c01420 msgr2=0x5604e4c01800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 -- 172.21.15.59:0/853326362 shutdown_connections 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 -- 172.21.15.59:0/853326362 wait complete. 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44be5d4640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f44a40b2a10 0x7f44a40b4ed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009fd0 tx=0x7f44b0009500 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.192+0000 7f44c2e2e740 1 -- start start 2024-05-08T05:38:01.195 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44c2e2e740 1 Processor -- start 2024-05-08T05:38:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c0cf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c05830 0x5604e4c0d470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.199 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5604e4c12cc0 con 0x5604e4c100f0 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5604e4b27370 con 0x5604e4c05830 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5604e4c12e40 con 0x5604e4c04f10 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44bddd3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c0cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44bddd3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c0cf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:51622/0 (socket says 172.21.15.59:51622) 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44bddd3640 1 -- 172.21.15.59:0/301662046 learned_addr learned my addr 172.21.15.59:0/301662046 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44be5d4640 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44c2e2e740 10 client.0 _ll_register_callbacks cb 0x5604e487ba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.193+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c05830 0x5604e4c0d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44bddd3640 1 -- 172.21.15.59:0/301662046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c05830 msgr2=0x5604e4c0d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44bddd3640 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c05830 0x5604e4c0d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44bddd3640 1 -- 172.21.15.59:0/301662046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 msgr2=0x5604e4c0d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44bddd3640 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.200 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44bddd3640 1 -- 172.21.15.59:0/301662046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e4948620 con 0x5604e4c04f10 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c05830 0x5604e4c0d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44bddd3640 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c0cf30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44b40e0610 tx=0x7f44b40a9310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ba7e91fcad4d769f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44a17fa640 1 -- 172.21.15.59:0/301662046 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40aa2a0 con 0x5604e4c04f10 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44a17fa640 1 -- 172.21.15.59:0/301662046 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f44b40c41b0 con 0x5604e4c04f10 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44c2e2e740 1 -- 172.21.15.59:0/301662046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a8004030 con 0x5604e4c04f10 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44a17fa640 1 -- 172.21.15.59:0/301662046 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f44b40a9ba0 con 0x5604e4c04f10 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44c2e2e740 10 client.25963 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.194+0000 7f44c2e2e740 1 -- 172.21.15.59:0/301662046 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5604e4c1eb00 con 0x5604e4c04f10 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.195+0000 7f44a17fa640 1 -- 172.21.15.59:0/301662046 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b40cae50 con 0x5604e4c04f10 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.195+0000 7f44a17fa640 1 -- 172.21.15.59:0/301662046 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f44b4004190 con 0x5604e4c04f10 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.195+0000 7f44c2e2e740 10 client.25963 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.195+0000 7f44c2e2e740 10 client.25963 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:01.201 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.195+0000 7f44c2e2e740 20 client.25963 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.195+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5604e4c1ffc0 0x5604e4c223b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.195+0000 7f44c2e2e740 4 client.25963 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.195+0000 7f44c2e2e740 1 -- 172.21.15.59:0/301662046 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f44a8004030 con 0x5604e4c1ffc0 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.195+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5604e4c1ffc0 0x5604e4c223b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.196+0000 7f44a17fa640 10 client.25963 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.196+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5604e4c1ffc0 0x5604e4c223b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8deb6cc07ee164 server_cookie=abedadced2a7e862 in_seq=0 out_seq=0 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.196+0000 7f44a17fa640 1 -- 172.21.15.59:0/301662046 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f44a8004030 con 0x5604e4c1ffc0 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.196+0000 7f44a17fa640 10 client.25963 handle_command_reply: tid=0 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.196+0000 7f44c2e2e740 1 client.25963 shutdown 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 -- 172.21.15.59:0/301662046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 msgr2=0x5604e4c0cf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c0cf30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f44b40e0610 tx=0x7f44b40a9310 comp rx=0 tx=0).stop 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 -- 172.21.15.59:0/301662046 shutdown_connections 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5604e4c1ffc0 0x5604e4c223b0 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.202 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44a17fa640 0 client.25963 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c05830 0x5604e4c0d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/301662046 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c0cf30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 -- 172.21.15.59:0/301662046 >> 172.21.15.59:0/301662046 conn(0x5604e4c01420 msgr2=0x5604e4c071a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 -- 172.21.15.59:0/301662046 shutdown_connections 2024-05-08T05:38:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 1 -- 172.21.15.59:0/301662046 wait complete. 2024-05-08T05:38:01.203 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.197+0000 7f44c2e2e740 20 client.25963 trim_cache size 0 max 16384 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.213+0000 7f44c2e2e740 1 Processor -- start 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.213+0000 7f44c2e2e740 1 -- start start 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.213+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 0x5604e4c0c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.213+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c05830 0x5604e4c078d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.213+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.213+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b2050 con 0x5604e4c100f0 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.213+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b3b90 con 0x5604e4c04f10 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.213+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b2720 con 0x5604e4c05830 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44be5d4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44be5d4640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35680/0 (socket says 172.21.15.59:35680) 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44be5d4640 1 -- 172.21.15.59:0/1516633734 learned_addr learned my addr 172.21.15.59:0/1516633734 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c05830 0x5604e4c078d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.216 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44bddd3640 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 0x5604e4c0c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44be5d4640 1 -- 172.21.15.59:0/1516633734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c05830 msgr2=0x5604e4c078d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44be5d4640 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c05830 0x5604e4c078d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44be5d4640 1 -- 172.21.15.59:0/1516633734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 msgr2=0x5604e4c0c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44be5d4640 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 0x5604e4c0c090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44be5d4640 1 -- 172.21.15.59:0/1516633734 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604e4c22720 con 0x5604e4c100f0 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c05830 0x5604e4c078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44be5d4640 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d7d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f44b0003de0 tx=0x7f44b001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=585d299d373b0144 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44a37fe640 1 -- 172.21.15.59:0/1516633734 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b000fda0 con 0x5604e4c100f0 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44a37fe640 1 -- 172.21.15.59:0/1516633734 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b0012020 con 0x5604e4c100f0 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.214+0000 7f44a37fe640 1 -- 172.21.15.59:0/1516633734 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b0005280 con 0x5604e4c100f0 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.215+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1516633734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 msgr2=0x5604e4c0d7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.215+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d7d0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f44b0003de0 tx=0x7f44b001b000 comp rx=0 tx=0).stop 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.215+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1516633734 shutdown_connections 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.215+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c100f0 0x5604e4c0d7d0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.217 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.215+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c05830 0x5604e4c078d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.215+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1516633734 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c04f10 0x5604e4c0c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.215+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1516633734 >> 172.21.15.59:0/1516633734 conn(0x5604e4c01420 msgr2=0x5604e4bffcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.215+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1516633734 shutdown_connections 2024-05-08T05:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.215+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1516633734 wait complete. 2024-05-08T05:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.216+0000 7f44c2e2e740 1 -- start start 2024-05-08T05:38:01.218 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.216+0000 7f44c2e2e740 1 Processor -- start 2024-05-08T05:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.216+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c181e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.216+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 0x5604e4c18720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.216+0000 7f44c2e2e740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c100f0 0x5604e4c13770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.216+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b34c0 con 0x5604e4c05830 2024-05-08T05:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.216+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b2df0 con 0x5604e4c100f0 2024-05-08T05:38:01.221 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.216+0000 7f44c2e2e740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f44b81b0050 con 0x5604e4c04f10 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44c2e2e740 10 client.0 _ll_register_callbacks cb 0x7f44b4004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bd5d2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 0x5604e4c18720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bddd3640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c181e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bd5d2640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 0x5604e4c18720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:35696/0 (socket says 172.21.15.59:35696) 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bd5d2640 1 -- 172.21.15.59:0/1974112517 learned_addr learned my addr 172.21.15.59:0/1974112517 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bddd3640 1 -- 172.21.15.59:0/1974112517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c100f0 msgr2=0x5604e4c13770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bddd3640 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c100f0 0x5604e4c13770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bddd3640 1 -- 172.21.15.59:0/1974112517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 msgr2=0x5604e4c18720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bddd3640 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 0x5604e4c18720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bddd3640 1 -- 172.21.15.59:0/1974112517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604e4c13fa0 con 0x5604e4c04f10 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 0x5604e4c18720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44bddd3640 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c181e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009920 tx=0x7f44b4004440 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66385007dd3ca9c3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:01.222 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44a17fa640 1 -- 172.21.15.59:0/1974112517 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44b40b2340 con 0x5604e4c04f10 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44a17fa640 1 -- 172.21.15.59:0/1974112517 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f44b40a8e00 con 0x5604e4c04f10 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.217+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1974112517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44b0012020 con 0x5604e4c04f10 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44a17fa640 1 -- 172.21.15.59:0/1974112517 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f44b40aa2a0 con 0x5604e4c04f10 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44c2e2e740 10 client.25969 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1974112517 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5604e4c8a3f0 con 0x5604e4c04f10 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44a17fa640 1 -- 172.21.15.59:0/1974112517 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44b0012020 con 0x5604e4c04f10 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44a17fa640 1 -- 172.21.15.59:0/1974112517 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f44b40af0e0 con 0x5604e4c04f10 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44c2e2e740 10 client.25969 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44c2e2e740 10 client.25969 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44c2e2e740 20 client.25969 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5604e4c8ab70 0x5604e4c8cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44c2e2e740 4 client.25969 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1974112517 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5604e4b781e0 con 0x5604e4c8ab70 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.218+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5604e4c8ab70 0x5604e4c8cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:01.223 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.219+0000 7f44bd5d2640 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5604e4c8ab70 0x5604e4c8cf90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f65bd4bc346bc2d4 server_cookie=11001432f45c42d5 in_seq=0 out_seq=0 2024-05-08T05:38:01.224 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.219+0000 7f44a17fa640 10 client.25969 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44a17fa640 1 -- 172.21.15.59:0/1974112517 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5604e4b781e0 con 0x5604e4c8ab70 2024-05-08T05:38:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44a17fa640 10 client.25969 handle_command_reply: tid=0 2024-05-08T05:38:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 client.25969 shutdown 2024-05-08T05:38:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1974112517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 msgr2=0x5604e4c181e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c181e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f44b4009920 tx=0x7f44b4004440 comp rx=0 tx=0).stop 2024-05-08T05:38:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1974112517 shutdown_connections 2024-05-08T05:38:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5604e4c8ab70 0x5604e4c8cf90 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.241 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5604e4c100f0 0x5604e4c13770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5604e4c05830 0x5604e4c18720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44a17fa640 0 client.25969 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 --2- 172.21.15.59:0/1974112517 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5604e4c04f10 0x5604e4c181e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1974112517 >> 172.21.15.59:0/1974112517 conn(0x5604e4c01420 msgr2=0x5604e4c00580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1974112517 shutdown_connections 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 1 -- 172.21.15.59:0/1974112517 wait complete. 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.240+0000 7f44c2e2e740 20 client.25969 trim_cache size 0 max 16384 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:38:01.242 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 -- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f44a40b2a10 msgr2=0x7f44a40b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f44a40b2a10 0x7f44a40b4ed0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f44b0009fd0 tx=0x7f44b0009500 comp rx=0 tx=0).stop 2024-05-08T05:38:01.243 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 -- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8103b60 msgr2=0x7f44b819b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8103b60 0x7f44b819b410 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f44b400ea60 tx=0x7f44b400ef20 comp rx=0 tx=0).stop 2024-05-08T05:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 -- 172.21.15.59:0/1736402404 shutdown_connections 2024-05-08T05:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f44a40b2a10 0x7f44a40b4ed0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f44b810cf80 0x7f44b819fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f44b8104480 0x7f44b819b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 --2- 172.21.15.59:0/1736402404 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f44b8103b60 0x7f44b819b410 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 -- 172.21.15.59:0/1736402404 >> 172.21.15.59:0/1736402404 conn(0x7f44b80fd380 msgr2=0x7f44b8102880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 -- 172.21.15.59:0/1736402404 shutdown_connections 2024-05-08T05:38:01.244 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:01.241+0000 7f44c0037640 1 -- 172.21.15.59:0/1736402404 wait complete. 2024-05-08T05:38:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:01 smithi143 ceph-mon[34337]: pgmap v261: 129 pgs: 129 active+clean; 94 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 715 op/s 2024-05-08T05:38:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:01 smithi096 ceph-mon[34253]: pgmap v261: 129 pgs: 129 active+clean; 94 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 715 op/s 2024-05-08T05:38:01.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:01 smithi059 ceph-mon[30304]: pgmap v261: 129 pgs: 129 active+clean; 94 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 15 MiB/s rd, 2.1 MiB/s wr, 715 op/s 2024-05-08T05:38:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:03 smithi143 ceph-mon[34337]: pgmap v262: 129 pgs: 129 active+clean; 94 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 478 op/s 2024-05-08T05:38:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:03 smithi143 ceph-mon[34337]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T05:38:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:03 smithi096 ceph-mon[34253]: pgmap v262: 129 pgs: 129 active+clean; 94 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 478 op/s 2024-05-08T05:38:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:03 smithi096 ceph-mon[34253]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T05:38:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:03 smithi059 ceph-mon[30304]: pgmap v262: 129 pgs: 129 active+clean; 94 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.4 MiB/s wr, 478 op/s 2024-05-08T05:38:03.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:03 smithi059 ceph-mon[30304]: osdmap e97: 12 total, 12 up, 12 in 2024-05-08T05:38:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:05 smithi143 ceph-mon[34337]: pgmap v264: 129 pgs: 129 active+clean; 94 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 573 op/s 2024-05-08T05:38:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:05 smithi096 ceph-mon[34253]: pgmap v264: 129 pgs: 129 active+clean; 94 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 573 op/s 2024-05-08T05:38:05.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:05 smithi059 ceph-mon[30304]: pgmap v264: 129 pgs: 129 active+clean; 94 MiB data, 2.2 GiB used, 1.0 TiB / 1.0 TiB avail; 13 MiB/s rd, 1.7 MiB/s wr, 573 op/s 2024-05-08T05:38:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:07 smithi143 ceph-mon[34337]: pgmap v265: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-05-08T05:38:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:07 smithi096 ceph-mon[34253]: pgmap v265: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-05-08T05:38:07.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:07 smithi059 ceph-mon[30304]: pgmap v265: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-05-08T05:38:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:09 smithi143 ceph-mon[34337]: pgmap v266: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-05-08T05:38:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:09 smithi096 ceph-mon[34253]: pgmap v266: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-05-08T05:38:09.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:09 smithi059 ceph-mon[30304]: pgmap v266: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.5 MiB/s wr, 500 op/s 2024-05-08T05:38:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:11 smithi143 ceph-mon[34337]: pgmap v267: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.5 MiB/s wr, 505 op/s 2024-05-08T05:38:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:11 smithi143 ceph-mon[34337]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T05:38:11.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:11 smithi096 ceph-mon[34253]: pgmap v267: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.5 MiB/s wr, 505 op/s 2024-05-08T05:38:11.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:11 smithi096 ceph-mon[34253]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T05:38:11.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:11 smithi059 ceph-mon[30304]: pgmap v267: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.5 MiB/s wr, 505 op/s 2024-05-08T05:38:11.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:11 smithi059 ceph-mon[30304]: osdmap e98: 12 total, 12 up, 12 in 2024-05-08T05:38:13.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:13 smithi143 ceph-mon[34337]: pgmap v269: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.5 MiB/s wr, 511 op/s 2024-05-08T05:38:13.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:13 smithi096 ceph-mon[34253]: pgmap v269: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.5 MiB/s wr, 511 op/s 2024-05-08T05:38:13.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:13 smithi059 ceph-mon[30304]: pgmap v269: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.5 MiB/s rd, 1.5 MiB/s wr, 511 op/s 2024-05-08T05:38:14.363 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.359+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1186219500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdaa810cff0 msgr2=0x7fdaa810f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.359+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1186219500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdaa810cff0 0x7fdaa810f440 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009ca0 tx=0x7fdaa002a950 comp rx=0 tx=0).stop 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1186219500 shutdown_connections 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1186219500 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdaa810cff0 0x7fdaa810f440 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1186219500 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdaa8104540 0x7fdaa810ca20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1186219500 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaa8103c20 0x7fdaa8104000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1186219500 >> 172.21.15.59:0/1186219500 conn(0x7fdaa8078ac0 msgr2=0x7fdaa8078ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1186219500 shutdown_connections 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1186219500 wait complete. 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 Processor -- start 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 -- start start 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdaa8103c20 0x7fdaa81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.360+0000 7fdaaf1f7640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdaa8104540 0x7fdaa8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.364 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaaf1f7640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaa810cff0 0x7fdaa819d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaaf1f7640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8112c50 con 0x7fdaa8104540 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaaf1f7640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8112ad0 con 0x7fdaa810cff0 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaaf1f7640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa8112dd0 con 0x7fdaa8103c20 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaacf93640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdaa8103c20 0x7fdaa81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaa7fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdaa8104540 0x7fdaa8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaacf93640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdaa8103c20 0x7fdaa81990f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:55848/0 (socket says 172.21.15.59:55848) 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaacf93640 1 -- 172.21.15.59:0/1484090134 learned_addr learned my addr 172.21.15.59:0/1484090134 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaad794640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaa810cff0 0x7fdaa819d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaacf93640 1 -- 172.21.15.59:0/1484090134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaa810cff0 msgr2=0x7fdaa819d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaacf93640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaa810cff0 0x7fdaa819d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaacf93640 1 -- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdaa8104540 msgr2=0x7fdaa8199630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaacf93640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdaa8104540 0x7fdaa8199630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaacf93640 1 -- 172.21.15.59:0/1484090134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaa819e0e0 con 0x7fdaa8103c20 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaa7fff640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdaa8104540 0x7fdaa8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:38:14.365 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaacf93640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdaa8103c20 0x7fdaa81990f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00e9f0 tx=0x7fda9c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.361+0000 7fdaa5ffb640 1 -- 172.21.15.59:0/1484090134 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c00cd80 con 0x7fdaa8103c20 2024-05-08T05:38:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.362+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1484090134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaa819e3d0 con 0x7fdaa8103c20 2024-05-08T05:38:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.362+0000 7fdaa5ffb640 1 -- 172.21.15.59:0/1484090134 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fda9c0108e0 con 0x7fdaa8103c20 2024-05-08T05:38:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.362+0000 7fdaa5ffb640 1 -- 172.21.15.59:0/1484090134 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fda9c018980 con 0x7fdaa8103c20 2024-05-08T05:38:14.366 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.362+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1484090134 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaa81a5c60 con 0x7fdaa8103c20 2024-05-08T05:38:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdab1feb740 1 Processor -- start 2024-05-08T05:38:14.371 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdab1feb740 1 -- start start 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdaa5ffb640 1 -- 172.21.15.59:0/1484090134 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fda9c010430 con 0x7fdaa8103c20 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527e5f10 0x5654527e62f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527f0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527f10f0 0x5654527f34e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdab1feb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x56545274b5d0 con 0x5654527e5f10 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdab1feb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x565452704890 con 0x5654527e6830 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdab1feb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x565452747590 con 0x5654527f10f0 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdaad794640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527e5f10 0x5654527e62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdaad794640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527e5f10 0x5654527e62f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45634/0 (socket says 172.21.15.59:45634) 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdaacf93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527f0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdaad794640 1 -- 172.21.15.59:0/205465546 learned_addr learned my addr 172.21.15.59:0/205465546 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.365+0000 7fdaa7fff640 1 --2- 172.21.15.59:0/205465546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527f10f0 0x5654527f34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.372 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaad794640 1 -- 172.21.15.59:0/205465546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527f10f0 msgr2=0x5654527f34e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaad794640 1 --2- 172.21.15.59:0/205465546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527f10f0 0x5654527f34e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaad794640 1 -- 172.21.15.59:0/205465546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 msgr2=0x5654527f0bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaad794640 1 --2- 172.21.15.59:0/205465546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527f0bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaad794640 1 -- 172.21.15.59:0/205465546 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56545245d210 con 0x5654527e5f10 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaacf93640 1 --2- 172.21.15.59:0/205465546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527f0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaa5ffb640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fda980b2a10 0x7fda980b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaad794640 1 --2- 172.21.15.59:0/205465546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527e5f10 0x5654527e62f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0035000 tx=0x7fdaa00093e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b40c478054396e3 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaacf93640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fda980b2a10 0x7fda980b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fdaa5ffb640 1 -- 172.21.15.59:0/1484090134 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fda9c0e2d10 con 0x7fdaa8103c20 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fda92ffd640 1 -- 172.21.15.59:0/205465546 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0048070 con 0x5654527e5f10 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fda92ffd640 1 -- 172.21.15.59:0/205465546 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa0003e20 con 0x5654527e5f10 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.366+0000 7fda92ffd640 1 -- 172.21.15.59:0/205465546 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0034040 con 0x5654527e5f10 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 -- 172.21.15.59:0/205465546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527e5f10 msgr2=0x5654527e62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdaacf93640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fda980b2a10 0x7fda980b4ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0e0140 tx=0x7fda9c0e6860 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:14.373 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 --2- 172.21.15.59:0/205465546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527e5f10 0x5654527e62f0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0035000 tx=0x7fdaa00093e0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 -- 172.21.15.59:0/205465546 shutdown_connections 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 --2- 172.21.15.59:0/205465546 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527f10f0 0x5654527f34e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 --2- 172.21.15.59:0/205465546 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527f0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 --2- 172.21.15.59:0/205465546 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527e5f10 0x5654527e62f0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 -- 172.21.15.59:0/205465546 >> 172.21.15.59:0/205465546 conn(0x5654527e2420 msgr2=0x5654527e2800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 -- 172.21.15.59:0/205465546 shutdown_connections 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 -- 172.21.15.59:0/205465546 wait complete. 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 -- start start 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.367+0000 7fdab1feb740 1 Processor -- start 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e5f10 0x5654527ee0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e6830 0x5654527ee5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x5654527eeb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdab1feb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5654527f3cc0 con 0x5654527f10f0 2024-05-08T05:38:14.374 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdab1feb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x565452708370 con 0x5654527e5f10 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdab1feb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5654527f3e40 con 0x5654527e6830 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdab1feb740 10 client.0 _ll_register_callbacks cb 0x56545245ca00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdaad794640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x5654527eeb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdaad794640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x5654527eeb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45640/0 (socket says 172.21.15.59:45640) 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdaad794640 1 -- 172.21.15.59:0/563583964 learned_addr learned my addr 172.21.15.59:0/563583964 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdaad794640 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e5f10 0x5654527ee0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdaad794640 1 -- 172.21.15.59:0/563583964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e6830 msgr2=0x5654527ee5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdaad794640 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e6830 0x5654527ee5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdaad794640 1 -- 172.21.15.59:0/563583964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e5f10 msgr2=0x5654527ee0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdaad794640 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e5f10 0x5654527ee0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.368+0000 7fdaad794640 1 -- 172.21.15.59:0/563583964 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565452529620 con 0x5654527f10f0 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.369+0000 7fdaad794640 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x5654527eeb20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009bf0 tx=0x7fdaa002f3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=145550f009148be2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.369+0000 7fda87fff640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.369+0000 7fda87fff640 1 -- 172.21.15.59:0/563583964 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0048030 con 0x5654527f10f0 2024-05-08T05:38:14.375 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.369+0000 7fda87fff640 1 -- 172.21.15.59:0/563583964 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fdaa0004040 con 0x5654527f10f0 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.369+0000 7fdab1feb740 1 -- 172.21.15.59:0/563583964 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa0003e20 con 0x5654527f10f0 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.369+0000 7fda87fff640 1 -- 172.21.15.59:0/563583964 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdaa005b050 con 0x5654527f10f0 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.369+0000 7fdab1feb740 10 client.16473 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.369+0000 7fdab1feb740 1 -- 172.21.15.59:0/563583964 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x5654527ffbb0 con 0x5654527f10f0 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.369+0000 7fda87fff640 1 -- 172.21.15.59:0/563583964 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa0049040 con 0x5654527f10f0 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fda87fff640 1 -- 172.21.15.59:0/563583964 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fdaa0052750 con 0x5654527f10f0 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fdab1feb740 10 client.16473 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fdab1feb740 10 client.16473 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fdab1feb740 20 client.16473 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fdab1feb740 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x565452800f00 0x5654528032f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fdab1feb740 4 client.16473 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fdab1feb740 1 -- 172.21.15.59:0/563583964 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fdaa0003e20 con 0x565452800f00 2024-05-08T05:38:14.376 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fdaa7fff640 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x565452800f00 0x5654528032f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.377 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fda87fff640 10 client.16473 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.370+0000 7fdaa7fff640 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x565452800f00 0x5654528032f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d189dac61cf35a3 server_cookie=844caddb7b23e733 in_seq=0 out_seq=0 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.371+0000 7fda87fff640 1 -- 172.21.15.59:0/563583964 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7fdaa0003e20 con 0x565452800f00 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.371+0000 7fda87fff640 10 client.16473 handle_command_reply: tid=0 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.371+0000 7fdab1feb740 1 client.16473 shutdown 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.371+0000 7fdab1feb740 1 -- 172.21.15.59:0/563583964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 msgr2=0x5654527eeb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.371+0000 7fdab1feb740 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x5654527eeb20 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009bf0 tx=0x7fdaa002f3a0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fdab1feb740 1 -- 172.21.15.59:0/563583964 shutdown_connections 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fdab1feb740 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x565452800f00 0x5654528032f0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fdab1feb740 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x5654527eeb20 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fdab1feb740 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e6830 0x5654527ee5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fdab1feb740 1 --2- 172.21.15.59:0/563583964 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e5f10 0x5654527ee0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fda87fff640 0 client.16473 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:14.378 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fdab1feb740 1 -- 172.21.15.59:0/563583964 >> 172.21.15.59:0/563583964 conn(0x5654527e2420 msgr2=0x5654527e81a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fdab1feb740 1 -- 172.21.15.59:0/563583964 shutdown_connections 2024-05-08T05:38:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fdab1feb740 1 -- 172.21.15.59:0/563583964 wait complete. 2024-05-08T05:38:14.379 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.372+0000 7fdab1feb740 20 client.16473 trim_cache size 0 max 16384 2024-05-08T05:38:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdab1feb740 1 Processor -- start 2024-05-08T05:38:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdab1feb740 1 -- start start 2024-05-08T05:38:14.390 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x5654527e8920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527fc7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452806310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdab1feb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81afce0 con 0x5654527f10f0 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdab1feb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81b1820 con 0x5654527e6830 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdab1feb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81b03b0 con 0x5654527e5f10 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdaad794640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452806310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdaa7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527fc7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.388+0000 7fdaad794640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452806310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45644/0 (socket says 172.21.15.59:45644) 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.389+0000 7fdaa7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527fc7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:57344/0 (socket says 172.21.15.59:57344) 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.389+0000 7fdaad794640 1 -- 172.21.15.59:0/4132075079 learned_addr learned my addr 172.21.15.59:0/4132075079 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.389+0000 7fdaad794640 1 --2- 172.21.15.59:0/4132075079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x5654527e8920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.389+0000 7fdaad794640 1 -- 172.21.15.59:0/4132075079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 msgr2=0x5654527e8920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.389+0000 7fdaad794640 1 --2- 172.21.15.59:0/4132075079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x5654527e8920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.389+0000 7fdaad794640 1 -- 172.21.15.59:0/4132075079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 msgr2=0x5654527fc7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:14.391 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.389+0000 7fdaad794640 1 --2- 172.21.15.59:0/4132075079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527fc7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.392 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.389+0000 7fdaad794640 1 -- 172.21.15.59:0/4132075079 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654527e2d60 con 0x5654527f10f0 2024-05-08T05:38:14.397 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.389+0000 7fdaa7fff640 1 --2- 172.21.15.59:0/4132075079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:38:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.396+0000 7fdaad794640 1 --2- 172.21.15.59:0/4132075079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452806310 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fdaa002f4f0 tx=0x7fdaa0000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cc08b0896cf93d0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.396+0000 7fda927fc640 1 -- 172.21.15.59:0/4132075079 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0048070 con 0x5654527f10f0 2024-05-08T05:38:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.396+0000 7fda927fc640 1 -- 172.21.15.59:0/4132075079 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa0009800 con 0x5654527f10f0 2024-05-08T05:38:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.396+0000 7fda927fc640 1 -- 172.21.15.59:0/4132075079 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0052910 con 0x5654527f10f0 2024-05-08T05:38:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.397+0000 7fdab1feb740 1 -- 172.21.15.59:0/4132075079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 msgr2=0x565452806310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:14.398 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.397+0000 7fdab1feb740 1 --2- 172.21.15.59:0/4132075079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452806310 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fdaa002f4f0 tx=0x7fdaa0000f40 comp rx=0 tx=0).stop 2024-05-08T05:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.402+0000 7fdab1feb740 1 -- 172.21.15.59:0/4132075079 shutdown_connections 2024-05-08T05:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.402+0000 7fdab1feb740 1 --2- 172.21.15.59:0/4132075079 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452806310 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.402+0000 7fdab1feb740 1 --2- 172.21.15.59:0/4132075079 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654527fc7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.402+0000 7fdab1feb740 1 --2- 172.21.15.59:0/4132075079 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x5654527e8920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.403 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.402+0000 7fdab1feb740 1 -- 172.21.15.59:0/4132075079 >> 172.21.15.59:0/4132075079 conn(0x5654527e2420 msgr2=0x5654527fc090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:14.406 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.405+0000 7fdab1feb740 1 -- 172.21.15.59:0/4132075079 shutdown_connections 2024-05-08T05:38:14.408 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.407+0000 7fdab1feb740 1 -- 172.21.15.59:0/4132075079 wait complete. 2024-05-08T05:38:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.408+0000 7fdab1feb740 1 -- start start 2024-05-08T05:38:14.409 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.408+0000 7fdab1feb740 1 Processor -- start 2024-05-08T05:38:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x565452806560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654528038f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdab1feb740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452803e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdab1feb740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81b1150 con 0x5654527f10f0 2024-05-08T05:38:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdab1feb740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81b0a80 con 0x5654527e6830 2024-05-08T05:38:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdab1feb740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdaa81adce0 con 0x5654527e5f10 2024-05-08T05:38:14.410 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdab1feb740 10 client.0 _ll_register_callbacks cb 0x7fdaa0046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdaa7fff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654528038f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdaad794640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x565452806560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdaad794640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x565452806560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:55884/0 (socket says 172.21.15.59:55884) 2024-05-08T05:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdaad794640 1 -- 172.21.15.59:0/193711254 learned_addr learned my addr 172.21.15.59:0/193711254 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.409+0000 7fdaad794640 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452803e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.411 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.410+0000 7fdaad794640 1 -- 172.21.15.59:0/193711254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 msgr2=0x5654528038f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.410+0000 7fdaad794640 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654528038f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.410+0000 7fdaad794640 1 -- 172.21.15.59:0/193711254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 msgr2=0x565452803e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.410+0000 7fdaad794640 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452803e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.410+0000 7fdaad794640 1 -- 172.21.15.59:0/193711254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565452804370 con 0x5654527e5f10 2024-05-08T05:38:14.412 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.411+0000 7fdaad794640 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x565452806560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009a60 tx=0x7fdaa0030000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ad9461819acc332c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.412+0000 7fda87fff640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.412+0000 7fda87fff640 1 -- 172.21.15.59:0/193711254 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0048040 con 0x5654527e5f10 2024-05-08T05:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.412+0000 7fdab1feb740 1 -- 172.21.15.59:0/193711254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdaa0009800 con 0x5654527e5f10 2024-05-08T05:38:14.414 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.412+0000 7fda87fff640 1 -- 172.21.15.59:0/193711254 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fdaa00528c0 con 0x5654527e5f10 2024-05-08T05:38:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.413+0000 7fda87fff640 1 -- 172.21.15.59:0/193711254 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fdaa006e050 con 0x5654527e5f10 2024-05-08T05:38:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.413+0000 7fdab1feb740 10 client.25984 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.413+0000 7fdab1feb740 1 -- 172.21.15.59:0/193711254 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x56545286b120 con 0x5654527e5f10 2024-05-08T05:38:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.413+0000 7fda87fff640 1 -- 172.21.15.59:0/193711254 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa0009800 con 0x5654527e5f10 2024-05-08T05:38:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.414+0000 7fda87fff640 1 -- 172.21.15.59:0/193711254 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fdaa0030600 con 0x5654527e5f10 2024-05-08T05:38:14.415 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.414+0000 7fdab1feb740 10 client.25984 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.414+0000 7fdab1feb740 10 client.25984 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.414+0000 7fdab1feb740 20 client.25984 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.414+0000 7fdab1feb740 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56545286b7d0 0x56545286dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.414+0000 7fdab1feb740 4 client.25984 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.414+0000 7fdab1feb740 1 -- 172.21.15.59:0/193711254 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5654527591e0 con 0x56545286b7d0 2024-05-08T05:38:14.416 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.415+0000 7fdaa7fff640 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56545286b7d0 0x56545286dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.415+0000 7fdaa7fff640 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56545286b7d0 0x56545286dbf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d77a7546f36ca90a server_cookie=383190b59b6171e7 in_seq=0 out_seq=0 2024-05-08T05:38:14.417 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.415+0000 7fda87fff640 10 client.25984 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.434+0000 7fda87fff640 1 -- 172.21.15.59:0/193711254 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5654527591e0 con 0x56545286b7d0 2024-05-08T05:38:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.434+0000 7fda87fff640 10 client.25984 handle_command_reply: tid=0 2024-05-08T05:38:14.436 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.434+0000 7fdab1feb740 1 client.25984 shutdown 2024-05-08T05:38:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.435+0000 7fdab1feb740 1 -- 172.21.15.59:0/193711254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 msgr2=0x565452806560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:14.437 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.436+0000 7fdab1feb740 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x565452806560 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009a60 tx=0x7fdaa0030000 comp rx=0 tx=0).stop 2024-05-08T05:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.436+0000 7fdab1feb740 1 -- 172.21.15.59:0/193711254 shutdown_connections 2024-05-08T05:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.436+0000 7fdab1feb740 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x56545286b7d0 0x56545286dbf0 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.436+0000 7fdab1feb740 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5654527f10f0 0x565452803e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.436+0000 7fdab1feb740 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5654527e6830 0x5654528038f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.436+0000 7fdab1feb740 1 --2- 172.21.15.59:0/193711254 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5654527e5f10 0x565452806560 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.438 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.436+0000 7fdab1feb740 1 -- 172.21.15.59:0/193711254 >> 172.21.15.59:0/193711254 conn(0x5654527e2420 msgr2=0x5654527ee8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.437+0000 7fda87fff640 0 client.25984 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:14.439 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.438+0000 7fdab1feb740 1 -- 172.21.15.59:0/193711254 shutdown_connections 2024-05-08T05:38:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.440+0000 7fdab1feb740 1 -- 172.21.15.59:0/193711254 wait complete. 2024-05-08T05:38:14.441 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.440+0000 7fdab1feb740 20 client.25984 trim_cache size 0 max 16384 2024-05-08T05:38:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:38:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:38:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:38:14.442 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:38:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.442+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fda980b2a10 msgr2=0x7fda980b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.442+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fda980b2a10 0x7fda980b4ed0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fda9c0e0140 tx=0x7fda9c0e6860 comp rx=0 tx=0).stop 2024-05-08T05:38:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.442+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1484090134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdaa8103c20 msgr2=0x7fdaa81990f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.442+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdaa8103c20 0x7fdaa81990f0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fda9c00e9f0 tx=0x7fda9c00eeb0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.444 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.443+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1484090134 shutdown_connections 2024-05-08T05:38:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.443+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fda980b2a10 0x7fda980b4ed0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.443+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdaa810cff0 0x7fdaa819d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.443+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdaa8104540 0x7fdaa8199630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.443+0000 7fdaaf1f7640 1 --2- 172.21.15.59:0/1484090134 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdaa8103c20 0x7fdaa81990f0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.443+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1484090134 >> 172.21.15.59:0/1484090134 conn(0x7fdaa8078ac0 msgr2=0x7fdaa80ffb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.443+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1484090134 shutdown_connections 2024-05-08T05:38:14.445 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:14.443+0000 7fdaaf1f7640 1 -- 172.21.15.59:0/1484090134 wait complete. 2024-05-08T05:38:15.080 INFO:tasks.quiescer.fs.[cephfs]:Total duration (60) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-05-08T05:38:15.080 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-08T05:38:15.081 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:38:15.451 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:38:15.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:15 smithi143 ceph-mon[34337]: pgmap v270: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.5 MiB/s wr, 504 op/s 2024-05-08T05:38:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:15 smithi096 ceph-mon[34253]: pgmap v270: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.5 MiB/s wr, 504 op/s 2024-05-08T05:38:15.584 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:15 smithi059 ceph-mon[30304]: pgmap v270: 129 pgs: 129 active+clean; 87 MiB data, 2.3 GiB used, 1.0 TiB / 1.0 TiB avail; 7.4 MiB/s rd, 1.5 MiB/s wr, 504 op/s 2024-05-08T05:38:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:17 smithi059 ceph-mon[30304]: pgmap v271: 129 pgs: 129 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.7 MiB/s wr, 554 op/s 2024-05-08T05:38:17.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:17 smithi143 ceph-mon[34337]: pgmap v271: 129 pgs: 129 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.7 MiB/s wr, 554 op/s 2024-05-08T05:38:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:17 smithi096 ceph-mon[34253]: pgmap v271: 129 pgs: 129 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 MiB/s rd, 1.7 MiB/s wr, 554 op/s 2024-05-08T05:38:18.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.091+0000 7fa18e768700 1 -- 172.21.15.59:0/145818924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 msgr2=0x7fa188110340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:18.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.091+0000 7fa18e768700 1 --2- 172.21.15.59:0/145818924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 0x7fa188110340 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa180009f90 tx=0x7fa18000b330 comp rx=0 tx=0).stop 2024-05-08T05:38:18.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.091+0000 7fa18e768700 1 -- 172.21.15.59:0/145818924 shutdown_connections 2024-05-08T05:38:18.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.091+0000 7fa18e768700 1 --2- 172.21.15.59:0/145818924 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 0x7fa188110340 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:18.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.091+0000 7fa18e768700 1 --2- 172.21.15.59:0/145818924 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa188109520 0x7fa188109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:18.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.091+0000 7fa18e768700 1 --2- 172.21.15.59:0/145818924 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa18806bb70 0x7fa18806bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:18.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.091+0000 7fa18e768700 1 -- 172.21.15.59:0/145818924 >> 172.21.15.59:0/145818924 conn(0x7fa188079350 msgr2=0x7fa188079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:18.224 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.091+0000 7fa18e768700 1 -- 172.21.15.59:0/145818924 shutdown_connections 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.091+0000 7fa18e768700 1 -- 172.21.15.59:0/145818924 wait complete. 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.092+0000 7fa18e768700 1 Processor -- start 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.092+0000 7fa18e768700 1 -- start start 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.092+0000 7fa18e768700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa18806bb70 0x7fa188102850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.092+0000 7fa18e768700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 0x7fa188102d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.092+0000 7fa18e768700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa188109520 0x7fa1881032d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.092+0000 7fa18e768700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa188113680 con 0x7fa18806bb70 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.092+0000 7fa18e768700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa188113500 con 0x7fa18806c530 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.092+0000 7fa18e768700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa188113800 con 0x7fa188109520 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa187fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa18806bb70 0x7fa188102850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 0x7fa188102d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa187fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa18806bb70 0x7fa188102850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45672/0 (socket says 172.21.15.59:45672) 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1877fe700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 0x7fa188102d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:57384/0 (socket says 172.21.15.59:57384) 2024-05-08T05:38:18.225 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa187fff700 1 -- 172.21.15.59:0/1015069476 learned_addr learned my addr 172.21.15.59:0/1015069476 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa18cd05700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa188109520 0x7fa1881032d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1877fe700 1 -- 172.21.15.59:0/1015069476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa188109520 msgr2=0x7fa1881032d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1877fe700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa188109520 0x7fa1881032d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1877fe700 1 -- 172.21.15.59:0/1015069476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa18806bb70 msgr2=0x7fa188102850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1877fe700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa18806bb70 0x7fa188102850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1877fe700 1 -- 172.21.15.59:0/1015069476 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1881a8060 con 0x7fa18806c530 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1877fe700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 0x7fa188102d90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00e9f0 tx=0x7fa17c00ed00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1857fa700 1 -- 172.21.15.59:0/1015069476 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c00c9c0 con 0x7fa18806c530 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1857fa700 1 -- 172.21.15.59:0/1015069476 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa17c00cb60 con 0x7fa18806c530 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1881a8350 con 0x7fa18806c530 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa1857fa700 1 -- 172.21.15.59:0/1015069476 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa17c018680 con 0x7fa18806c530 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.093+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1881a88e0 con 0x7fa18806c530 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.094+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa188104bf0 con 0x7fa18806c530 2024-05-08T05:38:18.226 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.096+0000 7fa1857fa700 1 -- 172.21.15.59:0/1015069476 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa17c010430 con 0x7fa18806c530 2024-05-08T05:38:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.097+0000 7fa1857fa700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1700b2a70 0x7fa1700b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.097+0000 7fa187fff700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1700b2a70 0x7fa1700b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.097+0000 7fa1857fa700 1 -- 172.21.15.59:0/1015069476 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fa17c0e2e40 con 0x7fa18806c530 2024-05-08T05:38:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.097+0000 7fa187fff700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1700b2a70 0x7fa1700b4f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa178009750 tx=0x7fa178009310 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:18.227 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.107+0000 7fa1857fa700 1 -- 172.21.15.59:0/1015069476 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa17c0aa450 con 0x7fa18806c530 2024-05-08T05:38:18.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.460+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa18802e510 con 0x7fa18806c530 2024-05-08T05:38:18.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.461+0000 7fa1857fa700 1 -- 172.21.15.59:0/1015069476 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11929 (secure 0 0 0) 0x7fa17c0a9d50 con 0x7fa18806c530 2024-05-08T05:38:18.462 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:38:18.462 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:36:43.047617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25876":{"gid":25876,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2351774265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2351774265},{"type":"v1","addr":"172.21.15.143:6835","nonce":2351774265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1700b2a70 msgr2=0x7fa1700b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1700b2a70 0x7fa1700b4f30 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fa178009750 tx=0x7fa178009310 comp rx=0 tx=0).stop 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 msgr2=0x7fa188102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 0x7fa188102d90 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fa17c00e9f0 tx=0x7fa17c00ed00 comp rx=0 tx=0).stop 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 shutdown_connections 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa1700b2a70 0x7fa1700b4f30 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa188109520 0x7fa1881032d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa18806bb70 0x7fa188102850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 --2- 172.21.15.59:0/1015069476 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa18806c530 0x7fa188102d90 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 >> 172.21.15.59:0/1015069476 conn(0x7fa188079350 msgr2=0x7fa188101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 shutdown_connections 2024-05-08T05:38:18.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:18.463+0000 7fa18e768700 1 -- 172.21.15.59:0/1015069476 wait complete. 2024-05-08T05:38:18.466 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 38 2024-05-08T05:38:19.320 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-05-08T05:38:19.320 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-05-08T05:38:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:19 smithi143 ceph-mon[34337]: pgmap v272: 129 pgs: 129 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.7 MiB/s wr, 554 op/s 2024-05-08T05:38:19.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:19 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1015069476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:38:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:19 smithi096 ceph-mon[34253]: pgmap v272: 129 pgs: 129 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.7 MiB/s wr, 554 op/s 2024-05-08T05:38:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:19 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1015069476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:38:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:19 smithi059 ceph-mon[30304]: pgmap v272: 129 pgs: 129 active+clean; 83 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.8 MiB/s rd, 1.7 MiB/s wr, 554 op/s 2024-05-08T05:38:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:19 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1015069476' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:38:19.715 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:38:21.471 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:21 smithi059 ceph-mon[30304]: pgmap v273: 129 pgs: 129 active+clean; 97 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 532 op/s 2024-05-08T05:38:21.506 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:21 smithi143 ceph-mon[34337]: pgmap v273: 129 pgs: 129 active+clean; 97 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 532 op/s 2024-05-08T05:38:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:21 smithi096 ceph-mon[34253]: pgmap v273: 129 pgs: 129 active+clean; 97 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 11 MiB/s rd, 1.6 MiB/s wr, 532 op/s 2024-05-08T05:38:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.782+0000 7fb250274700 1 -- 172.21.15.59:0/2370503862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb248108460 msgr2=0x7fb248108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.782+0000 7fb250274700 1 --2- 172.21.15.59:0/2370503862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb248108460 0x7fb248108840 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fb244009990 tx=0x7fb244009ca0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.782+0000 7fb250274700 1 -- 172.21.15.59:0/2370503862 shutdown_connections 2024-05-08T05:38:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.782+0000 7fb250274700 1 --2- 172.21.15.59:0/2370503862 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb248102450 0x7fb2481103b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.782+0000 7fb250274700 1 --2- 172.21.15.59:0/2370503862 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb248108460 0x7fb248108840 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.782+0000 7fb250274700 1 --2- 172.21.15.59:0/2370503862 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb248101a90 0x7fb248101f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.782+0000 7fb250274700 1 -- 172.21.15.59:0/2370503862 >> 172.21.15.59:0/2370503862 conn(0x7fb248078a40 msgr2=0x7fb2480ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.782+0000 7fb250274700 1 -- 172.21.15.59:0/2370503862 shutdown_connections 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.782+0000 7fb250274700 1 -- 172.21.15.59:0/2370503862 wait complete. 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.783+0000 7fb250274700 1 Processor -- start 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.783+0000 7fb250274700 1 -- start start 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.783+0000 7fb250274700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb248101a90 0x7fb24819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.783+0000 7fb250274700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb248102450 0x7fb24819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb250274700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb248108460 0x7fb248199920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb250274700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481136c0 con 0x7fb248108460 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24e010700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb248101a90 0x7fb24819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24d80f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb248102450 0x7fb24819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.787 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24e010700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb248101a90 0x7fb24819f710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:37186/0 (socket says 172.21.15.59:37186) 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24e010700 1 -- 172.21.15.59:0/4031790287 learned_addr learned my addr 172.21.15.59:0/4031790287 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb248113540 con 0x7fb248101a90 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb248113840 con 0x7fb248102450 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24e811700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb248108460 0x7fb248199920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24d80f700 1 -- 172.21.15.59:0/4031790287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb248101a90 msgr2=0x7fb24819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24d80f700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb248101a90 0x7fb24819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24d80f700 1 -- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb248108460 msgr2=0x7fb248199920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24d80f700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb248108460 0x7fb248199920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24d80f700 1 -- 172.21.15.59:0/4031790287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb24819a080 con 0x7fb248102450 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24e010700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb248101a90 0x7fb24819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.784+0000 7fb24d80f700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb248102450 0x7fb24819fc50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb23800e9f0 tx=0x7fb23800ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.785+0000 7fb23f7fe700 1 -- 172.21.15.59:0/4031790287 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb23800cb70 con 0x7fb248102450 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.785+0000 7fb23f7fe700 1 -- 172.21.15.59:0/4031790287 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb23800ee90 con 0x7fb248102450 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.785+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb24819a370 con 0x7fb248102450 2024-05-08T05:38:21.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.785+0000 7fb23f7fe700 1 -- 172.21.15.59:0/4031790287 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb238003d40 con 0x7fb248102450 2024-05-08T05:38:21.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.785+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb248102ba0 con 0x7fb248102450 2024-05-08T05:38:21.789 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.788+0000 7fb23f7fe700 1 -- 172.21.15.59:0/4031790287 <== mon.1 v2:172.21.15.143:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb238003ee0 con 0x7fb248102450 2024-05-08T05:38:21.790 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.789+0000 7fb23f7fe700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb2340b2b20 0x7fb2340b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.789+0000 7fb24e010700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb2340b2b20 0x7fb2340b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.789+0000 7fb23f7fe700 1 -- 172.21.15.59:0/4031790287 <== mon.1 v2:172.21.15.143:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fb2380e2ac0 con 0x7fb248102450 2024-05-08T05:38:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.789+0000 7fb25ab07b80 1 Processor -- start 2024-05-08T05:38:21.791 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb25ab07b80 1 -- start start 2024-05-08T05:38:21.792 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e010700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb2340b2b20 0x7fb2340b4fe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb24400b2e0 tx=0x7fb24401e040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92414c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9240ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d9241b610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb25ab07b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563d92360ad0 con 0x563d92414850 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb25ab07b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d922c5f30 con 0x563d9240f4c0 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb25ab07b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563d9235ca90 con 0x563d9240eb00 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e010700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92414c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e811700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d9241b610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e010700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92414c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33936/0 (socket says 172.21.15.59:33936) 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e010700 1 -- 172.21.15.59:0/1882584365 learned_addr learned my addr 172.21.15.59:0/1882584365 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24d80f700 1 --2- 172.21.15.59:0/1882584365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9240ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e811700 1 -- 172.21.15.59:0/1882584365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 msgr2=0x563d9240ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:21.793 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e811700 1 --2- 172.21.15.59:0/1882584365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9240ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e811700 1 -- 172.21.15.59:0/1882584365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 msgr2=0x563d92414c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e811700 1 --2- 172.21.15.59:0/1882584365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92414c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.790+0000 7fb24e811700 1 -- 172.21.15.59:0/1882584365 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d922ada60 con 0x563d9240f4c0 2024-05-08T05:38:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.791+0000 7fb24e811700 1 --2- 172.21.15.59:0/1882584365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d9241b610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb240011550 tx=0x7fb240009920 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f9b1d49dad3821ef server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.791+0000 7fb23cd70700 1 -- 172.21.15.59:0/1882584365 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb240004bb0 con 0x563d9240f4c0 2024-05-08T05:38:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.791+0000 7fb23cd70700 1 -- 172.21.15.59:0/1882584365 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb240004d50 con 0x563d9240f4c0 2024-05-08T05:38:21.794 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.791+0000 7fb23cd70700 1 -- 172.21.15.59:0/1882584365 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400046a0 con 0x563d9240f4c0 2024-05-08T05:38:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.792+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1882584365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 msgr2=0x563d9241b610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.792+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1882584365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d9241b610 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb240011550 tx=0x7fb240009920 comp rx=0 tx=0).stop 2024-05-08T05:38:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.792+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1882584365 shutdown_connections 2024-05-08T05:38:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.792+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1882584365 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d9241b610 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.792+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1882584365 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92414c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.792+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1882584365 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9240ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.792+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1882584365 >> 172.21.15.59:0/1882584365 conn(0x563d9233ad00 msgr2=0x563d92409880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.792+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1882584365 shutdown_connections 2024-05-08T05:38:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.792+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1882584365 wait complete. 2024-05-08T05:38:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.794+0000 7fb25ab07b80 1 -- start start 2024-05-08T05:38:21.796 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.794+0000 7fb25ab07b80 1 Processor -- start 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.794+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9241b170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.794+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d92418510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.795+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92418a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.795+0000 7fb25ab07b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x563d9241bb50 con 0x563d92414850 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.795+0000 7fb25ab07b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x563d9231a060 con 0x563d9240f4c0 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.795+0000 7fb25ab07b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x563d9241bcd0 con 0x563d9240eb00 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.795+0000 7fb25ab07b80 10 client.0 _ll_register_callbacks cb 0x563d922f4740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.795+0000 7fb24e811700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92418a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.795+0000 7fb24e811700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92418a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33950/0 (socket says 172.21.15.59:33950) 2024-05-08T05:38:21.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.795+0000 7fb24e811700 1 -- 172.21.15.59:0/2499555840 learned_addr learned my addr 172.21.15.59:0/2499555840 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:21.798 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb24d80f700 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d92418510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb24e811700 1 -- 172.21.15.59:0/2499555840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 msgr2=0x563d9241b170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb24e811700 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9241b170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb24e811700 1 -- 172.21.15.59:0/2499555840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 msgr2=0x563d92418510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb24e811700 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d92418510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb24e811700 1 -- 172.21.15.59:0/2499555840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d923178f0 con 0x563d92414850 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb24d80f700 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d92418510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb24e811700 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92418a50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb240004240 tx=0x7fb240006040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bae80382c101302 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb231ffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.796+0000 7fb231ffb700 1 -- 172.21.15.59:0/2499555840 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2400044a0 con 0x563d92414850 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.797+0000 7fb231ffb700 1 -- 172.21.15.59:0/2499555840 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fb240004640 con 0x563d92414850 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.797+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2499555840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb240004d50 con 0x563d92414850 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.797+0000 7fb231ffb700 1 -- 172.21.15.59:0/2499555840 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb240004880 con 0x563d92414850 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.797+0000 7fb25ab07b80 10 client.16497 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:21.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.797+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2499555840 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563d924261f0 con 0x563d92414850 2024-05-08T05:38:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.798+0000 7fb231ffb700 1 -- 172.21.15.59:0/2499555840 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb240004d50 con 0x563d92414850 2024-05-08T05:38:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.798+0000 7fb231ffb700 1 -- 172.21.15.59:0/2499555840 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb240025070 con 0x563d92414850 2024-05-08T05:38:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.798+0000 7fb25ab07b80 10 client.16497 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.798+0000 7fb25ab07b80 10 client.16497 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:38:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.798+0000 7fb25ab07b80 20 client.16497 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.798+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563d92428720 0x563d9242ab10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.798+0000 7fb25ab07b80 4 client.16497 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:38:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.798+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2499555840 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d9236e4b0 con 0x563d92428720 2024-05-08T05:38:21.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.798+0000 7fb24e010700 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563d92428720 0x563d9242ab10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.799+0000 7fb231ffb700 10 client.16497 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:21.801 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.800+0000 7fb24e010700 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563d92428720 0x563d9242ab10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7033aea8e152b330 server_cookie=3c2997f8accb19fd in_seq=0 out_seq=0 2024-05-08T05:38:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb231ffb700 1 -- 172.21.15.59:0/2499555840 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x563d9236e4b0 con 0x563d92428720 2024-05-08T05:38:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb231ffb700 10 client.16497 handle_command_reply: tid=0 2024-05-08T05:38:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb25ab07b80 1 client.16497 shutdown 2024-05-08T05:38:21.803 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2499555840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 msgr2=0x563d92418a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92418a50 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fb240004240 tx=0x7fb240006040 comp rx=0 tx=0).stop 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2499555840 shutdown_connections 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563d92428720 0x563d9242ab10 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb231ffb700 0 client.16497 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d92414850 0x563d92418a50 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9241b170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2499555840 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240f4c0 0x563d92418510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.801+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2499555840 >> 172.21.15.59:0/2499555840 conn(0x563d9233ad00 msgr2=0x563d92413bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.802+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2499555840 shutdown_connections 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.802+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2499555840 wait complete. 2024-05-08T05:38:21.804 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.802+0000 7fb25ab07b80 20 client.16497 trim_cache size 0 max 16384 2024-05-08T05:38:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.832+0000 7fb25ab07b80 1 Processor -- start 2024-05-08T05:38:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.832+0000 7fb25ab07b80 1 -- start start 2024-05-08T05:38:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240eb00 0x563d92403630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 0x563d9240daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d92414850 0x563d9240e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb24d80f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 0x563d9240daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb24e010700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240eb00 0x563d92403630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.835 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb24e010700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240eb00 0x563d92403630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:37206/0 (socket says 172.21.15.59:37206) 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb24e010700 1 -- 172.21.15.59:0/2827089811 learned_addr learned my addr 172.21.15.59:0/2827089811 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2827089811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481ad2c0 con 0x563d9240f4c0 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2827089811 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481aee00 con 0x563d9240eb00 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2827089811 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481ad990 con 0x563d92414850 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb24e811700 1 --2- 172.21.15.59:0/2827089811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d92414850 0x563d9240e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb24d80f700 1 -- 172.21.15.59:0/2827089811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d92414850 msgr2=0x563d9240e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.833+0000 7fb24d80f700 1 --2- 172.21.15.59:0/2827089811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d92414850 0x563d9240e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.834+0000 7fb24d80f700 1 -- 172.21.15.59:0/2827089811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240eb00 msgr2=0x563d92403630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.834+0000 7fb24d80f700 1 --2- 172.21.15.59:0/2827089811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240eb00 0x563d92403630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.834+0000 7fb24d80f700 1 -- 172.21.15.59:0/2827089811 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d924182e0 con 0x563d9240f4c0 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.834+0000 7fb24e010700 1 --2- 172.21.15.59:0/2827089811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240eb00 0x563d92403630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:38:21.836 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.834+0000 7fb24d80f700 1 --2- 172.21.15.59:0/2827089811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 0x563d9240daf0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb2380dfde0 tx=0x7fb2380a9210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36142e0c75666fe2 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:21.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.834+0000 7fb233fff700 1 -- 172.21.15.59:0/2827089811 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2380aa070 con 0x563d9240f4c0 2024-05-08T05:38:21.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.834+0000 7fb233fff700 1 -- 172.21.15.59:0/2827089811 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2380094d0 con 0x563d9240f4c0 2024-05-08T05:38:21.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.834+0000 7fb233fff700 1 -- 172.21.15.59:0/2827089811 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2380a9bd0 con 0x563d9240f4c0 2024-05-08T05:38:21.838 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.835+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2827089811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 msgr2=0x563d9240daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:21.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.835+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2827089811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 0x563d9240daf0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb2380dfde0 tx=0x7fb2380a9210 comp rx=0 tx=0).stop 2024-05-08T05:38:21.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.838+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2827089811 shutdown_connections 2024-05-08T05:38:21.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.838+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2827089811 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d92414850 0x563d9240e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.839 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.838+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2827089811 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d9240eb00 0x563d92403630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.838+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/2827089811 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 0x563d9240daf0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.838+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2827089811 >> 172.21.15.59:0/2827089811 conn(0x563d9233ad00 msgr2=0x563d92402f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:21.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.841+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2827089811 shutdown_connections 2024-05-08T05:38:21.842 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.841+0000 7fb25ab07b80 1 -- 172.21.15.59:0/2827089811 wait complete. 2024-05-08T05:38:21.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.842+0000 7fb25ab07b80 1 -- start start 2024-05-08T05:38:21.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.842+0000 7fb25ab07b80 1 Processor -- start 2024-05-08T05:38:21.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9242fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 0x563d92430350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb25ab07b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d92414850 0x563d924308b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb25ab07b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481ae730 con 0x563d9240f4c0 2024-05-08T05:38:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb25ab07b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481ae060 con 0x563d92414850 2024-05-08T05:38:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb25ab07b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb2481ab280 con 0x563d9240eb00 2024-05-08T05:38:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb25ab07b80 10 client.0 _ll_register_callbacks cb 0x563d922f4740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb24e010700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9242fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb24e010700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9242fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:55228/0 (socket says 172.21.15.59:55228) 2024-05-08T05:38:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb24e010700 1 -- 172.21.15.59:0/1009549274 learned_addr learned my addr 172.21.15.59:0/1009549274 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:21.845 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.843+0000 7fb24e811700 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d92414850 0x563d924308b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb24e010700 1 -- 172.21.15.59:0/1009549274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d92414850 msgr2=0x563d924308b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:21.846 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb24e010700 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d92414850 0x563d924308b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb24e010700 1 -- 172.21.15.59:0/1009549274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 msgr2=0x563d92430350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb24e010700 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 0x563d92430350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb24e010700 1 -- 172.21.15.59:0/1009549274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d92492190 con 0x563d9240eb00 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb24e811700 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d92414850 0x563d924308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb24e010700 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9242fe10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb2440056a0 tx=0x7fb244009ca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=509a9846e89db2cd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb231ffb700 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb231ffb700 1 -- 172.21.15.59:0/1009549274 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb24403a070 con 0x563d9240eb00 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb231ffb700 1 -- 172.21.15.59:0/1009549274 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fb244036900 con 0x563d9240eb00 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.844+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1009549274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb2380094d0 con 0x563d9240eb00 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.845+0000 7fb231ffb700 1 -- 172.21.15.59:0/1009549274 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fb244044050 con 0x563d9240eb00 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.845+0000 7fb25ab07b80 10 client.26005 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:21.847 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.845+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1009549274 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x563d924934b0 con 0x563d9240eb00 2024-05-08T05:38:21.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.845+0000 7fb231ffb700 1 -- 172.21.15.59:0/1009549274 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb2380094d0 con 0x563d9240eb00 2024-05-08T05:38:21.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.846+0000 7fb231ffb700 1 -- 172.21.15.59:0/1009549274 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fb244003980 con 0x563d9240eb00 2024-05-08T05:38:21.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.846+0000 7fb25ab07b80 10 client.26005 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:21.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.846+0000 7fb25ab07b80 10 client.26005 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:38:21.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.846+0000 7fb25ab07b80 20 client.26005 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:21.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.846+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563d92428720 0x563d92495c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:21.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.846+0000 7fb25ab07b80 4 client.26005 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-05-08T05:38:21.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.846+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1009549274 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x563d92368550 con 0x563d92428720 2024-05-08T05:38:21.849 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.846+0000 7fb24d80f700 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563d92428720 0x563d92495c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:21.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.852+0000 7fb24d80f700 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563d92428720 0x563d92495c90 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a8f57d7e765461c server_cookie=289519a8073ea6a in_seq=0 out_seq=0 2024-05-08T05:38:21.853 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:21.852+0000 7fb231ffb700 10 client.26005 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:22.811 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb231ffb700 1 -- 172.21.15.59:0/1009549274 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x563d92368550 con 0x563d92428720 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb231ffb700 10 client.26005 handle_command_reply: tid=0 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 client.26005 shutdown 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1009549274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 msgr2=0x563d9242fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9242fe10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb2440056a0 tx=0x7fb244009ca0 comp rx=0 tx=0).stop 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1009549274 shutdown_connections 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x563d92428720 0x563d92495c90 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x563d92414850 0x563d924308b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x563d9240eb00 0x563d9242fe10 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 --2- 172.21.15.59:0/1009549274 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x563d9240f4c0 0x563d92430350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb231ffb700 0 client.26005 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1009549274 >> 172.21.15.59:0/1009549274 conn(0x563d9233ad00 msgr2=0x563d9240daf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:22.812 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1009549274 shutdown_connections 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 1 -- 172.21.15.59:0/1009549274 wait complete. 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.810+0000 7fb25ab07b80 20 client.26005 trim_cache size 0 max 16384 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 38, 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 26, 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stdout: "d6c3f55e": { 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 26, 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:38:22.813 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 90.0, 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:22.814 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:22.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:22.815 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:22.815 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:38:22.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb2340b2b20 msgr2=0x7fb2340b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:22.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb2340b2b20 0x7fb2340b4fe0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fb24400b2e0 tx=0x7fb24401e040 comp rx=0 tx=0).stop 2024-05-08T05:38:22.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb248102450 msgr2=0x7fb24819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:22.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb248102450 0x7fb24819fc50 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb23800e9f0 tx=0x7fb23800ed00 comp rx=0 tx=0).stop 2024-05-08T05:38:22.815 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 shutdown_connections 2024-05-08T05:38:22.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb2340b2b20 0x7fb2340b4fe0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:22.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb248108460 0x7fb248199920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:22.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb248101a90 0x7fb24819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:22.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 --2- 172.21.15.59:0/4031790287 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb248102450 0x7fb24819fc50 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:22.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 >> 172.21.15.59:0/4031790287 conn(0x7fb248078a40 msgr2=0x7fb248106230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:22.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 shutdown_connections 2024-05-08T05:38:22.816 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:22.811+0000 7fb250274700 1 -- 172.21.15.59:0/4031790287 wait complete. 2024-05-08T05:38:23.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:23 smithi143 ceph-mon[34337]: pgmap v274: 129 pgs: 129 active+clean; 97 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.4 MiB/s wr, 471 op/s 2024-05-08T05:38:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:23 smithi096 ceph-mon[34253]: pgmap v274: 129 pgs: 129 active+clean; 97 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.4 MiB/s wr, 471 op/s 2024-05-08T05:38:23.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:23 smithi059 ceph-mon[30304]: pgmap v274: 129 pgs: 129 active+clean; 97 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 MiB/s rd, 1.4 MiB/s wr, 471 op/s 2024-05-08T05:38:23.738 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'd6c3f55e', quiesce took 8.7 seconds. Will release after: 30.0 2024-05-08T05:38:25.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:25 smithi143 ceph-mon[34337]: pgmap v275: 129 pgs: 129 active+clean; 97 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.4 MiB/s wr, 443 op/s 2024-05-08T05:38:25.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:25 smithi096 ceph-mon[34253]: pgmap v275: 129 pgs: 129 active+clean; 97 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.4 MiB/s wr, 443 op/s 2024-05-08T05:38:25.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:25 smithi059 ceph-mon[30304]: pgmap v275: 129 pgs: 129 active+clean; 97 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 MiB/s rd, 1.4 MiB/s wr, 443 op/s 2024-05-08T05:38:27.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:27 smithi143 ceph-mon[34337]: pgmap v276: 129 pgs: 129 active+clean; 96 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.6 MiB/s wr, 530 op/s 2024-05-08T05:38:27.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:27 smithi096 ceph-mon[34253]: pgmap v276: 129 pgs: 129 active+clean; 96 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.6 MiB/s wr, 530 op/s 2024-05-08T05:38:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.549+0000 7f9c2e62c640 1 -- 172.21.15.59:0/2034923850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28103c60 msgr2=0x7f9c28104040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.549+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/2034923850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28103c60 0x7f9c28104040 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f9c18009900 tx=0x7f9c1802a480 comp rx=0 tx=0).stop 2024-05-08T05:38:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.549+0000 7f9c2e62c640 1 -- 172.21.15.59:0/2034923850 shutdown_connections 2024-05-08T05:38:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.549+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/2034923850 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c2810d030 0x7f9c2810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.549+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/2034923850 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c28104580 0x7f9c2810ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.549+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/2034923850 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28103c60 0x7f9c28104040 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.549+0000 7f9c2e62c640 1 -- 172.21.15.59:0/2034923850 >> 172.21.15.59:0/2034923850 conn(0x7f9c28077c30 msgr2=0x7f9c28078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.549+0000 7f9c2e62c640 1 -- 172.21.15.59:0/2034923850 shutdown_connections 2024-05-08T05:38:27.550 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.549+0000 7f9c2e62c640 1 -- 172.21.15.59:0/2034923850 wait complete. 2024-05-08T05:38:27.552 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.550+0000 7f9c2e62c640 1 Processor -- start 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.550+0000 7f9c2e62c640 1 -- start start 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.550+0000 7f9c2e62c640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c28103c60 0x7f9c28199170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.550+0000 7f9c2e62c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28104580 0x7f9c281996b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.550+0000 7f9c2e62c640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c2810d030 0x7f9c2819d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.550+0000 7f9c2e62c640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28112c70 con 0x7f9c28104580 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.550+0000 7f9c2e62c640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28112af0 con 0x7f9c2810d030 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.550+0000 7f9c2e62c640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9c28112df0 con 0x7f9c28103c60 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28104580 0x7f9c281996b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28104580 0x7f9c281996b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33980/0 (socket says 172.21.15.59:33980) 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c277fe640 1 -- 172.21.15.59:0/3743349523 learned_addr learned my addr 172.21.15.59:0/3743349523 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c27fff640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c28103c60 0x7f9c28199170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c2cbc9640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c2810d030 0x7f9c2819d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.553 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c277fe640 1 -- 172.21.15.59:0/3743349523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c28103c60 msgr2=0x7f9c28199170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c277fe640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c28103c60 0x7f9c28199170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c277fe640 1 -- 172.21.15.59:0/3743349523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c2810d030 msgr2=0x7f9c2819d9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c277fe640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c2810d030 0x7f9c2819d9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c277fe640 1 -- 172.21.15.59:0/3743349523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c2819e110 con 0x7f9c28104580 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c277fe640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28104580 0x7f9c281996b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00e9f0 tx=0x7f9c1c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c257fa640 1 -- 172.21.15.59:0/3743349523 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c00cd80 con 0x7f9c28104580 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c257fa640 1 -- 172.21.15.59:0/3743349523 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c1c0054d0 con 0x7f9c28104580 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c2e62c640 1 -- 172.21.15.59:0/3743349523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c2819e400 con 0x7f9c28104580 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.551+0000 7f9c257fa640 1 -- 172.21.15.59:0/3743349523 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c010690 con 0x7f9c28104580 2024-05-08T05:38:27.554 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.552+0000 7f9c2e62c640 1 -- 172.21.15.59:0/3743349523 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c281a5c90 con 0x7f9c28104580 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c3141f740 1 Processor -- start 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c3141f740 1 -- start start 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c257fa640 1 -- 172.21.15.59:0/3743349523 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f9c1c004e90 con 0x7f9c28104580 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d288102f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 0x564d2881abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2881b0f0 0x564d2881d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c3141f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564d287755d0 con 0x564d2880ff10 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c3141f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564d2872e890 con 0x564d2881b0f0 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c3141f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564d28771590 con 0x564d28810830 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c2cbc9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d288102f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c2cbc9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d288102f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33986/0 (socket says 172.21.15.59:33986) 2024-05-08T05:38:27.557 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c2cbc9640 1 -- 172.21.15.59:0/3496161860 learned_addr learned my addr 172.21.15.59:0/3496161860 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.555+0000 7f9c277fe640 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2881b0f0 0x564d2881d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c27fff640 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 0x564d2881abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c2cbc9640 1 -- 172.21.15.59:0/3496161860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 msgr2=0x564d2881abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c2cbc9640 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 0x564d2881abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c2cbc9640 1 -- 172.21.15.59:0/3496161860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2881b0f0 msgr2=0x564d2881d4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c2cbc9640 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2881b0f0 0x564d2881d4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c2cbc9640 1 -- 172.21.15.59:0/3496161860 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d28487210 con 0x564d2880ff10 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c277fe640 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2881b0f0 0x564d2881d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c257fa640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9c100b2a10 0x7f9c100b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c2cbc9640 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d288102f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009ef0 tx=0x7f9c2000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=361e2f79170872a server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c27fff640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9c100b2a10 0x7f9c100b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c257fa640 1 -- 172.21.15.59:0/3743349523 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f9c1c0e2e90 con 0x7f9c28104580 2024-05-08T05:38:27.558 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c0e7fc640 1 -- 172.21.15.59:0/3496161860 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20015070 con 0x564d2880ff10 2024-05-08T05:38:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c0e7fc640 1 -- 172.21.15.59:0/3496161860 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c200118f0 con 0x564d2880ff10 2024-05-08T05:38:27.559 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.556+0000 7f9c0e7fc640 1 -- 172.21.15.59:0/3496161860 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20010870 con 0x564d2880ff10 2024-05-08T05:38:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 -- 172.21.15.59:0/3496161860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 msgr2=0x564d288102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c27fff640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9c100b2a10 0x7f9c100b4ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c18031440 tx=0x7f9c18005890 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d288102f0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f9c20009ef0 tx=0x7f9c2000f730 comp rx=0 tx=0).stop 2024-05-08T05:38:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 -- 172.21.15.59:0/3496161860 shutdown_connections 2024-05-08T05:38:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2881b0f0 0x564d2881d4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 0x564d2881abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 --2- 172.21.15.59:0/3496161860 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d288102f0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.563 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 -- 172.21.15.59:0/3496161860 >> 172.21.15.59:0/3496161860 conn(0x564d2880c420 msgr2=0x564d2880c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 -- 172.21.15.59:0/3496161860 shutdown_connections 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 -- 172.21.15.59:0/3496161860 wait complete. 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.557+0000 7f9c3141f740 1 -- start start 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c3141f740 1 Processor -- start 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 0x564d288180e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 0x564d28818620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2881b0f0 0x564d28818b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c3141f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564d2881dd20 con 0x564d2881b0f0 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c3141f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564d28732370 con 0x564d2880ff10 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c3141f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564d2881dea0 con 0x564d28810830 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c3141f740 10 client.0 _ll_register_callbacks cb 0x564d28486a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c2cbc9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2881b0f0 0x564d28818b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c2cbc9640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2881b0f0 0x564d28818b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:33996/0 (socket says 172.21.15.59:33996) 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c2cbc9640 1 -- 172.21.15.59:0/2967928490 learned_addr learned my addr 172.21.15.59:0/2967928490 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c27fff640 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 0x564d288180e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.564 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.558+0000 7f9c277fe640 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 0x564d28818620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c2cbc9640 1 -- 172.21.15.59:0/2967928490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 msgr2=0x564d28818620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c2cbc9640 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 0x564d28818620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c2cbc9640 1 -- 172.21.15.59:0/2967928490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 msgr2=0x564d288180e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c2cbc9640 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 0x564d288180e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c2cbc9640 1 -- 172.21.15.59:0/2967928490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d28553620 con 0x564d2881b0f0 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c2cbc9640 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2881b0f0 0x564d28818b60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9c20005cc0 tx=0x7f9c20011670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324e19c381c333e9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c077fe640 1 -- 172.21.15.59:0/2967928490 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c20028060 con 0x564d2881b0f0 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c077fe640 1 -- 172.21.15.59:0/2967928490 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f9c20011cb0 con 0x564d2881b0f0 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c3141f740 1 -- 172.21.15.59:0/2967928490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c200118f0 con 0x564d2881b0f0 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c077fe640 1 -- 172.21.15.59:0/2967928490 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9c2002f050 con 0x564d2881b0f0 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c3141f740 10 client.16512 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c3141f740 1 -- 172.21.15.59:0/2967928490 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564d28829c50 con 0x564d2881b0f0 2024-05-08T05:38:27.565 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.559+0000 7f9c077fe640 1 -- 172.21.15.59:0/2967928490 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c200118f0 con 0x564d2881b0f0 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c077fe640 1 -- 172.21.15.59:0/2967928490 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9c20010b60 con 0x564d2881b0f0 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c3141f740 10 client.16512 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c3141f740 10 client.16512 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c3141f740 20 client.16512 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c3141f740 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564d2882b050 0x564d2882d440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c3141f740 4 client.16512 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c3141f740 1 -- 172.21.15.59:0/2967928490 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d287831e0 con 0x564d2882b050 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c27fff640 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564d2882b050 0x564d2882d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c077fe640 10 client.16512 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.560+0000 7f9c27fff640 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564d2882b050 0x564d2882d440 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72a89fc46e817a28 server_cookie=e0f1f14fd46f285a in_seq=0 out_seq=0 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c077fe640 1 -- 172.21.15.59:0/2967928490 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564d287831e0 con 0x564d2882b050 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c077fe640 10 client.16512 handle_command_reply: tid=0 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c3141f740 1 client.16512 shutdown 2024-05-08T05:38:27.566 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c3141f740 1 -- 172.21.15.59:0/2967928490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2881b0f0 msgr2=0x564d28818b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c3141f740 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2881b0f0 0x564d28818b60 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f9c20005cc0 tx=0x7f9c20011670 comp rx=0 tx=0).stop 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c3141f740 1 -- 172.21.15.59:0/2967928490 shutdown_connections 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c3141f740 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564d2882b050 0x564d2882d440 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c3141f740 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2881b0f0 0x564d28818b60 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c3141f740 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d28810830 0x564d28818620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c077fe640 0 client.16512 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c3141f740 1 --2- 172.21.15.59:0/2967928490 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 0x564d288180e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.561+0000 7f9c3141f740 1 -- 172.21.15.59:0/2967928490 >> 172.21.15.59:0/2967928490 conn(0x564d2880c420 msgr2=0x564d28812150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.562+0000 7f9c3141f740 1 -- 172.21.15.59:0/2967928490 shutdown_connections 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.562+0000 7f9c3141f740 1 -- 172.21.15.59:0/2967928490 wait complete. 2024-05-08T05:38:27.567 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.562+0000 7f9c3141f740 20 client.16512 trim_cache size 0 max 16384 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.577+0000 7f9c3141f740 1 Processor -- start 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.577+0000 7f9c3141f740 1 -- start start 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.577+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d28816880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.577+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d28810830 0x564d28832a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.577+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 0x564d28832f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.577+0000 7f9c3141f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281afe10 con 0x564d2880ff10 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.577+0000 7f9c3141f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b1950 con 0x564d28810830 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c3141f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b04e0 con 0x564d2881b0f0 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d28816880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c27fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d28816880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34010/0 (socket says 172.21.15.59:34010) 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c27fff640 1 -- 172.21.15.59:0/993559817 learned_addr learned my addr 172.21.15.59:0/993559817 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:27.580 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c27fff640 1 -- 172.21.15.59:0/993559817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 msgr2=0x564d28832f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c27fff640 1 --2- 172.21.15.59:0/993559817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 0x564d28832f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c27fff640 1 -- 172.21.15.59:0/993559817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d28810830 msgr2=0x564d28832a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c27fff640 1 --2- 172.21.15.59:0/993559817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d28810830 0x564d28832a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c27fff640 1 -- 172.21.15.59:0/993559817 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d2881d4e0 con 0x564d2880ff10 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c27fff640 1 --2- 172.21.15.59:0/993559817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d28816880 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9c180096c0 tx=0x7f9c1803e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26dedb4cc6a434c8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c0dffb640 1 -- 172.21.15.59:0/993559817 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18056070 con 0x564d2880ff10 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c0dffb640 1 -- 172.21.15.59:0/993559817 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c180507a0 con 0x564d2880ff10 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.578+0000 7f9c0dffb640 1 -- 172.21.15.59:0/993559817 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c18035040 con 0x564d2880ff10 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 -- 172.21.15.59:0/993559817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 msgr2=0x564d28816880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 --2- 172.21.15.59:0/993559817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d28816880 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f9c180096c0 tx=0x7f9c1803e000 comp rx=0 tx=0).stop 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 -- 172.21.15.59:0/993559817 shutdown_connections 2024-05-08T05:38:27.581 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 --2- 172.21.15.59:0/993559817 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 0x564d28832f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 --2- 172.21.15.59:0/993559817 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d28810830 0x564d28832a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 --2- 172.21.15.59:0/993559817 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d2880ff10 0x564d28816880 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 -- 172.21.15.59:0/993559817 >> 172.21.15.59:0/993559817 conn(0x564d2880c420 msgr2=0x564d28818930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 -- 172.21.15.59:0/993559817 shutdown_connections 2024-05-08T05:38:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 -- 172.21.15.59:0/993559817 wait complete. 2024-05-08T05:38:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.579+0000 7f9c3141f740 1 -- start start 2024-05-08T05:38:27.582 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c3141f740 1 Processor -- start 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 0x564d2881ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d28810830 0x564d2881f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c3141f740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 0x564d2881f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c3141f740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b1280 con 0x564d28810830 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c3141f740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281b0bb0 con 0x564d2880ff10 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c3141f740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f9c281ade10 con 0x564d2881b0f0 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c3141f740 10 client.0 _ll_register_callbacks cb 0x7f9c20009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d28810830 0x564d2881f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c277fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d28810830 0x564d2881f250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:34026/0 (socket says 172.21.15.59:34026) 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c277fe640 1 -- 172.21.15.59:0/4007841531 learned_addr learned my addr 172.21.15.59:0/4007841531 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:27.585 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c27fff640 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 0x564d2881ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c277fe640 1 -- 172.21.15.59:0/4007841531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 msgr2=0x564d2881f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c2cbc9640 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 0x564d2881f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c277fe640 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 0x564d2881f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c277fe640 1 -- 172.21.15.59:0/4007841531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 msgr2=0x564d2881ed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c277fe640 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 0x564d2881ed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c277fe640 1 -- 172.21.15.59:0/4007841531 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d288945f0 con 0x564d28810830 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c2cbc9640 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 0x564d2881f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c277fe640 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d28810830 0x564d2881f250 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00c6a0 tx=0x7f9c1c0a5d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19f031a4e8d624f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.580+0000 7f9c077fe640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c077fe640 1 -- 172.21.15.59:0/4007841531 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c1c005180 con 0x564d28810830 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c077fe640 1 -- 172.21.15.59:0/4007841531 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f9c1c0b2980 con 0x564d28810830 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c3141f740 1 -- 172.21.15.59:0/4007841531 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9c180507a0 con 0x564d28810830 2024-05-08T05:38:27.586 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c077fe640 1 -- 172.21.15.59:0/4007841531 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f9c1c0a98b0 con 0x564d28810830 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c3141f740 10 client.16518 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c3141f740 1 -- 172.21.15.59:0/4007841531 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564d28895940 con 0x564d28810830 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c077fe640 1 -- 172.21.15.59:0/4007841531 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c180507a0 con 0x564d28810830 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c077fe640 1 -- 172.21.15.59:0/4007841531 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f9c1c0a5710 con 0x564d28810830 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c3141f740 10 client.16518 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c3141f740 10 client.16518 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c3141f740 20 client.16518 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c3141f740 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564d288960f0 0x564d28898510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c3141f740 4 client.16518 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c3141f740 1 -- 172.21.15.59:0/4007841531 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564d2877d050 con 0x564d288960f0 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.581+0000 7f9c27fff640 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564d288960f0 0x564d28898510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.582+0000 7f9c27fff640 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564d288960f0 0x564d28898510 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb2f2860dd4eaff2 server_cookie=2b7d377ba596ec1b in_seq=0 out_seq=0 2024-05-08T05:38:27.587 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.582+0000 7f9c077fe640 10 client.16518 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:27.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:27 smithi059 ceph-mon[30304]: pgmap v276: 129 pgs: 129 active+clean; 96 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.8 MiB/s rd, 1.6 MiB/s wr, 530 op/s 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.598+0000 7f9c077fe640 1 -- 172.21.15.59:0/4007841531 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564d2877d050 con 0x564d288960f0 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.598+0000 7f9c077fe640 10 client.16518 handle_command_reply: tid=0 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.598+0000 7f9c3141f740 1 client.16518 shutdown 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 -- 172.21.15.59:0/4007841531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d28810830 msgr2=0x564d2881f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d28810830 0x564d2881f250 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00c6a0 tx=0x7f9c1c0a5d20 comp rx=0 tx=0).stop 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 -- 172.21.15.59:0/4007841531 shutdown_connections 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564d288960f0 0x564d28898510 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564d2881b0f0 0x564d2881f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564d28810830 0x564d2881f250 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.600 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 --2- 172.21.15.59:0/4007841531 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564d2880ff10 0x564d2881ed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c077fe640 0 client.16518 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 -- 172.21.15.59:0/4007841531 >> 172.21.15.59:0/4007841531 conn(0x564d2880c420 msgr2=0x564d288118c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 -- 172.21.15.59:0/4007841531 shutdown_connections 2024-05-08T05:38:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 1 -- 172.21.15.59:0/4007841531 wait complete. 2024-05-08T05:38:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.599+0000 7f9c3141f740 20 client.16518 trim_cache size 0 max 16384 2024-05-08T05:38:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:38:27.601 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:38:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:38:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:38:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 -- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9c100b2a10 msgr2=0x7f9c100b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:27.602 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9c100b2a10 0x7f9c100b4ed0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9c18031440 tx=0x7f9c18005890 comp rx=0 tx=0).stop 2024-05-08T05:38:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 -- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28104580 msgr2=0x7f9c281996b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28104580 0x7f9c281996b0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f9c1c00e9f0 tx=0x7f9c1c00edf0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 -- 172.21.15.59:0/3743349523 shutdown_connections 2024-05-08T05:38:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f9c100b2a10 0x7f9c100b4ed0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f9c2810d030 0x7f9c2819d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f9c28104580 0x7f9c281996b0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 --2- 172.21.15.59:0/3743349523 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f9c28103c60 0x7f9c28199170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 -- 172.21.15.59:0/3743349523 >> 172.21.15.59:0/3743349523 conn(0x7f9c28077c30 msgr2=0x7f9c280fe7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:27.603 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 -- 172.21.15.59:0/3743349523 shutdown_connections 2024-05-08T05:38:27.604 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:27.600+0000 7f9c2e62c640 1 -- 172.21.15.59:0/3743349523 wait complete. 2024-05-08T05:38:29.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:29 smithi143 ceph-mon[34337]: pgmap v277: 129 pgs: 129 active+clean; 96 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 925 KiB/s wr, 307 op/s 2024-05-08T05:38:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:29 smithi096 ceph-mon[34253]: pgmap v277: 129 pgs: 129 active+clean; 96 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 925 KiB/s wr, 307 op/s 2024-05-08T05:38:29.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:29 smithi059 ceph-mon[30304]: pgmap v277: 129 pgs: 129 active+clean; 96 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 925 KiB/s wr, 307 op/s 2024-05-08T05:38:31.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:31 smithi143 ceph-mon[34337]: pgmap v278: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 934 KiB/s wr, 314 op/s 2024-05-08T05:38:31.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:31 smithi096 ceph-mon[34253]: pgmap v278: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 934 KiB/s wr, 314 op/s 2024-05-08T05:38:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:31 smithi059 ceph-mon[30304]: pgmap v278: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 5.8 MiB/s rd, 934 KiB/s wr, 314 op/s 2024-05-08T05:38:33.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:33 smithi143 ceph-mon[34337]: pgmap v279: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 271 KiB/s wr, 94 op/s 2024-05-08T05:38:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:33 smithi096 ceph-mon[34253]: pgmap v279: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 271 KiB/s wr, 94 op/s 2024-05-08T05:38:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:33 smithi059 ceph-mon[30304]: pgmap v279: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 271 KiB/s wr, 94 op/s 2024-05-08T05:38:35.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:35 smithi143 ceph-mon[34337]: pgmap v280: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 271 KiB/s wr, 94 op/s 2024-05-08T05:38:35.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:35 smithi096 ceph-mon[34253]: pgmap v280: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 271 KiB/s wr, 94 op/s 2024-05-08T05:38:35.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:35 smithi059 ceph-mon[30304]: pgmap v280: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 785 KiB/s rd, 271 KiB/s wr, 94 op/s 2024-05-08T05:38:37.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:37 smithi143 ceph-mon[34337]: pgmap v281: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 273 KiB/s wr, 99 op/s 2024-05-08T05:38:37.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:37 smithi096 ceph-mon[34253]: pgmap v281: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 273 KiB/s wr, 99 op/s 2024-05-08T05:38:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:37 smithi059 ceph-mon[30304]: pgmap v281: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 790 KiB/s rd, 273 KiB/s wr, 99 op/s 2024-05-08T05:38:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:39 smithi143 ceph-mon[34337]: pgmap v282: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-08T05:38:39.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:39 smithi096 ceph-mon[34253]: pgmap v282: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-08T05:38:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:39 smithi059 ceph-mon[30304]: pgmap v282: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 11 KiB/s wr, 11 op/s 2024-05-08T05:38:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.701+0000 7fc5b706b640 1 -- 172.21.15.59:0/1377842912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 msgr2=0x7fc5b010e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.701+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1377842912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 0x7fc5b010e7a0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009cd0 tx=0x7fc5a802a8e0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.701+0000 7fc5b706b640 1 -- 172.21.15.59:0/1377842912 shutdown_connections 2024-05-08T05:38:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.701+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1377842912 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 0x7fc5b010e7a0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.701+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1377842912 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5b01018d0 0x7fc5b0101d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.701+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1377842912 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5b01078e0 0x7fc5b0107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.702 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.701+0000 7fc5b706b640 1 -- 172.21.15.59:0/1377842912 >> 172.21.15.59:0/1377842912 conn(0x7fc5b0077c30 msgr2=0x7fc5b0078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.701+0000 7fc5b706b640 1 -- 172.21.15.59:0/1377842912 shutdown_connections 2024-05-08T05:38:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.701+0000 7fc5b706b640 1 -- 172.21.15.59:0/1377842912 wait complete. 2024-05-08T05:38:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5b706b640 1 Processor -- start 2024-05-08T05:38:40.703 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5b706b640 1 -- start start 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5b706b640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5b01018d0 0x7fc5b0199060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5b706b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 0x7fc5b01995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5b706b640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5b01078e0 0x7fc5b019d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5b706b640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0112ca0 con 0x7fc5b0102290 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5b706b640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0112b20 con 0x7fc5b01018d0 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5b706b640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b0112e20 con 0x7fc5b01078e0 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 0x7fc5b01995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 0x7fc5b01995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58532/0 (socket says 172.21.15.59:58532) 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.702+0000 7fc5a7fff640 1 -- 172.21.15.59:0/1503877413 learned_addr learned my addr 172.21.15.59:0/1503877413 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:40.705 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5b4e07640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5b01018d0 0x7fc5b0199060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5b5608640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5b01078e0 0x7fc5b019d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5a7fff640 1 -- 172.21.15.59:0/1503877413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5b01078e0 msgr2=0x7fc5b019d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5b01078e0 0x7fc5b019d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5a7fff640 1 -- 172.21.15.59:0/1503877413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5b01018d0 msgr2=0x7fc5b0199060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5b01018d0 0x7fc5b0199060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5a7fff640 1 -- 172.21.15.59:0/1503877413 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5b019e000 con 0x7fc5b0102290 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5b5608640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5b01078e0 0x7fc5b019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5b4e07640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5b01018d0 0x7fc5b0199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 0x7fc5b01995a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000c920 tx=0x7fc5a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:40.706 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5a5ffb640 1 -- 172.21.15.59:0/1503877413 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0007c70 con 0x7fc5b0102290 2024-05-08T05:38:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5b706b640 1 -- 172.21.15.59:0/1503877413 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5b019e2f0 con 0x7fc5b0102290 2024-05-08T05:38:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5a5ffb640 1 -- 172.21.15.59:0/1503877413 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a0007e10 con 0x7fc5b0102290 2024-05-08T05:38:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5a5ffb640 1 -- 172.21.15.59:0/1503877413 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a0018820 con 0x7fc5b0102290 2024-05-08T05:38:40.707 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.703+0000 7fc5b706b640 1 -- 172.21.15.59:0/1503877413 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5b01a5b80 con 0x7fc5b0102290 2024-05-08T05:38:40.713 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.706+0000 7fc5a5ffb640 1 -- 172.21.15.59:0/1503877413 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fc5a000f450 con 0x7fc5b0102290 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.706+0000 7fc5b9d88740 1 Processor -- start 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.706+0000 7fc5b9d88740 1 -- start start 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 0x55d2a07352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a073fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a07424e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d2a069a5d0 con 0x55d2a07400f0 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2a0653890 con 0x55d2a0734f10 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d2a0696590 con 0x55d2a0735830 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a07424e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5b5608640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 0x55d2a07352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.714 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a5ffb640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc5980b2a10 0x7fc5980b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a07424e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58546/0 (socket says 172.21.15.59:58546) 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5b4e07640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a073fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a7fff640 1 -- 172.21.15.59:0/727560117 learned_addr learned my addr 172.21.15.59:0/727560117 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5b5608640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc5980b2a10 0x7fc5980b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a5ffb640 1 -- 172.21.15.59:0/1503877413 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fc5a00e2c10 con 0x7fc5b0102290 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a7fff640 1 -- 172.21.15.59:0/727560117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 msgr2=0x55d2a073fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/727560117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a073fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a7fff640 1 -- 172.21.15.59:0/727560117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 msgr2=0x55d2a07352f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/727560117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 0x55d2a07352f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.707+0000 7fc5a7fff640 1 -- 172.21.15.59:0/727560117 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a03ac210 con 0x55d2a07400f0 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.708+0000 7fc5b5608640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc5980b2a10 0x7fc5980b4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80317f0 tx=0x7fc5a802a8e0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.708+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/727560117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a07424e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00a9ab0 tx=0x7fc5a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c6aa67f4a07be28 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.708+0000 7fc596ffd640 1 -- 172.21.15.59:0/727560117 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00aa220 con 0x55d2a07400f0 2024-05-08T05:38:40.715 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.708+0000 7fc596ffd640 1 -- 172.21.15.59:0/727560117 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a00a9b20 con 0x55d2a07400f0 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.708+0000 7fc596ffd640 1 -- 172.21.15.59:0/727560117 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00a5db0 con 0x55d2a07400f0 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.708+0000 7fc5b9d88740 1 -- 172.21.15.59:0/727560117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 msgr2=0x55d2a07424e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.708+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/727560117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a07424e0 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00a9ab0 tx=0x7fc5a0000bc0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 -- 172.21.15.59:0/727560117 shutdown_connections 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/727560117 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a07424e0 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/727560117 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a073fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/727560117 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 0x55d2a07352f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 -- 172.21.15.59:0/727560117 >> 172.21.15.59:0/727560117 conn(0x55d2a0731420 msgr2=0x55d2a0731800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 -- 172.21.15.59:0/727560117 shutdown_connections 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 -- 172.21.15.59:0/727560117 wait complete. 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 -- start start 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 Processor -- start 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0734f10 0x55d2a073d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.716 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a073d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 0x55d2a073db30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d2a0742d20 con 0x55d2a0734f10 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d2a0657370 con 0x55d2a07400f0 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.709+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d2a0742ea0 con 0x55d2a0735830 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b9d88740 10 client.0 _ll_register_callbacks cb 0x55d2a03aba00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b4e07640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0734f10 0x55d2a073d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b4e07640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0734f10 0x55d2a073d0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58556/0 (socket says 172.21.15.59:58556) 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b4e07640 1 -- 172.21.15.59:0/3863692201 learned_addr learned my addr 172.21.15.59:0/3863692201 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b5608640 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 0x55d2a073db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a073d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b4e07640 1 -- 172.21.15.59:0/3863692201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 msgr2=0x55d2a073d5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b4e07640 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a073d5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.717 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b4e07640 1 -- 172.21.15.59:0/3863692201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 msgr2=0x55d2a073db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b4e07640 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 0x55d2a073db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b4e07640 1 -- 172.21.15.59:0/3863692201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a0478620 con 0x55d2a0734f10 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b4e07640 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0734f10 0x55d2a073d0b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009ed0 tx=0x7fc5ac0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cafcfc992266afee server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc58ffff640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc58ffff640 1 -- 172.21.15.59:0/3863692201 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac012860 con 0x55d2a0734f10 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc58ffff640 1 -- 172.21.15.59:0/3863692201 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fc5ac012a00 con 0x55d2a0734f10 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.710+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3863692201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5a00a9b20 con 0x55d2a0734f10 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc58ffff640 1 -- 172.21.15.59:0/3863692201 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc5ac025050 con 0x55d2a0734f10 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc5b9d88740 10 client.16530 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3863692201 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d2a074ec80 con 0x55d2a0734f10 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc58ffff640 1 -- 172.21.15.59:0/3863692201 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a00a9b20 con 0x55d2a0734f10 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc58ffff640 1 -- 172.21.15.59:0/3863692201 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fc5ac00f040 con 0x55d2a0734f10 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc5b9d88740 10 client.16530 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:40.718 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc5b9d88740 10 client.16530 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc5b9d88740 20 client.16530 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d2a0750190 0x55d2a0752580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc5b9d88740 4 client.16530 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3863692201 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d2a06a81e0 con 0x55d2a0750190 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.711+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d2a0750190 0x55d2a0752580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.712+0000 7fc58ffff640 10 client.16530 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.712+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d2a0750190 0x55d2a0752580 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=993de634f8b4130d server_cookie=2040167a9e07d34a in_seq=0 out_seq=0 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.712+0000 7fc58ffff640 1 -- 172.21.15.59:0/3863692201 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d2a06a81e0 con 0x55d2a0750190 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.712+0000 7fc58ffff640 10 client.16530 handle_command_reply: tid=0 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 client.16530 shutdown 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3863692201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0734f10 msgr2=0x55d2a073d0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0734f10 0x55d2a073d0b0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009ed0 tx=0x7fc5ac0117b0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3863692201 shutdown_connections 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d2a0750190 0x55d2a0752580 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.719 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 0x55d2a073db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a073d5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc58ffff640 0 client.16530 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3863692201 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0734f10 0x55d2a073d0b0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3863692201 >> 172.21.15.59:0/3863692201 conn(0x55d2a0731420 msgr2=0x55d2a0737150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3863692201 shutdown_connections 2024-05-08T05:38:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3863692201 wait complete. 2024-05-08T05:38:40.720 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.713+0000 7fc5b9d88740 20 client.16530 trim_cache size 0 max 16384 2024-05-08T05:38:40.733 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.728+0000 7fc5b9d88740 1 Processor -- start 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5b9d88740 1 -- start start 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0734f10 0x55d2a0730580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0735830 0x55d2a073d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 0x55d2a073eff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01afdb0 con 0x55d2a0735830 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01b18f0 con 0x55d2a07400f0 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01b0480 con 0x55d2a0734f10 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0735830 0x55d2a073d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5a7fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0735830 0x55d2a073d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58564/0 (socket says 172.21.15.59:58564) 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5a7fff640 1 -- 172.21.15.59:0/808786997 learned_addr learned my addr 172.21.15.59:0/808786997 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5b4e07640 1 --2- 172.21.15.59:0/808786997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0734f10 0x55d2a0730580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5b5608640 1 --2- 172.21.15.59:0/808786997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 0x55d2a073eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.734 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5a7fff640 1 -- 172.21.15.59:0/808786997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0734f10 msgr2=0x55d2a0730580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/808786997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0734f10 0x55d2a0730580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5a7fff640 1 -- 172.21.15.59:0/808786997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 msgr2=0x55d2a073eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/808786997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 0x55d2a073eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5a7fff640 1 -- 172.21.15.59:0/808786997 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d2a07325b0 con 0x55d2a0735830 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/808786997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0735830 0x55d2a073d950 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00e0590 tx=0x7fc5a0091040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e2509782a8fd418 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.729+0000 7fc5967fc640 1 -- 172.21.15.59:0/808786997 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00c3070 con 0x55d2a0735830 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5967fc640 1 -- 172.21.15.59:0/808786997 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a00aae50 con 0x55d2a0735830 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5967fc640 1 -- 172.21.15.59:0/808786997 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a00a9b20 con 0x55d2a0735830 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 -- 172.21.15.59:0/808786997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0735830 msgr2=0x55d2a073d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/808786997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0735830 0x55d2a073d950 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fc5a00e0590 tx=0x7fc5a0091040 comp rx=0 tx=0).stop 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 -- 172.21.15.59:0/808786997 shutdown_connections 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/808786997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a07400f0 0x55d2a073eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/808786997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a0735830 0x55d2a073d950 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.735 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/808786997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0734f10 0x55d2a0730580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 -- 172.21.15.59:0/808786997 >> 172.21.15.59:0/808786997 conn(0x55d2a0731420 msgr2=0x55d2a073e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 -- 172.21.15.59:0/808786997 shutdown_connections 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 -- 172.21.15.59:0/808786997 wait complete. 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.730+0000 7fc5b9d88740 1 -- start start 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b9d88740 1 Processor -- start 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 0x55d2a07487f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a0743b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b9d88740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a0744250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01b1220 con 0x55d2a07400f0 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01b0b50 con 0x55d2a0734f10 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b9d88740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fc5b01addb0 con 0x55d2a0735830 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b9d88740 10 client.0 _ll_register_callbacks cb 0x7fc5ac009670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b5608640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a0744250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b5608640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a0744250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:58566/0 (socket says 172.21.15.59:58566) 2024-05-08T05:38:40.736 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b5608640 1 -- 172.21.15.59:0/3601999590 learned_addr learned my addr 172.21.15.59:0/3601999590 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b4e07640 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 0x55d2a07487f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5a7fff640 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a0743b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b5608640 1 -- 172.21.15.59:0/3601999590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 msgr2=0x55d2a0743b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.731+0000 7fc5b5608640 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a0743b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc5b5608640 1 -- 172.21.15.59:0/3601999590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 msgr2=0x55d2a07487f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc5b5608640 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 0x55d2a07487f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc5b5608640 1 -- 172.21.15.59:0/3601999590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d2a07b9a20 con 0x55d2a07400f0 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc5b5608640 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a0744250 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009cd0 tx=0x7fc5a80095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83f98f86da97a34b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc58ffff640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc58ffff640 1 -- 172.21.15.59:0/3601999590 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5a8051070 con 0x55d2a07400f0 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc58ffff640 1 -- 172.21.15.59:0/3601999590 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fc5a8004450 con 0x55d2a07400f0 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3601999590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc5a00aae50 con 0x55d2a07400f0 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc58ffff640 1 -- 172.21.15.59:0/3601999590 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fc5a805e050 con 0x55d2a07400f0 2024-05-08T05:38:40.737 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc5b9d88740 10 client.16536 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3601999590 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55d2a07babd0 con 0x55d2a07400f0 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.732+0000 7fc58ffff640 1 -- 172.21.15.59:0/3601999590 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5a00aae50 con 0x55d2a07400f0 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc58ffff640 1 -- 172.21.15.59:0/3601999590 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fc5a8030070 con 0x55d2a07400f0 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc5b9d88740 10 client.16536 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc5b9d88740 10 client.16536 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc5b9d88740 20 client.16536 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d2a07bb350 0x55d2a07bd770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc5b9d88740 4 client.16536 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3601999590 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d2a06a2050 con 0x55d2a07bb350 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc5b4e07640 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d2a07bb350 0x55d2a07bd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc5b4e07640 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d2a07bb350 0x55d2a07bd770 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba74c04a2428817c server_cookie=e014e0b27507c170 in_seq=0 out_seq=0 2024-05-08T05:38:40.738 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.733+0000 7fc58ffff640 10 client.16536 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc58ffff640 1 -- 172.21.15.59:0/3601999590 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d2a06a2050 con 0x55d2a07bb350 2024-05-08T05:38:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc58ffff640 10 client.16536 handle_command_reply: tid=0 2024-05-08T05:38:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 client.16536 shutdown 2024-05-08T05:38:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3601999590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 msgr2=0x55d2a0744250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:40.748 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a0744250 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7fc5a8009cd0 tx=0x7fc5a80095c0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3601999590 shutdown_connections 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d2a07bb350 0x55d2a07bd770 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d2a07400f0 0x55d2a0744250 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d2a0735830 0x55d2a0743b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 --2- 172.21.15.59:0/3601999590 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d2a0734f10 0x55d2a07487f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3601999590 >> 172.21.15.59:0/3601999590 conn(0x55d2a0731420 msgr2=0x55d2a073f170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc58ffff640 0 client.16536 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3601999590 shutdown_connections 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 1 -- 172.21.15.59:0/3601999590 wait complete. 2024-05-08T05:38:40.749 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.747+0000 7fc5b9d88740 20 client.16536 trim_cache size 0 max 16384 2024-05-08T05:38:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:38:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:38:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:38:40.750 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 -- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc5980b2a10 msgr2=0x7fc5980b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc5980b2a10 0x7fc5980b4ed0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc5a80317f0 tx=0x7fc5a802a8e0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 -- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 msgr2=0x7fc5b01995a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 0x7fc5b01995a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fc5a000c920 tx=0x7fc5a000cde0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 -- 172.21.15.59:0/1503877413 shutdown_connections 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fc5980b2a10 0x7fc5980b4ed0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fc5b01078e0 0x7fc5b019d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fc5b0102290 0x7fc5b01995a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 --2- 172.21.15.59:0/1503877413 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fc5b01018d0 0x7fc5b0199060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 -- 172.21.15.59:0/1503877413 >> 172.21.15.59:0/1503877413 conn(0x7fc5b0077c30 msgr2=0x7fc5b00ff5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:40.751 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 -- 172.21.15.59:0/1503877413 shutdown_connections 2024-05-08T05:38:40.752 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:40.748+0000 7fc5b706b640 1 -- 172.21.15.59:0/1503877413 wait complete. 2024-05-08T05:38:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:41 smithi143 ceph-mon[34337]: pgmap v283: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-05-08T05:38:41.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:41 smithi096 ceph-mon[34253]: pgmap v283: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-05-08T05:38:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:41 smithi059 ceph-mon[30304]: pgmap v283: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 11 KiB/s wr, 15 op/s 2024-05-08T05:38:43.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:43 smithi143 ceph-mon[34337]: pgmap v284: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-05-08T05:38:43.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:43 smithi096 ceph-mon[34253]: pgmap v284: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-05-08T05:38:43.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:43 smithi059 ceph-mon[30304]: pgmap v284: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-05-08T05:38:45.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:45 smithi143 ceph-mon[34337]: pgmap v285: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-05-08T05:38:45.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:45 smithi096 ceph-mon[34253]: pgmap v285: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-05-08T05:38:45.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:45 smithi059 ceph-mon[30304]: pgmap v285: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 2.3 KiB/s wr, 8 op/s 2024-05-08T05:38:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:47 smithi143 ceph-mon[34337]: pgmap v286: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-05-08T05:38:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:47 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:38:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:47 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:38:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:47 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:38:47.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:47 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:38:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:47 smithi096 ceph-mon[34253]: pgmap v286: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-05-08T05:38:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:47 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:38:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:47 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:38:47.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:47 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:38:47.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:47 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:38:47.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:47 smithi059 ceph-mon[30304]: pgmap v286: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 23 KiB/s wr, 15 op/s 2024-05-08T05:38:47.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:47 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:38:47.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:47 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:38:47.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:47 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:38:47.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:47 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:38:49.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:49 smithi143 ceph-mon[34337]: pgmap v287: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T05:38:49.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:49 smithi096 ceph-mon[34253]: pgmap v287: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T05:38:49.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:49 smithi059 ceph-mon[30304]: pgmap v287: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T05:38:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:51 smithi143 ceph-mon[34337]: pgmap v288: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-05-08T05:38:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:51 smithi096 ceph-mon[34253]: pgmap v288: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-05-08T05:38:51.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:51 smithi059 ceph-mon[30304]: pgmap v288: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 21 KiB/s wr, 14 op/s 2024-05-08T05:38:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:53 smithi143 ceph-mon[34337]: pgmap v289: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T05:38:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:53 smithi096 ceph-mon[34253]: pgmap v289: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T05:38:53.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:53 smithi059 ceph-mon[30304]: pgmap v289: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T05:38:53.739 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:38:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.847+0000 7fcad9064640 1 -- 172.21.15.59:0/506474325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41078a0 msgr2=0x7fcad4107c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.847+0000 7fcad9064640 1 --2- 172.21.15.59:0/506474325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41078a0 0x7fcad4107c80 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7fcac4009930 tx=0x7fcac402a420 comp rx=0 tx=0).stop 2024-05-08T05:38:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.847+0000 7fcad9064640 1 -- 172.21.15.59:0/506474325 shutdown_connections 2024-05-08T05:38:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.847+0000 7fcad9064640 1 --2- 172.21.15.59:0/506474325 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcad41022a0 0x7fcad410e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.847+0000 7fcad9064640 1 --2- 172.21.15.59:0/506474325 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcad41018e0 0x7fcad4101d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.847+0000 7fcad9064640 1 --2- 172.21.15.59:0/506474325 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41078a0 0x7fcad4107c80 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.847+0000 7fcad9064640 1 -- 172.21.15.59:0/506474325 >> 172.21.15.59:0/506474325 conn(0x7fcad4078a30 msgr2=0x7fcad4078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.847+0000 7fcad9064640 1 -- 172.21.15.59:0/506474325 shutdown_connections 2024-05-08T05:38:53.848 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.847+0000 7fcad9064640 1 -- 172.21.15.59:0/506474325 wait complete. 2024-05-08T05:38:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad9064640 1 Processor -- start 2024-05-08T05:38:53.849 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad9064640 1 -- start start 2024-05-08T05:38:53.850 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad9064640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41018e0 0x7fcad4199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad9064640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcad41022a0 0x7fcad4199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad9064640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcad41078a0 0x7fcad419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad9064640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4111b60 con 0x7fcad41018e0 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad9064640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41119e0 con 0x7fcad41078a0 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad9064640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcad4111ce0 con 0x7fcad41022a0 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad2d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41018e0 0x7fcad4199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad2d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41018e0 0x7fcad4199210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42786/0 (socket says 172.21.15.59:42786) 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad2d9d640 1 -- 172.21.15.59:0/1508108595 learned_addr learned my addr 172.21.15.59:0/1508108595 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad359e640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcad41078a0 0x7fcad419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcacbfff640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcad41022a0 0x7fcad4199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.848+0000 7fcad2d9d640 1 -- 172.21.15.59:0/1508108595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcad41022a0 msgr2=0x7fcad4199750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcad41022a0 0x7fcad4199750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.851 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad2d9d640 1 -- 172.21.15.59:0/1508108595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcad41078a0 msgr2=0x7fcad419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcad41078a0 0x7fcad419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad2d9d640 1 -- 172.21.15.59:0/1508108595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcad419e200 con 0x7fcad41018e0 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcacbfff640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcad41022a0 0x7fcad4199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad359e640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcad41078a0 0x7fcad419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41018e0 0x7fcad4199210 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcac4004770 tx=0x7fcac40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad0d99640 1 -- 172.21.15.59:0/1508108595 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac403f070 con 0x7fcad41018e0 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad9064640 1 -- 172.21.15.59:0/1508108595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcad419e490 con 0x7fcad41018e0 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad0d99640 1 -- 172.21.15.59:0/1508108595 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcac403b460 con 0x7fcad41018e0 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad0d99640 1 -- 172.21.15.59:0/1508108595 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcac4035070 con 0x7fcad41018e0 2024-05-08T05:38:53.852 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.849+0000 7fcad9064640 1 -- 172.21.15.59:0/1508108595 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcad41a5d80 con 0x7fcad41018e0 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcad0d99640 1 -- 172.21.15.59:0/1508108595 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fcac403b8d0 con 0x7fcad41018e0 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcadbd7d740 1 Processor -- start 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcadbd7d740 1 -- start start 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 0x564f82ffc2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffc830 0x564f83006bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f830094e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f82f615d0 con 0x564f830070f0 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564f82f1a890 con 0x564f82ffbf10 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564f82f5d590 con 0x564f82ffc830 2024-05-08T05:38:53.858 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f830094e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.852+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f830094e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42792/0 (socket says 172.21.15.59:42792) 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcacbfff640 1 -- 172.21.15.59:0/2819428443 learned_addr learned my addr 172.21.15.59:0/2819428443 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcad359e640 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 0x564f82ffc2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffc830 0x564f83006bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcacbfff640 1 -- 172.21.15.59:0/2819428443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffc830 msgr2=0x564f83006bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcacbfff640 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffc830 0x564f83006bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcacbfff640 1 -- 172.21.15.59:0/2819428443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 msgr2=0x564f82ffc2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcacbfff640 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 0x564f82ffc2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcacbfff640 1 -- 172.21.15.59:0/2819428443 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f82c73210 con 0x564f830070f0 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffc830 0x564f83006bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcacbfff640 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f830094e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009c20 tx=0x7fcabc00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c9897a919b854f4 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcad0d99640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fcac00b2a10 0x7fcac00b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcac97fa640 1 -- 172.21.15.59:0/2819428443 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc016070 con 0x564f830070f0 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcac97fa640 1 -- 172.21.15.59:0/2819428443 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcabc011920 con 0x564f830070f0 2024-05-08T05:38:53.859 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcac97fa640 1 -- 172.21.15.59:0/2819428443 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc0107d0 con 0x564f830070f0 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.853+0000 7fcad359e640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fcac00b2a10 0x7fcac00b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcad0d99640 1 -- 172.21.15.59:0/1508108595 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fcac4102e10 con 0x7fcad41018e0 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2819428443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 msgr2=0x564f830094e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcad359e640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fcac00b2a10 0x7fcac00b4ed0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c610 tx=0x7fcacc00e040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f830094e0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7fcabc009c20 tx=0x7fcabc00e6c0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2819428443 shutdown_connections 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f830094e0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffc830 0x564f83006bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2819428443 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 0x564f82ffc2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2819428443 >> 172.21.15.59:0/2819428443 conn(0x564f82ff8420 msgr2=0x564f82ff8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2819428443 shutdown_connections 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.854+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2819428443 wait complete. 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcadbd7d740 1 -- start start 2024-05-08T05:38:53.860 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcadbd7d740 1 Processor -- start 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 0x564f83003f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f830044b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f830070f0 0x564f830049f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x564f83009d20 con 0x564f82ffc830 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x564f82f1e370 con 0x564f82ffbf10 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x564f83009ea0 con 0x564f830070f0 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcadbd7d740 10 client.0 _ll_register_callbacks cb 0x564f82c72a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f830044b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f830044b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42794/0 (socket says 172.21.15.59:42794) 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcacbfff640 1 -- 172.21.15.59:0/1122710147 learned_addr learned my addr 172.21.15.59:0/1122710147 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 0x564f83003f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcacbfff640 1 -- 172.21.15.59:0/1122710147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f830070f0 msgr2=0x564f830049f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcacbfff640 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f830070f0 0x564f830049f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcacbfff640 1 -- 172.21.15.59:0/1122710147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 msgr2=0x564f83003f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcacbfff640 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 0x564f83003f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.855+0000 7fcacbfff640 1 -- 172.21.15.59:0/1122710147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f82d3f620 con 0x564f82ffc830 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcacbfff640 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f830044b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcabc000c00 tx=0x7fcabc0097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a14e1ad3986eb0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcaba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcaba7fc640 1 -- 172.21.15.59:0/1122710147 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc027020 con 0x564f82ffc830 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcaba7fc640 1 -- 172.21.15.59:0/1122710147 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fcabc004020 con 0x564f82ffc830 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1122710147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcabc011920 con 0x564f82ffc830 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcaba7fc640 1 -- 172.21.15.59:0/1122710147 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcabc02f050 con 0x564f82ffc830 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcadbd7d740 10 client.16548 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1122710147 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564f83015a90 con 0x564f82ffc830 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcaba7fc640 1 -- 172.21.15.59:0/1122710147 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcabc011920 con 0x564f82ffc830 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcaba7fc640 1 -- 172.21.15.59:0/1122710147 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fcabc00f040 con 0x564f82ffc830 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.856+0000 7fcadbd7d740 10 client.16548 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.857+0000 7fcadbd7d740 10 client.16548 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:53.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.857+0000 7fcadbd7d740 20 client.16548 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.857+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564f83016ea0 0x564f83019290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.857+0000 7fcadbd7d740 4 client.16548 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:38:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.857+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1122710147 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f82f6f1e0 con 0x564f83016ea0 2024-05-08T05:38:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.857+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564f83016ea0 0x564f83019290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.857+0000 7fcaba7fc640 10 client.16548 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:53.863 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.857+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564f83016ea0 0x564f83019290 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27448cf56df30b0b server_cookie=952b7a82de44d76c in_seq=0 out_seq=0 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcaba7fc640 1 -- 172.21.15.59:0/1122710147 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x564f82f6f1e0 con 0x564f83016ea0 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcaba7fc640 10 client.16548 handle_command_reply: tid=0 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 client.16548 shutdown 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1122710147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 msgr2=0x564f830044b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f830044b0 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7fcabc000c00 tx=0x7fcabc0097d0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1122710147 shutdown_connections 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564f83016ea0 0x564f83019290 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f830070f0 0x564f830049f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f830044b0 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcaba7fc640 0 client.16548 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1122710147 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffbf10 0x564f83003f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1122710147 >> 172.21.15.59:0/1122710147 conn(0x564f82ff8420 msgr2=0x564f82ffe150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1122710147 shutdown_connections 2024-05-08T05:38:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1122710147 wait complete. 2024-05-08T05:38:53.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.858+0000 7fcadbd7d740 20 client.16548 trim_cache size 0 max 16384 2024-05-08T05:38:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.874+0000 7fcadbd7d740 1 Processor -- start 2024-05-08T05:38:53.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.874+0000 7fcadbd7d740 1 -- start start 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.874+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffbf10 0x564f830030d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.874+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffc830 0x564f82ff7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.874+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f8301e920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.874+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41afe10 con 0x564f830070f0 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.874+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b1950 con 0x564f82ffc830 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.874+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b04e0 con 0x564f82ffbf10 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.874+0000 7fcad359e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f8301e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcad359e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f8301e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42808/0 (socket says 172.21.15.59:42808) 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcad359e640 1 -- 172.21.15.59:0/1930659037 learned_addr learned my addr 172.21.15.59:0/1930659037 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcacbfff640 1 --2- 172.21.15.59:0/1930659037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffc830 0x564f82ff7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/1930659037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffbf10 0x564f830030d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcad359e640 1 -- 172.21.15.59:0/1930659037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffbf10 msgr2=0x564f830030d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcad359e640 1 --2- 172.21.15.59:0/1930659037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffbf10 0x564f830030d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcad359e640 1 -- 172.21.15.59:0/1930659037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffc830 msgr2=0x564f82ff7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcad359e640 1 --2- 172.21.15.59:0/1930659037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffc830 0x564f82ff7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcad359e640 1 -- 172.21.15.59:0/1930659037 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f82ffed80 con 0x564f830070f0 2024-05-08T05:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcad359e640 1 --2- 172.21.15.59:0/1930659037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f8301e920 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0076f0 tx=0x7fcacc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ead548cf07be2c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcac8ff9640 1 -- 172.21.15.59:0/1930659037 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc01e040 con 0x564f830070f0 2024-05-08T05:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcac8ff9640 1 -- 172.21.15.59:0/1930659037 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcacc027070 con 0x564f830070f0 2024-05-08T05:38:53.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.875+0000 7fcac8ff9640 1 -- 172.21.15.59:0/1930659037 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcacc022b90 con 0x564f830070f0 2024-05-08T05:38:53.880 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.876+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1930659037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 msgr2=0x564f8301e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.876+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1930659037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f8301e920 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7fcacc0076f0 tx=0x7fcacc014000 comp rx=0 tx=0).stop 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.876+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1930659037 shutdown_connections 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.876+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1930659037 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f830070f0 0x564f8301e920 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.876+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1930659037 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f82ffc830 0x564f82ff7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.876+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/1930659037 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffbf10 0x564f830030d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.876+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1930659037 >> 172.21.15.59:0/1930659037 conn(0x564f82ff8420 msgr2=0x564f82ff6cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.876+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1930659037 shutdown_connections 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.876+0000 7fcadbd7d740 1 -- 172.21.15.59:0/1930659037 wait complete. 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcadbd7d740 1 -- start start 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcadbd7d740 1 Processor -- start 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffbf10 0x564f8300ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f8300b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.881 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcadbd7d740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f830070f0 0x564f8300b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b1280 con 0x564f82ffc830 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41b0bb0 con 0x564f830070f0 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcadbd7d740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fcad41ade10 con 0x564f82ffbf10 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcadbd7d740 10 client.0 _ll_register_callbacks cb 0x7fcacc0096f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f8300b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcacbfff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f8300b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42814/0 (socket says 172.21.15.59:42814) 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcacbfff640 1 -- 172.21.15.59:0/2983883502 learned_addr learned my addr 172.21.15.59:0/2983883502 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcad359e640 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f830070f0 0x564f8300b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcacbfff640 1 -- 172.21.15.59:0/2983883502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffbf10 msgr2=0x564f8300ae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcacbfff640 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffbf10 0x564f8300ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcacbfff640 1 -- 172.21.15.59:0/2983883502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f830070f0 msgr2=0x564f8300b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcacbfff640 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f830070f0 0x564f8300b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.877+0000 7fcacbfff640 1 -- 172.21.15.59:0/2983883502 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f82ffd3e0 con 0x564f82ffc830 2024-05-08T05:38:53.882 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcacbfff640 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f8300b3b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x564f83005540 tx=0x7fcabc005ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deba124c992a2636 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcaba7fc640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcaba7fc640 1 -- 172.21.15.59:0/2983883502 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcabc037020 con 0x564f82ffc830 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcaba7fc640 1 -- 172.21.15.59:0/2983883502 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fcabc004ac0 con 0x564f82ffc830 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2983883502 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcacc027070 con 0x564f82ffc830 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcaba7fc640 1 -- 172.21.15.59:0/2983883502 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7fcabc042050 con 0x564f82ffc830 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcadbd7d740 10 client.16554 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2983883502 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x564f830817d0 con 0x564f82ffc830 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcaba7fc640 1 -- 172.21.15.59:0/2983883502 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcacc027070 con 0x564f82ffc830 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcaba7fc640 1 -- 172.21.15.59:0/2983883502 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7fcabc01b860 con 0x564f82ffc830 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcadbd7d740 10 client.16554 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcadbd7d740 10 client.16554 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcadbd7d740 20 client.16554 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564f83081eb0 0x564f830842d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:53.883 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcadbd7d740 4 client.16554 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:38:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.878+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2983883502 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564f82f69050 con 0x564f83081eb0 2024-05-08T05:38:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.879+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564f83081eb0 0x564f830842d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.879+0000 7fcaba7fc640 10 client.16554 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:53.884 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.879+0000 7fcad2d9d640 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564f83081eb0 0x564f830842d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29b25db2d3821b2a server_cookie=df096379c6a1a489 in_seq=0 out_seq=0 2024-05-08T05:38:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.890+0000 7fcaba7fc640 1 -- 172.21.15.59:0/2983883502 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564f82f69050 con 0x564f83081eb0 2024-05-08T05:38:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.890+0000 7fcaba7fc640 10 client.16554 handle_command_reply: tid=0 2024-05-08T05:38:53.891 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 client.16554 shutdown 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2983883502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 msgr2=0x564f8300b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f8300b3b0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x564f83005540 tx=0x7fcabc005ed0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2983883502 shutdown_connections 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x564f83081eb0 0x564f830842d0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x564f830070f0 0x564f8300b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x564f82ffc830 0x564f8300b3b0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcaba7fc640 0 client.16554 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 --2- 172.21.15.59:0/2983883502 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x564f82ffbf10 0x564f8300ae70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2983883502 >> 172.21.15.59:0/2983883502 conn(0x564f82ff8420 msgr2=0x564f82ff7580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2983883502 shutdown_connections 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 1 -- 172.21.15.59:0/2983883502 wait complete. 2024-05-08T05:38:53.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcadbd7d740 20 client.16554 trim_cache size 0 max 16384 2024-05-08T05:38:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:38:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:38:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:38:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:38:53.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcad9064640 1 -- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fcac00b2a10 msgr2=0x7fcac00b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcad9064640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fcac00b2a10 0x7fcac00b4ed0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcacc00c610 tx=0x7fcacc00e040 comp rx=0 tx=0).stop 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcad9064640 1 -- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41018e0 msgr2=0x7fcad4199210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.891+0000 7fcad9064640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41018e0 0x7fcad4199210 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7fcac4004770 tx=0x7fcac40042f0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.892+0000 7fcad9064640 1 -- 172.21.15.59:0/1508108595 shutdown_connections 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.892+0000 7fcad9064640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fcac00b2a10 0x7fcac00b4ed0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.892+0000 7fcad9064640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fcad41078a0 0x7fcad419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.892+0000 7fcad9064640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fcad41022a0 0x7fcad4199750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.892+0000 7fcad9064640 1 --2- 172.21.15.59:0/1508108595 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fcad41018e0 0x7fcad4199210 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.892+0000 7fcad9064640 1 -- 172.21.15.59:0/1508108595 >> 172.21.15.59:0/1508108595 conn(0x7fcad4078a30 msgr2=0x7fcad406a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.892+0000 7fcad9064640 1 -- 172.21.15.59:0/1508108595 shutdown_connections 2024-05-08T05:38:53.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:38:53.892+0000 7fcad9064640 1 -- 172.21.15.59:0/1508108595 wait complete. 2024-05-08T05:38:54.010 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:38:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:55 smithi143 ceph-mon[34337]: pgmap v290: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T05:38:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:55 smithi096 ceph-mon[34253]: pgmap v290: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T05:38:55.532 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:55 smithi059 ceph-mon[30304]: pgmap v290: 129 pgs: 129 active+clean; 92 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 9.7 KiB/s rd, 21 KiB/s wr, 10 op/s 2024-05-08T05:38:55.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.923+0000 7f3ab8f25700 1 -- 172.21.15.59:0/1947269708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 msgr2=0x7f3ab4108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:55.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.923+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/1947269708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 0x7f3ab4108120 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f3aa80099c0 tx=0x7f3aa8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.924+0000 7f3ab8f25700 1 -- 172.21.15.59:0/1947269708 shutdown_connections 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.924+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/1947269708 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ab4108850 0x7f3ab41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.924+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/1947269708 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ab4103750 0x7f3ab4103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.924+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/1947269708 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 0x7f3ab4108120 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.924+0000 7f3ab8f25700 1 -- 172.21.15.59:0/1947269708 >> 172.21.15.59:0/1947269708 conn(0x7f3ab407a600 msgr2=0x7f3ab407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.924+0000 7f3ab8f25700 1 -- 172.21.15.59:0/1947269708 shutdown_connections 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.924+0000 7f3ab8f25700 1 -- 172.21.15.59:0/1947269708 wait complete. 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.924+0000 7f3ab8f25700 1 Processor -- start 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.924+0000 7f3ab8f25700 1 -- start start 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab8f25700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ab4103750 0x7f3ab40757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab8f25700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 0x7f3ab4075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab8f25700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ab4108850 0x7f3ab4076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:55.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab8f25700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4113690 con 0x7f3ab4104100 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab8f25700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4113510 con 0x7f3ab4103750 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab8f25700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f3ab4113810 con 0x7f3ab4108850 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab1d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 0x7f3ab4075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab1d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 0x7f3ab4075ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:42844/0 (socket says 172.21.15.59:42844) 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab1d9b700 1 -- 172.21.15.59:0/801623917 learned_addr learned my addr 172.21.15.59:0/801623917 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab2d9d700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ab4108850 0x7f3ab4076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab259c700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ab4103750 0x7f3ab40757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab1d9b700 1 -- 172.21.15.59:0/801623917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ab4108850 msgr2=0x7f3ab4076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab1d9b700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ab4108850 0x7f3ab4076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab1d9b700 1 -- 172.21.15.59:0/801623917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ab4103750 msgr2=0x7f3ab40757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab1d9b700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ab4103750 0x7f3ab40757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab1d9b700 1 -- 172.21.15.59:0/801623917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ab41a8090 con 0x7f3ab4104100 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3ab1d9b700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 0x7f3ab4075ce0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800b580 tx=0x7f3aa80037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:55.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.925+0000 7f3aa37fe700 1 -- 172.21.15.59:0/801623917 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa8027070 con 0x7f3ab4104100 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.926+0000 7f3aa37fe700 1 -- 172.21.15.59:0/801623917 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3aa8004110 con 0x7f3ab4104100 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.926+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ab41a8320 con 0x7f3ab4104100 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.926+0000 7f3aa37fe700 1 -- 172.21.15.59:0/801623917 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3aa800f460 con 0x7f3ab4104100 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.926+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ab41a87d0 con 0x7f3ab4104100 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.927+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3ab406a190 con 0x7f3ab4104100 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.928+0000 7f3aa37fe700 1 -- 172.21.15.59:0/801623917 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f3aa802a070 con 0x7f3ab4104100 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.929+0000 7f3aa37fe700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f3a9c0b2a70 0x7f3a9c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.929+0000 7f3aa37fe700 1 -- 172.21.15.59:0/801623917 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f3aa80e3d30 con 0x7f3ab4104100 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.929+0000 7f3ab259c700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f3a9c0b2a70 0x7f3a9c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.929+0000 7f3ab259c700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f3a9c0b2a70 0x7f3a9c0b4f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4006fd0 tx=0x7f3aa4008040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:55.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:55.933+0000 7f3aa37fe700 1 -- 172.21.15.59:0/801623917 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3aa8031070 con 0x7f3ab4104100 2024-05-08T05:38:56.196 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.195+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f3ab402e6d0 con 0x7f3ab4104100 2024-05-08T05:38:56.197 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.196+0000 7f3aa37fe700 1 -- 172.21.15.59:0/801623917 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11929 (secure 0 0 0) 0x7f3aa8013070 con 0x7f3ab4104100 2024-05-08T05:38:56.197 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:38:56.197 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:36:43.047617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25876":{"gid":25876,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2351774265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2351774265},{"type":"v1","addr":"172.21.15.143:6835","nonce":2351774265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:38:56.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f3a9c0b2a70 msgr2=0x7f3a9c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:56.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f3a9c0b2a70 0x7f3a9c0b4f30 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f3aa4006fd0 tx=0x7f3aa4008040 comp rx=0 tx=0).stop 2024-05-08T05:38:56.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 msgr2=0x7f3ab4075ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:56.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 0x7f3ab4075ce0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f3aa800b580 tx=0x7f3aa80037c0 comp rx=0 tx=0).stop 2024-05-08T05:38:56.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 shutdown_connections 2024-05-08T05:38:56.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f3a9c0b2a70 0x7f3a9c0b4f30 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:56.199 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3ab4108850 0x7f3ab4076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:56.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3ab4103750 0x7f3ab40757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:56.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 --2- 172.21.15.59:0/801623917 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3ab4104100 0x7f3ab4075ce0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:56.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 >> 172.21.15.59:0/801623917 conn(0x7f3ab407a600 msgr2=0x7f3ab41017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:56.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 shutdown_connections 2024-05-08T05:38:56.200 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:56.199+0000 7f3ab8f25700 1 -- 172.21.15.59:0/801623917 wait complete. 2024-05-08T05:38:56.200 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 38 2024-05-08T05:38:57.005 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db --set-id d6c3f55e --release --await' 2024-05-08T05:38:57.006 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db --set-id d6c3f55e --release --await 2024-05-08T05:38:57.272 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:38:57.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:57 smithi059 ceph-mon[30304]: pgmap v291: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-05-08T05:38:57.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:57 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/801623917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:38:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:57 smithi143 ceph-mon[34337]: pgmap v291: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-05-08T05:38:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:57 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/801623917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:38:57.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:57 smithi096 ceph-mon[34253]: pgmap v291: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-05-08T05:38:57.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:57 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/801623917' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:38:59.108 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.106+0000 7f24dfb85700 1 -- 172.21.15.59:0/3430526472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d806c590 msgr2=0x7f24d8110380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.106+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3430526472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d806c590 0x7f24d8110380 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f24d000b0d0 tx=0x7f24d000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.106+0000 7f24dfb85700 1 -- 172.21.15.59:0/3430526472 shutdown_connections 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.106+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3430526472 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d806c590 0x7f24d8110380 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.106+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3430526472 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d81095b0 0x7f24d8109990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.106+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3430526472 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d806bbd0 0x7f24d806c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.106+0000 7f24dfb85700 1 -- 172.21.15.59:0/3430526472 >> 172.21.15.59:0/3430526472 conn(0x7f24d8078210 msgr2=0x7f24d8078620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.106+0000 7f24dfb85700 1 -- 172.21.15.59:0/3430526472 shutdown_connections 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.106+0000 7f24dfb85700 1 -- 172.21.15.59:0/3430526472 wait complete. 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24dfb85700 1 Processor -- start 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24dfb85700 1 -- start start 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24dfb85700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d806bbd0 0x7f24d819f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24dfb85700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d806c590 0x7f24d819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24dfb85700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d81095b0 0x7f24d8199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24dfb85700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8113710 con 0x7f24d81095b0 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24dfb85700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8113590 con 0x7f24d806bbd0 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24dfb85700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f24d8113890 con 0x7f24d806c590 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24de122700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d81095b0 0x7f24d8199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24de122700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d81095b0 0x7f24d8199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47148/0 (socket says 172.21.15.59:47148) 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24de122700 1 -- 172.21.15.59:0/3474837736 learned_addr learned my addr 172.21.15.59:0/3474837736 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24dd921700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d806bbd0 0x7f24d819f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24de122700 1 -- 172.21.15.59:0/3474837736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d806c590 msgr2=0x7f24d819fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24de122700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d806c590 0x7f24d819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24de122700 1 -- 172.21.15.59:0/3474837736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d806bbd0 msgr2=0x7f24d819f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24de122700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d806bbd0 0x7f24d819f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.107+0000 7f24de122700 1 -- 172.21.15.59:0/3474837736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24d819a0b0 con 0x7f24d81095b0 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.108+0000 7f24de122700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d81095b0 0x7f24d8199950 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f24d00079a0 tx=0x7f24d00079d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:59.110 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.108+0000 7f24ceffd700 1 -- 172.21.15.59:0/3474837736 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d001f070 con 0x7f24d81095b0 2024-05-08T05:38:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.108+0000 7f24ceffd700 1 -- 172.21.15.59:0/3474837736 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d0003c00 con 0x7f24d81095b0 2024-05-08T05:38:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.108+0000 7f24ceffd700 1 -- 172.21.15.59:0/3474837736 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d001b690 con 0x7f24d81095b0 2024-05-08T05:38:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.108+0000 7f24dfb85700 1 -- 172.21.15.59:0/3474837736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24d819a340 con 0x7f24d81095b0 2024-05-08T05:38:59.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.108+0000 7f24dfb85700 1 -- 172.21.15.59:0/3474837736 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24d81a6660 con 0x7f24d81095b0 2024-05-08T05:38:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.111+0000 7f24ceffd700 1 -- 172.21.15.59:0/3474837736 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f24d0022070 con 0x7f24d81095b0 2024-05-08T05:38:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.111+0000 7f24ceffd700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f24c40b2a70 0x7f24c40b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.111+0000 7f24dd921700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f24c40b2a70 0x7f24c40b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.111+0000 7f24ceffd700 1 -- 172.21.15.59:0/3474837736 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f24d00e36d0 con 0x7f24d81095b0 2024-05-08T05:38:59.112 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24ea418b80 1 Processor -- start 2024-05-08T05:38:59.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24ea418b80 1 -- start start 2024-05-08T05:38:59.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24dd921700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f24c40b2a70 0x7f24c40b4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24d819aa90 tx=0x7f24d4009260 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 0x55a36ad58f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 0x55a36ad52e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad537e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24ea418b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a36aca0a40 con 0x55a36ad533c0 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24ea418b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a36ac05ea0 con 0x55a36ad59440 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24ea418b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a36ac9ca00 con 0x55a36ad58b20 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24de122700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad537e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24de122700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad537e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47164/0 (socket says 172.21.15.59:47164) 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24de122700 1 -- 172.21.15.59:0/2536214749 learned_addr learned my addr 172.21.15.59:0/2536214749 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.112+0000 7f24dd120700 1 --2- 172.21.15.59:0/2536214749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 0x55a36ad52e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.113+0000 7f24de122700 1 -- 172.21.15.59:0/2536214749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 msgr2=0x55a36ad58f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.113+0000 7f24de122700 1 --2- 172.21.15.59:0/2536214749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 0x55a36ad58f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.113+0000 7f24de122700 1 -- 172.21.15.59:0/2536214749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 msgr2=0x55a36ad52e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:59.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.113+0000 7f24de122700 1 --2- 172.21.15.59:0/2536214749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 0x55a36ad52e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.113+0000 7f24de122700 1 -- 172.21.15.59:0/2536214749 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36abed9d0 con 0x55a36ad533c0 2024-05-08T05:38:59.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.113+0000 7f24de122700 1 --2- 172.21.15.59:0/2536214749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad537e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f24d00aa600 tx=0x7f24d00099e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a564a1784dc306e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:59.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.113+0000 7f24c3fff700 1 -- 172.21.15.59:0/2536214749 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d0029030 con 0x55a36ad533c0 2024-05-08T05:38:59.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.113+0000 7f24c3fff700 1 -- 172.21.15.59:0/2536214749 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d00aa670 con 0x55a36ad533c0 2024-05-08T05:38:59.115 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.113+0000 7f24c3fff700 1 -- 172.21.15.59:0/2536214749 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d00aa8d0 con 0x55a36ad533c0 2024-05-08T05:38:59.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 -- 172.21.15.59:0/2536214749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 msgr2=0x55a36ad537e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:59.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 --2- 172.21.15.59:0/2536214749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad537e0 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f24d00aa600 tx=0x7f24d00099e0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 -- 172.21.15.59:0/2536214749 shutdown_connections 2024-05-08T05:38:59.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 --2- 172.21.15.59:0/2536214749 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad537e0 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 --2- 172.21.15.59:0/2536214749 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 0x55a36ad58f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 --2- 172.21.15.59:0/2536214749 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 0x55a36ad52e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.119 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 -- 172.21.15.59:0/2536214749 >> 172.21.15.59:0/2536214749 conn(0x55a36ac7ac70 msgr2=0x55a36ad497f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 -- 172.21.15.59:0/2536214749 shutdown_connections 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 -- 172.21.15.59:0/2536214749 wait complete. 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 -- start start 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.114+0000 7f24ea418b80 1 Processor -- start 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad533c0 0x55a36ad5f180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad58b20 0x55a36ad5c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 0x55a36ad5ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24ea418b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55a36ad5fba0 con 0x55a36ad58b20 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24ea418b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55a36ac59fd0 con 0x55a36ad533c0 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24ea418b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55a36ad5fd20 con 0x55a36ad59440 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24ea418b80 10 client.0 _ll_register_callbacks cb 0x55a36ac346b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd120700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad58b20 0x55a36ad5c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd120700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad58b20 0x55a36ad5c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47172/0 (socket says 172.21.15.59:47172) 2024-05-08T05:38:59.120 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd120700 1 -- 172.21.15.59:0/86561615 learned_addr learned my addr 172.21.15.59:0/86561615 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd921700 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad533c0 0x55a36ad5f180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24de122700 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 0x55a36ad5ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd120700 1 -- 172.21.15.59:0/86561615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 msgr2=0x55a36ad5ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd120700 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 0x55a36ad5ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd120700 1 -- 172.21.15.59:0/86561615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad533c0 msgr2=0x55a36ad5f180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd120700 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad533c0 0x55a36ad5f180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd120700 1 -- 172.21.15.59:0/86561615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36ac57860 con 0x55a36ad58b20 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24de122700 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 0x55a36ad5ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.115+0000 7f24dd921700 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad533c0 0x55a36ad5f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24dd120700 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad58b20 0x55a36ad5c520 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f24c8003930 tx=0x7f24c8003cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2edf2d5ac0b91f09 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24c17fa700 1 -- 172.21.15.59:0/86561615 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24c801a470 con 0x55a36ad58b20 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24ea418b80 1 -- 172.21.15.59:0/86561615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d00aa670 con 0x55a36ad58b20 2024-05-08T05:38:59.121 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24c17fa700 1 -- 172.21.15.59:0/86561615 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f24c801a610 con 0x55a36ad58b20 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24c17fa700 1 -- 172.21.15.59:0/86561615 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f24c8023050 con 0x55a36ad58b20 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24ea418b80 10 client.16572 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24ea418b80 1 -- 172.21.15.59:0/86561615 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a36ad6a470 con 0x55a36ad58b20 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24c17fa700 1 -- 172.21.15.59:0/86561615 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d00aa670 con 0x55a36ad58b20 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24c17fa700 1 -- 172.21.15.59:0/86561615 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f24c80117e0 con 0x55a36ad58b20 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24ea418b80 10 client.16572 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24ea418b80 10 client.16572 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24ea418b80 20 client.16572 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24ea418b80 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a36ad6cab0 0x55a36ad6eea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24ea418b80 4 client.16572 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.116+0000 7f24ea418b80 1 -- 172.21.15.59:0/86561615 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a36acae420 con 0x55a36ad6cab0 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.117+0000 7f24dd921700 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a36ad6cab0 0x55a36ad6eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.117+0000 7f24c17fa700 10 client.16572 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.117+0000 7f24dd921700 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a36ad6cab0 0x55a36ad6eea0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=507dad3713f8900f server_cookie=461ec1e0021af946 in_seq=0 out_seq=0 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24c17fa700 1 -- 172.21.15.59:0/86561615 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55a36acae420 con 0x55a36ad6cab0 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24c17fa700 10 client.16572 handle_command_reply: tid=0 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 client.16572 shutdown 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 -- 172.21.15.59:0/86561615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad58b20 msgr2=0x55a36ad5c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad58b20 0x55a36ad5c520 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f24c8003930 tx=0x7f24c8003cf0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 -- 172.21.15.59:0/86561615 shutdown_connections 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a36ad6cab0 0x55a36ad6eea0 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 0x55a36ad5ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad533c0 0x55a36ad5f180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 --2- 172.21.15.59:0/86561615 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad58b20 0x55a36ad5c520 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24c17fa700 0 client.16572 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 -- 172.21.15.59:0/86561615 >> 172.21.15.59:0/86561615 conn(0x55a36ac7ac70 msgr2=0x55a36ad43d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 -- 172.21.15.59:0/86561615 shutdown_connections 2024-05-08T05:38:59.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 1 -- 172.21.15.59:0/86561615 wait complete. 2024-05-08T05:38:59.124 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.118+0000 7f24ea418b80 20 client.16572 trim_cache size 0 max 16384 2024-05-08T05:38:59.139 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.138+0000 7f24ea418b80 1 Processor -- start 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.138+0000 7f24ea418b80 1 -- start start 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.138+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad68b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.138+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad58b20 0x55a36ad43ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.138+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 0x55a36ad59e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.138+0000 7f24ea418b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81ad1d0 con 0x55a36ad533c0 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.138+0000 7f24ea418b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81aed10 con 0x55a36ad58b20 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.138+0000 7f24ea418b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81ad8a0 con 0x55a36ad59440 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd921700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad68b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd921700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad68b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47188/0 (socket says 172.21.15.59:47188) 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd921700 1 -- 172.21.15.59:0/910428514 learned_addr learned my addr 172.21.15.59:0/910428514 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd921700 1 -- 172.21.15.59:0/910428514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 msgr2=0x55a36ad59e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:38:59.140 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd921700 1 --2- 172.21.15.59:0/910428514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 0x55a36ad59e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd120700 1 --2- 172.21.15.59:0/910428514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad58b20 0x55a36ad43ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd921700 1 -- 172.21.15.59:0/910428514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad58b20 msgr2=0x55a36ad43ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd921700 1 --2- 172.21.15.59:0/910428514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad58b20 0x55a36ad43ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd921700 1 -- 172.21.15.59:0/910428514 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a36ad6f480 con 0x55a36ad533c0 2024-05-08T05:38:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24dd921700 1 --2- 172.21.15.59:0/910428514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad68b60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55a36ad6f680 tx=0x7f24d4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4a7abde256da261 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24c37fe700 1 -- 172.21.15.59:0/910428514 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4011e60 con 0x55a36ad533c0 2024-05-08T05:38:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24c37fe700 1 -- 172.21.15.59:0/910428514 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d40044e0 con 0x55a36ad533c0 2024-05-08T05:38:59.141 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.139+0000 7f24c37fe700 1 -- 172.21.15.59:0/910428514 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4003680 con 0x55a36ad533c0 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 -- 172.21.15.59:0/910428514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 msgr2=0x55a36ad68b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 --2- 172.21.15.59:0/910428514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad68b60 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x55a36ad6f680 tx=0x7f24d4014000 comp rx=0 tx=0).stop 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 -- 172.21.15.59:0/910428514 shutdown_connections 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 --2- 172.21.15.59:0/910428514 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad59440 0x55a36ad59e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 --2- 172.21.15.59:0/910428514 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad68b60 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 --2- 172.21.15.59:0/910428514 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad58b20 0x55a36ad43ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 -- 172.21.15.59:0/910428514 >> 172.21.15.59:0/910428514 conn(0x55a36ac7ac70 msgr2=0x55a36ad4e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 -- 172.21.15.59:0/910428514 shutdown_connections 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 -- 172.21.15.59:0/910428514 wait complete. 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 -- start start 2024-05-08T05:38:59.144 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.140+0000 7f24ea418b80 1 Processor -- start 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad61eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 0x55a36ad623f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24ea418b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 0x55a36add72d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24ea418b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81ae640 con 0x55a36ad533c0 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24ea418b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81adf70 con 0x55a36ad59440 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24ea418b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f24d81ab1b0 con 0x55a36ad58b20 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24ea418b80 10 client.0 _ll_register_callbacks cb 0x55a36ac346b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd921700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad61eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd921700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad61eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47194/0 (socket says 172.21.15.59:47194) 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd921700 1 -- 172.21.15.59:0/1934689492 learned_addr learned my addr 172.21.15.59:0/1934689492 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24de122700 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 0x55a36add72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd120700 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 0x55a36ad623f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd921700 1 -- 172.21.15.59:0/1934689492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 msgr2=0x55a36ad623f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd921700 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 0x55a36ad623f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd921700 1 -- 172.21.15.59:0/1934689492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 msgr2=0x55a36add72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:38:59.145 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd921700 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 0x55a36add72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd921700 1 -- 172.21.15.59:0/1934689492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a36ad59150 con 0x55a36ad533c0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd120700 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 0x55a36ad623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24dd921700 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad61eb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009a00 tx=0x7f24d4021b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71becceb84057b7e server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.141+0000 7f24c17fa700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24c17fa700 1 -- 172.21.15.59:0/1934689492 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24d4036020 con 0x55a36ad533c0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24c17fa700 1 -- 172.21.15.59:0/1934689492 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f24d4028330 con 0x55a36ad533c0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24ea418b80 1 -- 172.21.15.59:0/1934689492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f24d40044e0 con 0x55a36ad533c0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24c17fa700 1 -- 172.21.15.59:0/1934689492 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=38) ==== 24+0+0 (secure 0 0 0) 0x7f24d4041050 con 0x55a36ad533c0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24ea418b80 10 client.16578 fetch_fsmap learned FSMap version 38 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24ea418b80 1 -- 172.21.15.59:0/1934689492 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=38,monmap=4+}) -- 0x55a36add8870 con 0x55a36ad533c0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24c17fa700 1 -- 172.21.15.59:0/1934689492 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f24d40044e0 con 0x55a36ad533c0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24c17fa700 1 -- 172.21.15.59:0/1934689492 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 38) ==== 8362+0+0 (secure 0 0 0) 0x7f24d40288e0 con 0x55a36ad533c0 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24ea418b80 10 client.16578 fetch_fsmap finished waiting for FSMap version 38 2024-05-08T05:38:59.146 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24ea418b80 10 client.16578 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:38:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24ea418b80 20 client.16578 populate_metadata read hostname 'smithi059' 2024-05-08T05:38:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24ea418b80 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a36add8ee0 0x55a36addb300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:38:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24ea418b80 4 client.16578 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "d6c3f55e", "release": true, "await": true}] 2024-05-08T05:38:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.142+0000 7f24ea418b80 1 -- 172.21.15.59:0/1934689492 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "d6c3f55e", "release": true, "await": true}) -- 0x55a36aca84c0 con 0x55a36add8ee0 2024-05-08T05:38:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.143+0000 7f24dd120700 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a36add8ee0 0x55a36addb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:38:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.143+0000 7f24dd120700 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a36add8ee0 0x55a36addb300 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d7a10efdbf50b72 server_cookie=a1d4960c903e784d in_seq=0 out_seq=0 2024-05-08T05:38:59.147 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.143+0000 7f24c17fa700 10 client.16578 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:59.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:38:59 smithi059 ceph-mon[30304]: pgmap v292: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-05-08T05:38:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24c17fa700 1 -- 172.21.15.59:0/1934689492 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55a36aca84c0 con 0x55a36add8ee0 2024-05-08T05:38:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24c17fa700 10 client.16578 handle_command_reply: tid=0 2024-05-08T05:38:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 client.16578 shutdown 2024-05-08T05:38:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 -- 172.21.15.59:0/1934689492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 msgr2=0x55a36ad61eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad61eb0 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f24d4009a00 tx=0x7f24d4021b90 comp rx=0 tx=0).stop 2024-05-08T05:38:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 -- 172.21.15.59:0/1934689492 shutdown_connections 2024-05-08T05:38:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55a36add8ee0 0x55a36addb300 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55a36ad59440 0x55a36add72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.347 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55a36ad533c0 0x55a36ad61eb0 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 --2- 172.21.15.59:0/1934689492 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55a36ad58b20 0x55a36ad623f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24c17fa700 0 client.16578 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 -- 172.21.15.59:0/1934689492 >> 172.21.15.59:0/1934689492 conn(0x55a36ac7ac70 msgr2=0x55a36ad4da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 -- 172.21.15.59:0/1934689492 shutdown_connections 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 1 -- 172.21.15.59:0/1934689492 wait complete. 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.346+0000 7f24ea418b80 20 client.16578 trim_cache size 0 max 16384 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 38, 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:38:59.348 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 28, 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "d6c3f55e": { 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 28, 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 90.0, 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:38:59.349 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:38:59.350 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:38:59.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:59.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:59.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:59.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:59.350 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:38:59.350 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:38:59.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 -- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f24c40b2a70 msgr2=0x7f24c40b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:59.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f24c40b2a70 0x7f24c40b4f30 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f24d819aa90 tx=0x7f24d4009260 comp rx=0 tx=0).stop 2024-05-08T05:38:59.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 -- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d81095b0 msgr2=0x7f24d8199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:38:59.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d81095b0 0x7f24d8199950 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7f24d00079a0 tx=0x7f24d00079d0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 -- 172.21.15.59:0/3474837736 shutdown_connections 2024-05-08T05:38:59.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f24c40b2a70 0x7f24c40b4f30 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f24d81095b0 0x7f24d8199950 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.351 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f24d806bbd0 0x7f24d819f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 --2- 172.21.15.59:0/3474837736 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f24d806c590 0x7f24d819fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:38:59.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 -- 172.21.15.59:0/3474837736 >> 172.21.15.59:0/3474837736 conn(0x7f24d8078210 msgr2=0x7f24d8101340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:38:59.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 -- 172.21.15.59:0/3474837736 shutdown_connections 2024-05-08T05:38:59.352 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:38:59.347+0000 7f24dfb85700 1 -- 172.21.15.59:0/3474837736 wait complete. 2024-05-08T05:38:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:38:59 smithi143 ceph-mon[34337]: pgmap v292: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-05-08T05:38:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:38:59 smithi096 ceph-mon[34253]: pgmap v292: 129 pgs: 129 active+clean; 93 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 5.9 KiB/s wr, 9 op/s 2024-05-08T05:39:00.045 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'd6c3f55e', seconds elapsed: 45.0 2024-05-08T05:39:00.045 DEBUG:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 30.0 2024-05-08T05:39:00.046 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:39:00.413 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:39:01.151 INFO:tasks.workunit.client.0.smithi059.stdout:All 50000 operations completed A-OK! 2024-05-08T05:39:01.156 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-05-08T05:39:01.157 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-05-08T05:39:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:01 smithi143 ceph-mon[34337]: pgmap v293: 129 pgs: 129 active+clean; 112 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-05-08T05:39:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:01 smithi096 ceph-mon[34253]: pgmap v293: 129 pgs: 129 active+clean; 112 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-05-08T05:39:01.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:01 smithi059 ceph-mon[30304]: pgmap v293: 129 pgs: 129 active+clean; 112 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 521 KiB/s rd, 82 KiB/s wr, 34 op/s 2024-05-08T05:39:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.316+0000 7f533ef7f700 1 -- 172.21.15.59:0/1155131001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5338102d60 msgr2=0x7f533810f250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.316+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1155131001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5338102d60 0x7f533810f250 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f533000b0a0 tx=0x7f533000b3b0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.316+0000 7f533ef7f700 1 -- 172.21.15.59:0/1155131001 shutdown_connections 2024-05-08T05:39:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.316+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1155131001 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f5338102d60 0x7f533810f250 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.316+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1155131001 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338108410 0x7f53381087f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.316+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1155131001 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f53381023a0 0x7f5338102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.316+0000 7f533ef7f700 1 -- 172.21.15.59:0/1155131001 >> 172.21.15.59:0/1155131001 conn(0x7f5338079350 msgr2=0x7f5338079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.316+0000 7f533ef7f700 1 -- 172.21.15.59:0/1155131001 shutdown_connections 2024-05-08T05:39:02.319 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.316+0000 7f533ef7f700 1 -- 172.21.15.59:0/1155131001 wait complete. 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533ef7f700 1 Processor -- start 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533ef7f700 1 -- start start 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533ef7f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53381023a0 0x7f533819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533ef7f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338102d60 0x7f533819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533ef7f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5338108410 0x7f5338199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533ef7f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f5338112560 con 0x7f53381023a0 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533ef7f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f53381123e0 con 0x7f5338102d60 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533ef7f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f53381126e0 con 0x7f5338108410 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f5337fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338102d60 0x7f533819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533cd1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53381023a0 0x7f533819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f5337fff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338102d60 0x7f533819fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:34244/0 (socket says 172.21.15.59:34244) 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f533cd1b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53381023a0 0x7f533819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47206/0 (socket says 172.21.15.59:47206) 2024-05-08T05:39:02.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.317+0000 7f5337fff700 1 -- 172.21.15.59:0/1978391406 learned_addr learned my addr 172.21.15.59:0/1978391406 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f533d51c700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5338108410 0x7f5338199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f5337fff700 1 -- 172.21.15.59:0/1978391406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5338108410 msgr2=0x7f5338199960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f5337fff700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5338108410 0x7f5338199960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f5337fff700 1 -- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53381023a0 msgr2=0x7f533819f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f5337fff700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53381023a0 0x7f533819f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f5337fff700 1 -- 172.21.15.59:0/1978391406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f533819a0c0 con 0x7f5338102d60 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f533d51c700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5338108410 0x7f5338199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f533cd1b700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53381023a0 0x7f533819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f5337fff700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338102d60 0x7f533819fc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f532000d980 tx=0x7f532000dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f5335ffb700 1 -- 172.21.15.59:0/1978391406 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320017070 con 0x7f5338102d60 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f533819a3b0 con 0x7f5338102d60 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f5335ffb700 1 -- 172.21.15.59:0/1978391406 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5320005020 con 0x7f5338102d60 2024-05-08T05:39:02.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f5335ffb700 1 -- 172.21.15.59:0/1978391406 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5320009b50 con 0x7f5338102d60 2024-05-08T05:39:02.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.318+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53381a6620 con 0x7f5338102d60 2024-05-08T05:39:02.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.319+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f533806a190 con 0x7f5338102d60 2024-05-08T05:39:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.321+0000 7f5335ffb700 1 -- 172.21.15.59:0/1978391406 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f53200051d0 con 0x7f5338102d60 2024-05-08T05:39:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.321+0000 7f5335ffb700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f53240b29f0 0x7f53240b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.322+0000 7f533cd1b700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f53240b29f0 0x7f53240b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.322+0000 7f5335ffb700 1 -- 172.21.15.59:0/1978391406 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f53200e2d80 con 0x7f5338102d60 2024-05-08T05:39:02.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.322+0000 7f533cd1b700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f53240b29f0 0x7f53240b4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5338100bf0 tx=0x7f5328009260 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:02.329 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.329+0000 7f5335ffb700 1 -- 172.21.15.59:0/1978391406 <== mon.2 v2:172.21.15.96:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f5320020020 con 0x7f5338102d60 2024-05-08T05:39:02.604 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.603+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f53381a6850 con 0x7f5338102d60 2024-05-08T05:39:02.605 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.604+0000 7f5335ffb700 1 -- 172.21.15.59:0/1978391406 <== mon.2 v2:172.21.15.96:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 38 v38) ==== 94+0+11929 (secure 0 0 0) 0x7f53200aa330 con 0x7f5338102d60 2024-05-08T05:39:02.605 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:39:02.605 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":38,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":38,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:36:43.047617+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25876":{"gid":25876,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2351774265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2351774265},{"type":"v1","addr":"172.21.15.143:6835","nonce":2351774265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24473,24443,24421,24419,24433]},"id":1}]} 2024-05-08T05:39:02.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f53240b29f0 msgr2=0x7f53240b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:02.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f53240b29f0 0x7f53240b4eb0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f5338100bf0 tx=0x7f5328009260 comp rx=0 tx=0).stop 2024-05-08T05:39:02.607 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338102d60 msgr2=0x7f533819fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338102d60 0x7f533819fc90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f532000d980 tx=0x7f532000dc90 comp rx=0 tx=0).stop 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 shutdown_connections 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f53240b29f0 0x7f53240b4eb0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f5338108410 0x7f5338199960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f53381023a0 0x7f533819f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 --2- 172.21.15.59:0/1978391406 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f5338102d60 0x7f533819fc90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 >> 172.21.15.59:0/1978391406 conn(0x7f5338079350 msgr2=0x7f5338101680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 shutdown_connections 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:02.607+0000 7f533ef7f700 1 -- 172.21.15.59:0/1978391406 wait complete. 2024-05-08T05:39:02.608 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 38 2024-05-08T05:39:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:03 smithi143 ceph-mon[34337]: pgmap v294: 129 pgs: 129 active+clean; 112 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 81 KiB/s wr, 30 op/s 2024-05-08T05:39:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:03 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1978391406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:39:03.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:03 smithi059 ceph-mon[30304]: pgmap v294: 129 pgs: 129 active+clean; 112 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 81 KiB/s wr, 30 op/s 2024-05-08T05:39:03.499 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:03 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1978391406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:39:03.527 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0' 2024-05-08T05:39:03.527 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db / --timeout 60 --expiration 90.0 --await-for 42.0 2024-05-08T05:39:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:03 smithi096 ceph-mon[34253]: pgmap v294: 129 pgs: 129 active+clean; 112 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 519 KiB/s rd, 81 KiB/s wr, 30 op/s 2024-05-08T05:39:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:03 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1978391406' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:39:03.785 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:39:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:04 smithi143 ceph-mon[34337]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T05:39:04.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:04 smithi096 ceph-mon[34253]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T05:39:04.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:04 smithi059 ceph-mon[30304]: osdmap e99: 12 total, 12 up, 12 in 2024-05-08T05:39:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:05 smithi143 ceph-mon[34337]: pgmap v296: 129 pgs: 129 active+clean; 112 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-05-08T05:39:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:05 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:39:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:05 smithi096 ceph-mon[34253]: pgmap v296: 129 pgs: 129 active+clean; 112 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-05-08T05:39:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:05 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:39:05.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:05 smithi059 ceph-mon[30304]: pgmap v296: 129 pgs: 129 active+clean; 112 MiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 622 KiB/s rd, 97 KiB/s wr, 36 op/s 2024-05-08T05:39:05.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:05 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:39:05.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.994+0000 7f91123cc700 1 -- 172.21.15.59:0/551161402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06bb70 msgr2=0x7f910c06bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:05.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.994+0000 7f91123cc700 1 --2- 172.21.15.59:0/551161402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06bb70 0x7f910c06bfd0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f90f40099c0 tx=0x7f90f4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:39:05.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.994+0000 7f91123cc700 1 -- 172.21.15.59:0/551161402 shutdown_connections 2024-05-08T05:39:05.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.994+0000 7f91123cc700 1 --2- 172.21.15.59:0/551161402 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f910c06c510 0x7f910c1103a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:05.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.994+0000 7f91123cc700 1 --2- 172.21.15.59:0/551161402 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f910c1084b0 0x7f910c108890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:05.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.994+0000 7f91123cc700 1 --2- 172.21.15.59:0/551161402 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06bb70 0x7f910c06bfd0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:05.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.994+0000 7f91123cc700 1 -- 172.21.15.59:0/551161402 >> 172.21.15.59:0/551161402 conn(0x7f910c079350 msgr2=0x7f910c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:05.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.994+0000 7f91123cc700 1 -- 172.21.15.59:0/551161402 shutdown_connections 2024-05-08T05:39:05.995 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.994+0000 7f91123cc700 1 -- 172.21.15.59:0/551161402 wait complete. 2024-05-08T05:39:05.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f91123cc700 1 Processor -- start 2024-05-08T05:39:05.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f91123cc700 1 -- start start 2024-05-08T05:39:05.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f91123cc700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f910c06bb70 0x7f910c19f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:05.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f91123cc700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06c510 0x7f910c19fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:05.997 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f91123cc700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f910c1084b0 0x7f910c199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f91123cc700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f910c113690 con 0x7f910c06c510 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f91123cc700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f910c113510 con 0x7f910c06bb70 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f91123cc700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f910c113810 con 0x7f910c1084b0 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f9103fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06c510 0x7f910c19fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f910bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f910c06bb70 0x7f910c19f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f9103fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06c510 0x7f910c19fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47228/0 (socket says 172.21.15.59:47228) 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f9103fff700 1 -- 172.21.15.59:0/919160997 learned_addr learned my addr 172.21.15.59:0/919160997 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.995+0000 7f9110969700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f910c1084b0 0x7f910c199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f910bfff700 1 -- 172.21.15.59:0/919160997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f910c1084b0 msgr2=0x7f910c199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f910bfff700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f910c1084b0 0x7f910c199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f910bfff700 1 -- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06c510 msgr2=0x7f910c19fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f910bfff700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06c510 0x7f910c19fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:05.998 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f910bfff700 1 -- 172.21.15.59:0/919160997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f910c19a070 con 0x7f910c06bb70 2024-05-08T05:39:05.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f9103fff700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06c510 0x7f910c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:39:05.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f910bfff700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f910c06bb70 0x7f910c19f750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00e9c0 tx=0x7f90fc00ecd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:05.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f9109ffb700 1 -- 172.21.15.59:0/919160997 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc004220 con 0x7f910c06bb70 2024-05-08T05:39:05.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f9109ffb700 1 -- 172.21.15.59:0/919160997 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90fc0043c0 con 0x7f910c06bb70 2024-05-08T05:39:05.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f9109ffb700 1 -- 172.21.15.59:0/919160997 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc003c50 con 0x7f910c06bb70 2024-05-08T05:39:05.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f91123cc700 1 -- 172.21.15.59:0/919160997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f910c19a360 con 0x7f910c06bb70 2024-05-08T05:39:05.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.996+0000 7f91123cc700 1 -- 172.21.15.59:0/919160997 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f910c1a6620 con 0x7f910c06bb70 2024-05-08T05:39:05.999 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:05.999+0000 7f9109ffb700 1 -- 172.21.15.59:0/919160997 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f90fc00c7c0 con 0x7f910c06bb70 2024-05-08T05:39:06.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.000+0000 7f9109ffb700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f90f80b2b20 0x7f90f80b4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.000 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.000+0000 7f9103fff700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f90f80b2b20 0x7f90f80b4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.001 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.000+0000 7f9109ffb700 1 -- 172.21.15.59:0/919160997 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f90fc0e2ae0 con 0x7f910c06bb70 2024-05-08T05:39:06.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f911cc5fb80 1 Processor -- start 2024-05-08T05:39:06.003 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f9103fff700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f90f80b2b20 0x7f90f80b4fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f910c079870 tx=0x7f90f4005960 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f911cc5fb80 1 -- start start 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e93410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e8e830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 0x55d356e8f2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d356de09f0 con 0x55d356e93950 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d356d7f950 con 0x55d356e8ee50 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d356ddc9b0 con 0x55d356e93030 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f9103fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e8e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f910bfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e93410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f9103fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e8e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47230/0 (socket says 172.21.15.59:47230) 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f9103fff700 1 -- 172.21.15.59:0/181968808 learned_addr learned my addr 172.21.15.59:0/181968808 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f9110969700 1 --2- 172.21.15.59:0/181968808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 0x55d356e8f2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.004 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f910bfff700 1 -- 172.21.15.59:0/181968808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 msgr2=0x55d356e8f2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f910bfff700 1 --2- 172.21.15.59:0/181968808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 0x55d356e8f2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f910bfff700 1 -- 172.21.15.59:0/181968808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 msgr2=0x55d356e8e830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f910bfff700 1 --2- 172.21.15.59:0/181968808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e8e830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f910bfff700 1 -- 172.21.15.59:0/181968808 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d356d2da60 con 0x55d356e93030 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.001+0000 7f9103fff700 1 --2- 172.21.15.59:0/181968808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e8e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f910bfff700 1 --2- 172.21.15.59:0/181968808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e93410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0dfdd0 tx=0x7f90fc0045f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f8efa4b1168fffa5 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f91027fc700 1 -- 172.21.15.59:0/181968808 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc0a9960 con 0x55d356e93030 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f91027fc700 1 -- 172.21.15.59:0/181968808 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90fc0add10 con 0x55d356e93030 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f91027fc700 1 -- 172.21.15.59:0/181968808 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90fc0a9bc0 con 0x55d356e93030 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f911cc5fb80 1 -- 172.21.15.59:0/181968808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 msgr2=0x55d356e93410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/181968808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e93410 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f90fc0dfdd0 tx=0x7f90fc0045f0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f911cc5fb80 1 -- 172.21.15.59:0/181968808 shutdown_connections 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/181968808 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 0x55d356e8f2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.005 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/181968808 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e93410 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/181968808 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e8e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.002+0000 7f911cc5fb80 1 -- 172.21.15.59:0/181968808 >> 172.21.15.59:0/181968808 conn(0x55d356dba290 msgr2=0x55d356e89070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:06.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.003+0000 7f911cc5fb80 1 -- 172.21.15.59:0/181968808 shutdown_connections 2024-05-08T05:39:06.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.003+0000 7f911cc5fb80 1 -- 172.21.15.59:0/181968808 wait complete. 2024-05-08T05:39:06.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.003+0000 7f911cc5fb80 1 -- start start 2024-05-08T05:39:06.006 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.003+0000 7f911cc5fb80 1 Processor -- start 2024-05-08T05:39:06.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.003+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 0x55d356e9a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.009 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93030 0x55d356e97b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93950 0x55d356e98060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55d356e9b5c0 con 0x55d356e93030 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55d356d46dc0 con 0x55d356e8ee50 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55d356e9b740 con 0x55d356e93950 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f911cc5fb80 10 client.0 _ll_register_callbacks cb 0x55d356d78ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9103fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93030 0x55d356e97b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f910bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 0x55d356e9a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9103fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93030 0x55d356e97b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47238/0 (socket says 172.21.15.59:47238) 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f910bfff700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 0x55d356e9a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:34292/0 (socket says 172.21.15.59:34292) 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9103fff700 1 -- 172.21.15.59:0/2874278665 learned_addr learned my addr 172.21.15.59:0/2874278665 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9110969700 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93950 0x55d356e98060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9103fff700 1 -- 172.21.15.59:0/2874278665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93950 msgr2=0x55d356e98060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:06.010 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9103fff700 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93950 0x55d356e98060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9103fff700 1 -- 172.21.15.59:0/2874278665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 msgr2=0x55d356e9a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9103fff700 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 0x55d356e9a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9103fff700 1 -- 172.21.15.59:0/2874278665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d356d7c820 con 0x55d356e93030 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f9103fff700 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93030 0x55d356e97b20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b580 tx=0x7f90f401a050 comp rx=0 tx=0).ready entity=mon.0 client_cookie=513fa9a590381447 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f90eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2874278665 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f90f4024820 con 0x55d356e93030 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2874278665 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f90f40249c0 con 0x55d356e93030 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.004+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2874278665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f90fc0add10 con 0x55d356e93030 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2874278665 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f90f403e050 con 0x55d356e93030 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f911cc5fb80 10 client.16596 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2874278665 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d356ea5e00 con 0x55d356e93030 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2874278665 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f90fc0add10 con 0x55d356e93030 2024-05-08T05:39:06.011 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2874278665 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f90f4030af0 con 0x55d356e93030 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f911cc5fb80 10 client.16596 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f911cc5fb80 10 client.16596 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f911cc5fb80 20 client.16596 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d356ea84e0 0x55d356eaa8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f911cc5fb80 4 client.16596 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2874278665 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d356e5a7e0 con 0x55d356ea84e0 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.005+0000 7f910bfff700 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d356ea84e0 0x55d356eaa8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.006+0000 7f90eb7fe700 10 client.16596 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.006+0000 7f910bfff700 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d356ea84e0 0x55d356eaa8d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39ceca69075afbf2 server_cookie=721972fd0134716a in_seq=0 out_seq=0 2024-05-08T05:39:06.012 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.007+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2874278665 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55d356e5a7e0 con 0x55d356ea84e0 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.007+0000 7f90eb7fe700 10 client.16596 handle_command_reply: tid=0 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.007+0000 7f911cc5fb80 1 client.16596 shutdown 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2874278665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93030 msgr2=0x55d356e97b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93030 0x55d356e97b20 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f90f400b580 tx=0x7f90f401a050 comp rx=0 tx=0).stop 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2874278665 shutdown_connections 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d356ea84e0 0x55d356eaa8d0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93950 0x55d356e98060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f90eb7fe700 0 client.16596 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e8ee50 0x55d356e9a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2874278665 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93030 0x55d356e97b20 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2874278665 >> 172.21.15.59:0/2874278665 conn(0x55d356dba290 msgr2=0x55d356e87e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:06.013 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2874278665 shutdown_connections 2024-05-08T05:39:06.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2874278665 wait complete. 2024-05-08T05:39:06.014 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.008+0000 7f911cc5fb80 20 client.16596 trim_cache size 0 max 16384 2024-05-08T05:39:06.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.032+0000 7f911cc5fb80 1 Processor -- start 2024-05-08T05:39:06.032 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.032+0000 7f911cc5fb80 1 -- start start 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e8ee50 0x55d356e98090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93030 0x55d356ea4680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e88540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec00ef00 con 0x55d356e93950 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec010a70 con 0x55d356e93030 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec00f5d0 con 0x55d356e8ee50 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f9110969700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e88540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f9110969700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e88540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47250/0 (socket says 172.21.15.59:47250) 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f9110969700 1 -- 172.21.15.59:0/1520814372 learned_addr learned my addr 172.21.15.59:0/1520814372 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.033+0000 7f910bfff700 1 --2- 172.21.15.59:0/1520814372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e8ee50 0x55d356e98090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.034+0000 7f9110969700 1 -- 172.21.15.59:0/1520814372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e8ee50 msgr2=0x55d356e98090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.034+0000 7f9110969700 1 --2- 172.21.15.59:0/1520814372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e8ee50 0x55d356e98090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.035 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.034+0000 7f9110969700 1 -- 172.21.15.59:0/1520814372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93030 msgr2=0x55d356ea4680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-05-08T05:39:06.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.034+0000 7f9110969700 1 --2- 172.21.15.59:0/1520814372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93030 0x55d356ea4680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.034+0000 7f9110969700 1 -- 172.21.15.59:0/1520814372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d356e89a10 con 0x55d356e93950 2024-05-08T05:39:06.036 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.034+0000 7f9110969700 1 --2- 172.21.15.59:0/1520814372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e88540 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9104009fc0 tx=0x7f9104007500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342d4f18c493610c server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.039+0000 7f9101ffb700 1 -- 172.21.15.59:0/1520814372 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104020470 con 0x55d356e93950 2024-05-08T05:39:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.039+0000 7f9101ffb700 1 -- 172.21.15.59:0/1520814372 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f910401b070 con 0x55d356e93950 2024-05-08T05:39:06.039 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.039+0000 7f9101ffb700 1 -- 172.21.15.59:0/1520814372 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91040179b0 con 0x55d356e93950 2024-05-08T05:39:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.040+0000 7f911cc5fb80 1 -- 172.21.15.59:0/1520814372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 msgr2=0x55d356e88540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.040+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/1520814372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e88540 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f9104009fc0 tx=0x7f9104007500 comp rx=0 tx=0).stop 2024-05-08T05:39:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.040+0000 7f911cc5fb80 1 -- 172.21.15.59:0/1520814372 shutdown_connections 2024-05-08T05:39:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.040+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/1520814372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e93950 0x55d356e88540 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.040+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/1520814372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e8ee50 0x55d356e98090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.040+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/1520814372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93030 0x55d356ea4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.041 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.040+0000 7f911cc5fb80 1 -- 172.21.15.59:0/1520814372 >> 172.21.15.59:0/1520814372 conn(0x55d356dba290 msgr2=0x55d356e87e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.043+0000 7f911cc5fb80 1 -- 172.21.15.59:0/1520814372 shutdown_connections 2024-05-08T05:39:06.043 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.043+0000 7f911cc5fb80 1 -- 172.21.15.59:0/1520814372 wait complete. 2024-05-08T05:39:06.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.044+0000 7f911cc5fb80 1 -- start start 2024-05-08T05:39:06.044 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.044+0000 7f911cc5fb80 1 Processor -- start 2024-05-08T05:39:06.045 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e8ee50 0x55d356ea0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e9b9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f911cc5fb80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93950 0x55d356e9bf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f910bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e8ee50 0x55d356ea0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f910bfff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e8ee50 0x55d356ea0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47260/0 (socket says 172.21.15.59:47260) 2024-05-08T05:39:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec0103a0 con 0x55d356e8ee50 2024-05-08T05:39:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec00fcd0 con 0x55d356e93950 2024-05-08T05:39:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f911cc5fb80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f90ec00cea0 con 0x55d356e93030 2024-05-08T05:39:06.046 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f9103fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e9b9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f9103fff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e9b9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:56834/0 (socket says 172.21.15.59:56834) 2024-05-08T05:39:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f9103fff700 1 -- 172.21.15.59:0/2239152168 learned_addr learned my addr 172.21.15.59:0/2239152168 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f9110969700 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93950 0x55d356e9bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.045+0000 7f911cc5fb80 10 client.0 _ll_register_callbacks cb 0x55d356d78ab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:06.047 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.046+0000 7f9110969700 1 -- 172.21.15.59:0/2239152168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 msgr2=0x55d356e9b9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.046+0000 7f9110969700 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e9b9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.046+0000 7f9110969700 1 -- 172.21.15.59:0/2239152168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e8ee50 msgr2=0x55d356ea0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.046+0000 7f9110969700 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e8ee50 0x55d356ea0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.046+0000 7f9110969700 1 -- 172.21.15.59:0/2239152168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d356e9c6f0 con 0x55d356e93950 2024-05-08T05:39:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.046+0000 7f9110969700 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93950 0x55d356e9bf00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55d356e96320 tx=0x7f9104017c00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16caa4eb48f8879b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:06.048 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.046+0000 7f910bfff700 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e8ee50 0x55d356ea0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:39:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.048+0000 7f90eb7fe700 10 client.0 ms_handle_connect on v2:172.21.15.96:3300/0 2024-05-08T05:39:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.048+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2239152168 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9104017e20 con 0x55d356e93950 2024-05-08T05:39:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.048+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2239152168 <== mon.2 v2:172.21.15.96:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f9104004ce0 con 0x55d356e93950 2024-05-08T05:39:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.048+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2239152168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f910401b070 con 0x55d356e93950 2024-05-08T05:39:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.048+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2239152168 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f9104020470 con 0x55d356e93950 2024-05-08T05:39:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.048+0000 7f911cc5fb80 10 client.26135 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.048+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2239152168 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55d356f13cf0 con 0x55d356e93950 2024-05-08T05:39:06.050 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.049+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2239152168 <== mon.2 v2:172.21.15.96:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f910401b070 con 0x55d356e93950 2024-05-08T05:39:06.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.049+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2239152168 <== mon.2 v2:172.21.15.96:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f9104005260 con 0x55d356e93950 2024-05-08T05:39:06.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.049+0000 7f911cc5fb80 10 client.26135 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:06.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.049+0000 7f911cc5fb80 10 client.26135 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:39:06.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.049+0000 7f911cc5fb80 20 client.26135 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:06.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.049+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d356ea84e0 0x55d356f16330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:06.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.049+0000 7f911cc5fb80 4 client.26135 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}] 2024-05-08T05:39:06.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.049+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2239152168 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 60.0, "expiration": 90.0, "await_for": 42.0}) -- 0x55d356dee3d0 con 0x55d356ea84e0 2024-05-08T05:39:06.051 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.050+0000 7f910bfff700 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d356ea84e0 0x55d356f16330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:06.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.051+0000 7f910bfff700 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d356ea84e0 0x55d356f16330 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dbcd75a9acbec97 server_cookie=6bb175467050b097 in_seq=0 out_seq=0 2024-05-08T05:39:06.052 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:06.051+0000 7f90eb7fe700 10 client.26135 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.036+0000 7f1404d78640 1 -- 172.21.15.59:0/3051964881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13f8098180 msgr2=0x7f13f8098560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.037 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.036+0000 7f1404d78640 1 --2- 172.21.15.59:0/3051964881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13f8098180 0x7f13f8098560 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009900 tx=0x7f13ec02a400 comp rx=0 tx=0).stop 2024-05-08T05:39:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.037+0000 7f1404d78640 1 -- 172.21.15.59:0/3051964881 shutdown_connections 2024-05-08T05:39:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.037+0000 7f1404d78640 1 --2- 172.21.15.59:0/3051964881 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13f809d250 0x7f13f80a4e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.037+0000 7f1404d78640 1 --2- 172.21.15.59:0/3051964881 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f8098b30 0x7f13f809cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.037+0000 7f1404d78640 1 --2- 172.21.15.59:0/3051964881 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13f8098180 0x7f13f8098560 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.037+0000 7f1404d78640 1 -- 172.21.15.59:0/3051964881 >> 172.21.15.59:0/3051964881 conn(0x7f13f80939c0 msgr2=0x7f13f8095de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.038+0000 7f1404d78640 1 -- 172.21.15.59:0/3051964881 shutdown_connections 2024-05-08T05:39:07.038 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.038+0000 7f1404d78640 1 -- 172.21.15.59:0/3051964881 wait complete. 2024-05-08T05:39:07.039 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.039+0000 7f1404d78640 1 Processor -- start 2024-05-08T05:39:07.040 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.039+0000 7f1404d78640 1 -- start start 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f1404d78640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13f8098180 0x7f13f80a4870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f1404d78640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f809d250 0x7f13f80a4db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f1404d78640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13f80a2ad0 0x7f13f80a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f1404d78640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13f800a040 con 0x7f13f8098180 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f1404d78640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13f8009ec0 con 0x7f13f80a2ad0 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f1404d78640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f13f800a1c0 con 0x7f13f809d250 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f13fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13f80a2ad0 0x7f13f80a2f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f13fffff640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13f80a2ad0 0x7f13f80a2f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:34320/0 (socket says 172.21.15.59:34320) 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f13fffff640 1 -- 172.21.15.59:0/1411877593 learned_addr learned my addr 172.21.15.59:0/1411877593 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f13feffd640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f809d250 0x7f13f80a4db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f13fffff640 1 -- 172.21.15.59:0/1411877593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f809d250 msgr2=0x7f13f80a4db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.041 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f13fffff640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f809d250 0x7f13f80a4db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f13fffff640 1 -- 172.21.15.59:0/1411877593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13f8098180 msgr2=0x7f13f80a4870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f13fffff640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13f8098180 0x7f13f80a4870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.040+0000 7f13fffff640 1 -- 172.21.15.59:0/1411877593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13f813f390 con 0x7f13f80a2ad0 2024-05-08T05:39:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.041+0000 7f13feffd640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f809d250 0x7f13f80a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:39:07.042 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.041+0000 7f13fffff640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13f80a2ad0 0x7f13f80a2f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1400052040 tx=0x7f1400069d00 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.042+0000 7f13fcff9640 1 -- 172.21.15.59:0/1411877593 <== mon.2 v2:172.21.15.96:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400083020 con 0x7f13f80a2ad0 2024-05-08T05:39:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.042+0000 7f13fcff9640 1 -- 172.21.15.59:0/1411877593 <== mon.2 v2:172.21.15.96:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f140007ddb0 con 0x7f13f80a2ad0 2024-05-08T05:39:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.042+0000 7f1404d78640 1 -- 172.21.15.59:0/1411877593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13f813f560 con 0x7f13f80a2ad0 2024-05-08T05:39:07.043 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.043+0000 7f1404d78640 1 -- 172.21.15.59:0/1411877593 --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13f813f9d0 con 0x7f13f80a2ad0 2024-05-08T05:39:07.044 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.043+0000 7f13fcff9640 1 -- 172.21.15.59:0/1411877593 <== mon.2 v2:172.21.15.96:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f140007fba0 con 0x7f13f80a2ad0 2024-05-08T05:39:07.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.047+0000 7f1408dcc740 1 Processor -- start 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.048+0000 7f13fcff9640 1 -- 172.21.15.59:0/1411877593 <== mon.2 v2:172.21.15.96:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1400086020 con 0x7f13f80a2ad0 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.048+0000 7f1408dcc740 1 -- start start 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea04d0e0 0x5583ea04d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea057d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea05a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f1408dcc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5583e9fb25d0 con 0x5583ea0582c0 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f1408dcc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583e9f6b890 con 0x5583ea04da00 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f1408dcc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5583e9fae590 con 0x5583ea04d0e0 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f13feffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea04d0e0 0x5583ea04d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f13feffd640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea04d0e0 0x5583ea04d4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:56850/0 (socket says 172.21.15.59:56850) 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f13feffd640 1 -- 172.21.15.59:0/1273671629 learned_addr learned my addr 172.21.15.59:0/1273671629 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f13ff7fe640 1 --2- 172.21.15.59:0/1273671629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea057d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f13feffd640 1 --2- 172.21.15.59:0/1273671629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea05a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.049+0000 7f13feffd640 1 -- 172.21.15.59:0/1273671629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 msgr2=0x5583ea057d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.050+0000 7f13feffd640 1 --2- 172.21.15.59:0/1273671629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea057d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.050+0000 7f13feffd640 1 -- 172.21.15.59:0/1273671629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 msgr2=0x5583ea05a6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.050+0000 7f13feffd640 1 --2- 172.21.15.59:0/1273671629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea05a6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.050+0000 7f13feffd640 1 -- 172.21.15.59:0/1273671629 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583e9cc4210 con 0x5583ea04d0e0 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.050+0000 7f13feffd640 1 --2- 172.21.15.59:0/1273671629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea04d0e0 0x5583ea04d4c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009370 tx=0x7f13f4009700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=669ce2867145ffe1 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.050+0000 7f13e9ffb640 1 -- 172.21.15.59:0/1273671629 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4010690 con 0x5583ea04d0e0 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.050+0000 7f13e9ffb640 1 -- 172.21.15.59:0/1273671629 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13f4010ce0 con 0x5583ea04d0e0 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.050+0000 7f13e9ffb640 1 -- 172.21.15.59:0/1273671629 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f403fd50 con 0x5583ea04d0e0 2024-05-08T05:39:07.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f13fcff9640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13f00b2b30 0x7f13f00b4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f13fcff9640 1 -- 172.21.15.59:0/1411877593 <== mon.2 v2:172.21.15.96:3300/0 5 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f140014fa90 con 0x7f13f80a2ad0 2024-05-08T05:39:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f1408dcc740 1 -- 172.21.15.59:0/1273671629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea04d0e0 msgr2=0x5583ea04d4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f1408dcc740 1 --2- 172.21.15.59:0/1273671629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea04d0e0 0x5583ea04d4c0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009370 tx=0x7f13f4009700 comp rx=0 tx=0).stop 2024-05-08T05:39:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f13ff7fe640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13f00b2b30 0x7f13f00b4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f1408dcc740 1 -- 172.21.15.59:0/1273671629 shutdown_connections 2024-05-08T05:39:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f1408dcc740 1 --2- 172.21.15.59:0/1273671629 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea05a6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.055 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f1408dcc740 1 --2- 172.21.15.59:0/1273671629 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea057d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f1408dcc740 1 --2- 172.21.15.59:0/1273671629 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea04d0e0 0x5583ea04d4c0 secure :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13f4009370 tx=0x7f13f4009700 comp rx=0 tx=0).stop 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.051+0000 7f1408dcc740 1 -- 172.21.15.59:0/1273671629 >> 172.21.15.59:0/1273671629 conn(0x5583ea0495f0 msgr2=0x5583ea0499d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.052+0000 7f1408dcc740 1 -- 172.21.15.59:0/1273671629 shutdown_connections 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.052+0000 7f1408dcc740 1 -- 172.21.15.59:0/1273671629 wait complete. 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.052+0000 7f13ff7fe640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13f00b2b30 0x7f13f00b4ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009230 tx=0x7f13ec031400 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.052+0000 7f1408dcc740 1 -- start start 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.052+0000 7f1408dcc740 1 Processor -- start 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea055360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea0558a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea064330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f1408dcc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5583ea05ae30 con 0x5583ea0582c0 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f1408dcc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5583e9f6f370 con 0x5583ea04da00 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f1408dcc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5583ea05afb0 con 0x5583ea055de0 2024-05-08T05:39:07.056 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f1408dcc740 10 client.0 _ll_register_callbacks cb 0x5583e9cc3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13fffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea064330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13fffff640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea064330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.143:3300/0 says I am v2:172.21.15.59:56854/0 (socket says 172.21.15.59:56854) 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13ff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea055360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13fffff640 1 -- 172.21.15.59:0/2244473911 learned_addr learned my addr 172.21.15.59:0/2244473911 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13feffd640 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea0558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13fffff640 1 -- 172.21.15.59:0/2244473911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 msgr2=0x5583ea055360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13fffff640 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea055360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13fffff640 1 -- 172.21.15.59:0/2244473911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 msgr2=0x5583ea0558a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13fffff640 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea0558a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.053+0000 7f13fffff640 1 -- 172.21.15.59:0/2244473911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583e9d90620 con 0x5583ea055de0 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.054+0000 7f13fffff640 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea064330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1400116610 tx=0x7f1400069f50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a0ec2d6b5f9ca89 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.054+0000 7f13deffd640 10 client.0 ms_handle_connect on v2:172.21.15.143:3300/0 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.054+0000 7f13deffd640 1 -- 172.21.15.59:0/2244473911 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1400130060 con 0x5583ea055de0 2024-05-08T05:39:07.057 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.054+0000 7f13deffd640 1 -- 172.21.15.59:0/2244473911 <== mon.1 v2:172.21.15.143:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f140014d000 con 0x5583ea055de0 2024-05-08T05:39:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.054+0000 7f1408dcc740 1 -- 172.21.15.59:0/2244473911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f4010ce0 con 0x5583ea055de0 2024-05-08T05:39:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.054+0000 7f13deffd640 1 -- 172.21.15.59:0/2244473911 <== mon.1 v2:172.21.15.143:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f140014d200 con 0x5583ea055de0 2024-05-08T05:39:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.054+0000 7f1408dcc740 10 client.26074 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.054+0000 7f1408dcc740 1 -- 172.21.15.59:0/2244473911 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5583ea0672c0 con 0x5583ea055de0 2024-05-08T05:39:07.058 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.055+0000 7f13deffd640 1 -- 172.21.15.59:0/2244473911 <== mon.1 v2:172.21.15.143:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1400116420 con 0x5583ea055de0 2024-05-08T05:39:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.056+0000 7f13deffd640 1 -- 172.21.15.59:0/2244473911 <== mon.1 v2:172.21.15.143:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f1400115b70 con 0x5583ea055de0 2024-05-08T05:39:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.056+0000 7f1408dcc740 10 client.26074 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.057+0000 7f1408dcc740 10 client.26074 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:39:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.057+0000 7f1408dcc740 20 client.26074 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.057+0000 7f1408dcc740 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5583ea068780 0x5583ea06abc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.057+0000 7f1408dcc740 4 client.26074 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:39:07.059 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.057+0000 7f1408dcc740 1 -- 172.21.15.59:0/2244473911 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f13f4010ce0 con 0x5583ea068780 2024-05-08T05:39:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.057+0000 7f13ff7fe640 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5583ea068780 0x5583ea06abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.059+0000 7f13ff7fe640 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5583ea068780 0x5583ea06abc0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f600684d518a557 server_cookie=9b4dd8b03a655f5e in_seq=0 out_seq=0 2024-05-08T05:39:07.060 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.059+0000 7f13deffd640 10 client.26074 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.060+0000 7f13deffd640 1 -- 172.21.15.59:0/2244473911 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x7f13f4010ce0 con 0x5583ea068780 2024-05-08T05:39:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.060+0000 7f13deffd640 10 client.26074 handle_command_reply: tid=0 2024-05-08T05:39:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.060+0000 7f1408dcc740 1 client.26074 shutdown 2024-05-08T05:39:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 -- 172.21.15.59:0/2244473911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 msgr2=0x5583ea064330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea064330 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1400116610 tx=0x7f1400069f50 comp rx=0 tx=0).stop 2024-05-08T05:39:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 -- 172.21.15.59:0/2244473911 shutdown_connections 2024-05-08T05:39:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5583ea068780 0x5583ea06abc0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.061 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea064330 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea0558a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f13deffd640 0 client.26074 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 --2- 172.21.15.59:0/2244473911 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea055360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 -- 172.21.15.59:0/2244473911 >> 172.21.15.59:0/2244473911 conn(0x5583ea0495f0 msgr2=0x5583ea04f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 -- 172.21.15.59:0/2244473911 shutdown_connections 2024-05-08T05:39:07.062 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 1 -- 172.21.15.59:0/2244473911 wait complete. 2024-05-08T05:39:07.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.061+0000 7f1408dcc740 20 client.26074 trim_cache size 0 max 16384 2024-05-08T05:39:07.091 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.091+0000 7f1408dcc740 1 Processor -- start 2024-05-08T05:39:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.091+0000 7f1408dcc740 1 -- start start 2024-05-08T05:39:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea0572a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea063f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.092 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea070cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f1408dcc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4010060 con 0x5583ea0582c0 2024-05-08T05:39:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f1408dcc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4011bd0 con 0x5583ea04da00 2024-05-08T05:39:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f1408dcc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4010730 con 0x5583ea055de0 2024-05-08T05:39:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13fffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea070cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13ff7fe640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea0572a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13fffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea070cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47310/0 (socket says 172.21.15.59:47310) 2024-05-08T05:39:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13fffff640 1 -- 172.21.15.59:0/33604710 learned_addr learned my addr 172.21.15.59:0/33604710 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:07.093 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13feffd640 1 --2- 172.21.15.59:0/33604710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea063f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13feffd640 1 -- 172.21.15.59:0/33604710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 msgr2=0x5583ea0572a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13feffd640 1 --2- 172.21.15.59:0/33604710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea0572a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13feffd640 1 -- 172.21.15.59:0/33604710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 msgr2=0x5583ea070cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13feffd640 1 --2- 172.21.15.59:0/33604710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea070cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13feffd640 1 -- 172.21.15.59:0/33604710 --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5583ea05aa20 con 0x5583ea055de0 2024-05-08T05:39:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.092+0000 7f13ff7fe640 1 --2- 172.21.15.59:0/33604710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea0572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:39:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.093+0000 7f13feffd640 1 --2- 172.21.15.59:0/33604710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea063f10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f13f4041040 tx=0x7f13f40093d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=40c30d5e883a81d8 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.093+0000 7f13e97fa640 1 -- 172.21.15.59:0/33604710 <== mon.1 v2:172.21.15.143:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f4009e30 con 0x5583ea055de0 2024-05-08T05:39:07.094 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.093+0000 7f13e97fa640 1 -- 172.21.15.59:0/33604710 <== mon.1 v2:172.21.15.143:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13f403f9a0 con 0x5583ea055de0 2024-05-08T05:39:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.094+0000 7f1408dcc740 1 -- 172.21.15.59:0/33604710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 msgr2=0x5583ea063f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.094+0000 7f1408dcc740 1 --2- 172.21.15.59:0/33604710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea063f10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f13f4041040 tx=0x7f13f40093d0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.095+0000 7f1408dcc740 1 -- 172.21.15.59:0/33604710 shutdown_connections 2024-05-08T05:39:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.095+0000 7f1408dcc740 1 --2- 172.21.15.59:0/33604710 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea0582c0 0x5583ea070cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.095+0000 7f1408dcc740 1 --2- 172.21.15.59:0/33604710 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea055de0 0x5583ea063f10 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.095 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.095+0000 7f1408dcc740 1 --2- 172.21.15.59:0/33604710 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea04da00 0x5583ea0572a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.096 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.095+0000 7f1408dcc740 1 -- 172.21.15.59:0/33604710 >> 172.21.15.59:0/33604710 conn(0x5583ea0495f0 msgr2=0x5583ea056b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.097+0000 7f1408dcc740 1 -- 172.21.15.59:0/33604710 shutdown_connections 2024-05-08T05:39:07.097 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.097+0000 7f1408dcc740 1 -- 172.21.15.59:0/33604710 wait complete. 2024-05-08T05:39:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.097+0000 7f1408dcc740 1 -- start start 2024-05-08T05:39:07.098 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.098+0000 7f1408dcc740 1 Processor -- start 2024-05-08T05:39:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.098+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea04da00 0x5583ea060870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.098+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea055de0 0x5583ea05d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.098+0000 7f1408dcc740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea0582c0 0x5583ea05d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.098+0000 7f1408dcc740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4011500 con 0x5583ea04da00 2024-05-08T05:39:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.098+0000 7f1408dcc740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f13e4010e30 con 0x5583ea055de0 2024-05-08T05:39:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.098+0000 7f1408dcc740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f13e400e040 con 0x5583ea0582c0 2024-05-08T05:39:07.099 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.098+0000 7f1408dcc740 10 client.0 _ll_register_callbacks cb 0x7f140007df60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13ff7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea04da00 0x5583ea060870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13ff7fe640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea04da00 0x5583ea060870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:47326/0 (socket says 172.21.15.59:47326) 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13ff7fe640 1 -- 172.21.15.59:0/3684721083 learned_addr learned my addr 172.21.15.59:0/3684721083 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13feffd640 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea055de0 0x5583ea05d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13fffff640 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea0582c0 0x5583ea05d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13ff7fe640 1 -- 172.21.15.59:0/3684721083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea0582c0 msgr2=0x5583ea05d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13ff7fe640 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea0582c0 0x5583ea05d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13ff7fe640 1 -- 172.21.15.59:0/3684721083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea055de0 msgr2=0x5583ea05d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13ff7fe640 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea055de0 0x5583ea05d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13ff7fe640 1 -- 172.21.15.59:0/3684721083 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5583ea05e020 con 0x5583ea04da00 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13fffff640 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea0582c0 0x5583ea05d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.099+0000 7f13feffd640 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea055de0 0x5583ea05d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.100+0000 7f13ff7fe640 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea04da00 0x5583ea060870 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009900 tx=0x7f13ec005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b8878d3cb47c72d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.100+0000 7f13deffd640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.100+0000 7f13deffd640 1 -- 172.21.15.59:0/3684721083 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13ec0318a0 con 0x5583ea04da00 2024-05-08T05:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.100+0000 7f1408dcc740 1 -- 172.21.15.59:0/3684721083 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f13f403f9a0 con 0x5583ea04da00 2024-05-08T05:39:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.100+0000 7f13deffd640 1 -- 172.21.15.59:0/3684721083 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(99..99 src has 1..99) ==== 9584+0+0 (secure 0 0 0) 0x7f13ec0053d0 con 0x5583ea04da00 2024-05-08T05:39:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.101+0000 7f13deffd640 1 -- 172.21.15.59:0/3684721083 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f13ec0055d0 con 0x5583ea04da00 2024-05-08T05:39:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.101+0000 7f1408dcc740 10 client.16617 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:07.102 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.101+0000 7f1408dcc740 1 -- 172.21.15.59:0/3684721083 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5583ea0d4910 con 0x5583ea04da00 2024-05-08T05:39:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.101+0000 7f13deffd640 1 -- 172.21.15.59:0/3684721083 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13f403f9a0 con 0x5583ea04da00 2024-05-08T05:39:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.102+0000 7f13deffd640 1 -- 172.21.15.59:0/3684721083 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f13ec03a040 con 0x5583ea04da00 2024-05-08T05:39:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.102+0000 7f1408dcc740 10 client.16617 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.102+0000 7f1408dcc740 10 client.16617 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:39:07.103 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.102+0000 7f1408dcc740 20 client.16617 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.102+0000 7f1408dcc740 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5583ea0d5080 0x5583ea0d74f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.102+0000 7f1408dcc740 4 client.16617 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:39:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.102+0000 7f1408dcc740 1 -- 172.21.15.59:0/3684721083 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5583e9fc01e0 con 0x5583ea0d5080 2024-05-08T05:39:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.104+0000 7f13feffd640 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5583ea0d5080 0x5583ea0d74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.105+0000 7f13feffd640 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5583ea0d5080 0x5583ea0d74f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e33de51f5c26d330 server_cookie=509ade1d0c6e64a9 in_seq=0 out_seq=0 2024-05-08T05:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.105+0000 7f13deffd640 10 client.16617 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:07 smithi143 ceph-mon[34337]: pgmap v297: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-05-08T05:39:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:07 smithi096 ceph-mon[34253]: pgmap v297: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-05-08T05:39:07.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:07 smithi059 ceph-mon[30304]: pgmap v297: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f90eb7fe700 1 -- 172.21.15.59:0/2239152168 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55d356dee3d0 con 0x55d356ea84e0 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f90eb7fe700 10 client.26135 handle_command_reply: tid=0 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 client.26135 shutdown 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2239152168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93950 msgr2=0x55d356e9bf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93950 0x55d356e9bf00 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x55d356e96320 tx=0x7f9104017c00 comp rx=0 tx=0).stop 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2239152168 shutdown_connections 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55d356ea84e0 0x55d356f16330 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55d356e93950 0x55d356e9bf00 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55d356e8ee50 0x55d356ea0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 --2- 172.21.15.59:0/2239152168 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55d356e93030 0x55d356e9b9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f90eb7fe700 0 client.26135 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2239152168 >> 172.21.15.59:0/2239152168 conn(0x55d356dba290 msgr2=0x55d356e886c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2239152168 shutdown_connections 2024-05-08T05:39:07.588 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 1 -- 172.21.15.59:0/2239152168 wait complete. 2024-05-08T05:39:07.589 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.587+0000 7f911cc5fb80 20 client.26135 trim_cache size 0 max 16384 2024-05-08T05:39:07.589 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:39:07.589 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 39, 2024-05-08T05:39:07.589 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:39:07.589 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 30, 2024-05-08T05:39:07.589 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:39:07.589 INFO:teuthology.orchestra.run.smithi059.stdout: "df019e95": { 2024-05-08T05:39:07.589 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 30, 2024-05-08T05:39:07.589 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 90.0, 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "QUIESCED", 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:07.590 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 -- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f90f80b2b20 msgr2=0x7f90f80b4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f90f80b2b20 0x7f90f80b4fe0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f910c079870 tx=0x7f90f4005960 comp rx=0 tx=0).stop 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 -- 172.21.15.59:0/919160997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f910c06bb70 msgr2=0x7f910c19f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f910c06bb70 0x7f910c19f750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00e9c0 tx=0x7f90fc00ecd0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 -- 172.21.15.59:0/919160997 shutdown_connections 2024-05-08T05:39:07.591 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f90f80b2b20 0x7f90f80b4fe0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f910c1084b0 0x7f910c199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f910c06bb70 0x7f910c19f750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 --2- 172.21.15.59:0/919160997 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f910c06c510 0x7f910c19fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.588+0000 7f91123cc700 1 -- 172.21.15.59:0/919160997 >> 172.21.15.59:0/919160997 conn(0x7f910c079350 msgr2=0x7f910c100790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.589+0000 7f91123cc700 1 -- 172.21.15.59:0/919160997 shutdown_connections 2024-05-08T05:39:07.592 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:07.589+0000 7f91123cc700 1 -- 172.21.15.59:0/919160997 wait complete. 2024-05-08T05:39:07.757 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.756+0000 7f13deffd640 1 -- 172.21.15.59:0/3684721083 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5583e9fc01e0 con 0x5583ea0d5080 2024-05-08T05:39:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.756+0000 7f13deffd640 10 client.16617 handle_command_reply: tid=0 2024-05-08T05:39:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.756+0000 7f1408dcc740 1 client.16617 shutdown 2024-05-08T05:39:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 -- 172.21.15.59:0/3684721083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea04da00 msgr2=0x5583ea060870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.758 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea04da00 0x5583ea060870 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009900 tx=0x7f13ec005fd0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 -- 172.21.15.59:0/3684721083 shutdown_connections 2024-05-08T05:39:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5583ea0d5080 0x5583ea0d74f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5583ea0582c0 0x5583ea05d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5583ea055de0 0x5583ea05d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 --2- 172.21.15.59:0/3684721083 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5583ea04da00 0x5583ea060870 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 -- 172.21.15.59:0/3684721083 >> 172.21.15.59:0/3684721083 conn(0x5583ea0495f0 msgr2=0x5583ea048750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f13deffd640 0 client.16617 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 -- 172.21.15.59:0/3684721083 shutdown_connections 2024-05-08T05:39:07.759 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 1 -- 172.21.15.59:0/3684721083 wait complete. 2024-05-08T05:39:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.757+0000 7f1408dcc740 20 client.16617 trim_cache size 0 max 16384 2024-05-08T05:39:07.760 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:39:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:39:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:39:07.761 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:39:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 -- 172.21.15.59:0/1411877593 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13f00b2b30 msgr2=0x7f13f00b4ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13f00b2b30 0x7f13f00b4ff0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f13ec009230 tx=0x7f13ec031400 comp rx=0 tx=0).stop 2024-05-08T05:39:07.762 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 -- 172.21.15.59:0/1411877593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13f80a2ad0 msgr2=0x7f13f80a2f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13f80a2ad0 0x7f13f80a2f50 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f1400052040 tx=0x7f1400069d00 comp rx=0 tx=0).stop 2024-05-08T05:39:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 -- 172.21.15.59:0/1411877593 shutdown_connections 2024-05-08T05:39:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f13f00b2b30 0x7f13f00b4ff0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f13f80a2ad0 0x7f13f80a2f50 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f13f809d250 0x7f13f80a4db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.763 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 --2- 172.21.15.59:0/1411877593 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f13f8098180 0x7f13f80a4870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 -- 172.21.15.59:0/1411877593 >> 172.21.15.59:0/1411877593 conn(0x7f13f80939c0 msgr2=0x7f13f8094d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 -- 172.21.15.59:0/1411877593 shutdown_connections 2024-05-08T05:39:07.764 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:07.758+0000 7f1404d78640 1 -- 172.21.15.59:0/1411877593 wait complete. 2024-05-08T05:39:08.267 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced set 'df019e95', quiesce took 8.2 seconds. Will release after: 30.0 2024-05-08T05:39:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:09 smithi143 ceph-mon[34337]: pgmap v298: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-05-08T05:39:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:09 smithi096 ceph-mon[34253]: pgmap v298: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-05-08T05:39:09.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:09 smithi059 ceph-mon[30304]: pgmap v298: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.3 MiB/s wr, 261 op/s 2024-05-08T05:39:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:11 smithi143 ceph-mon[34337]: pgmap v299: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 526 op/s 2024-05-08T05:39:11.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:11 smithi143 ceph-mon[34337]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T05:39:11.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:11 smithi096 ceph-mon[34253]: pgmap v299: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 526 op/s 2024-05-08T05:39:11.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:11 smithi096 ceph-mon[34253]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T05:39:11.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:11 smithi059 ceph-mon[30304]: pgmap v299: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 526 op/s 2024-05-08T05:39:11.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:11 smithi059 ceph-mon[30304]: osdmap e100: 12 total, 12 up, 12 in 2024-05-08T05:39:13.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:13 smithi096 ceph-mon[34253]: pgmap v301: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 598 op/s 2024-05-08T05:39:13.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:13 smithi059 ceph-mon[30304]: pgmap v301: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 598 op/s 2024-05-08T05:39:13.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:13 smithi143 ceph-mon[34337]: pgmap v301: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 2.1 MiB/s rd, 1.8 MiB/s wr, 598 op/s 2024-05-08T05:39:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:15 smithi096 ceph-mon[34253]: pgmap v302: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 526 op/s 2024-05-08T05:39:15.544 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:15 smithi143 ceph-mon[34337]: pgmap v302: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 526 op/s 2024-05-08T05:39:15.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:15 smithi059 ceph-mon[30304]: pgmap v302: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 1.9 MiB/s rd, 1.6 MiB/s wr, 526 op/s 2024-05-08T05:39:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:17 smithi096 ceph-mon[34253]: pgmap v303: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 396 KiB/s wr, 319 op/s 2024-05-08T05:39:17.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:17 smithi059 ceph-mon[30304]: pgmap v303: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 396 KiB/s wr, 319 op/s 2024-05-08T05:39:17.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:17 smithi143 ceph-mon[34337]: pgmap v303: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 396 KiB/s wr, 319 op/s 2024-05-08T05:39:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:19 smithi096 ceph-mon[34253]: pgmap v304: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 396 KiB/s wr, 319 op/s 2024-05-08T05:39:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:19 smithi059 ceph-mon[30304]: pgmap v304: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 396 KiB/s wr, 319 op/s 2024-05-08T05:39:19.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:19 smithi143 ceph-mon[34337]: pgmap v304: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 379 KiB/s rd, 396 KiB/s wr, 319 op/s 2024-05-08T05:39:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.860+0000 7fb93ca20640 1 -- 172.21.15.59:0/1849172681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9381089d0 msgr2=0x7fb938108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.860+0000 7fb93ca20640 1 --2- 172.21.15.59:0/1849172681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9381089d0 0x7fb938108db0 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7fb928009930 tx=0x7fb92802a420 comp rx=0 tx=0).stop 2024-05-08T05:39:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.860+0000 7fb93ca20640 1 -- 172.21.15.59:0/1849172681 shutdown_connections 2024-05-08T05:39:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.860+0000 7fb93ca20640 1 --2- 172.21.15.59:0/1849172681 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb938103380 0x7fb93810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.861 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.860+0000 7fb93ca20640 1 --2- 172.21.15.59:0/1849172681 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9381029c0 0x7fb938102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.860+0000 7fb93ca20640 1 --2- 172.21.15.59:0/1849172681 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb9381089d0 0x7fb938108db0 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.860+0000 7fb93ca20640 1 -- 172.21.15.59:0/1849172681 >> 172.21.15.59:0/1849172681 conn(0x7fb938079e40 msgr2=0x7fb93807a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.860+0000 7fb93ca20640 1 -- 172.21.15.59:0/1849172681 shutdown_connections 2024-05-08T05:39:20.862 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.861+0000 7fb93ca20640 1 -- 172.21.15.59:0/1849172681 wait complete. 2024-05-08T05:39:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.861+0000 7fb93ca20640 1 Processor -- start 2024-05-08T05:39:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.861+0000 7fb93ca20640 1 -- start start 2024-05-08T05:39:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb93ca20640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9381029c0 0x7fb938199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb93ca20640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb938103380 0x7fb938199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb93ca20640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9381089d0 0x7fb93819da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb93ca20640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb938112c60 con 0x7fb938103380 2024-05-08T05:39:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb93ca20640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb938112ae0 con 0x7fb9381029c0 2024-05-08T05:39:20.864 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb93ca20640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb938112de0 con 0x7fb9381089d0 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb935d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb938103380 0x7fb938199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb935d9b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb938103380 0x7fb938199680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39814/0 (socket says 172.21.15.59:39814) 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb935d9b640 1 -- 172.21.15.59:0/2479146257 learned_addr learned my addr 172.21.15.59:0/2479146257 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb93659c640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9381029c0 0x7fb938199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb936d9d640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9381089d0 0x7fb93819da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb935d9b640 1 -- 172.21.15.59:0/2479146257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9381089d0 msgr2=0x7fb93819da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb935d9b640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9381089d0 0x7fb93819da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb935d9b640 1 -- 172.21.15.59:0/2479146257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9381029c0 msgr2=0x7fb938199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb935d9b640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9381029c0 0x7fb938199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb935d9b640 1 -- 172.21.15.59:0/2479146257 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb93819e130 con 0x7fb938103380 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb936d9d640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9381089d0 0x7fb93819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb935d9b640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb938103380 0x7fb938199680 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00d950 tx=0x7fb92c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb9277fe640 1 -- 172.21.15.59:0/2479146257 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c017030 con 0x7fb938103380 2024-05-08T05:39:20.865 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb93ca20640 1 -- 172.21.15.59:0/2479146257 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb93819e420 con 0x7fb938103380 2024-05-08T05:39:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb9277fe640 1 -- 172.21.15.59:0/2479146257 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb92c004020 con 0x7fb938103380 2024-05-08T05:39:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb9277fe640 1 -- 172.21.15.59:0/2479146257 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92c0047a0 con 0x7fb938103380 2024-05-08T05:39:20.866 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.862+0000 7fb93ca20640 1 -- 172.21.15.59:0/2479146257 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9381a5cb0 con 0x7fb938103380 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.865+0000 7fb9277fe640 1 -- 172.21.15.59:0/2479146257 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb92c009a30 con 0x7fb938103380 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.865+0000 7fb93f815740 1 Processor -- start 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93f815740 1 -- start start 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93f815740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293cf10 0x55f41293d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93f815740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293d830 0x55f412947bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93f815740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41294a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93f815740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f4128a25d0 con 0x55f41293d830 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93f815740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f41285b890 con 0x55f41293cf10 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93f815740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f41289e590 con 0x55f4129480f0 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293d830 0x55f412947bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293d830 0x55f412947bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39820/0 (socket says 172.21.15.59:39820) 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93659c640 1 -- 172.21.15.59:0/310203048 learned_addr learned my addr 172.21.15.59:0/310203048 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:20.872 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb935d9b640 1 --2- 172.21.15.59:0/310203048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41294a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb936d9d640 1 --2- 172.21.15.59:0/310203048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293cf10 0x55f41293d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93659c640 1 -- 172.21.15.59:0/310203048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 msgr2=0x55f41294a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93659c640 1 --2- 172.21.15.59:0/310203048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41294a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93659c640 1 -- 172.21.15.59:0/310203048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293cf10 msgr2=0x55f41293d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93659c640 1 --2- 172.21.15.59:0/310203048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293cf10 0x55f41293d2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb93659c640 1 -- 172.21.15.59:0/310203048 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f4125b4210 con 0x55f41293d830 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.866+0000 7fb9277fe640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb9200b2a10 0x7fb9200b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.867+0000 7fb936d9d640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb9200b2a10 0x7fb9200b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.867+0000 7fb93659c640 1 --2- 172.21.15.59:0/310203048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293d830 0x55f412947bb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb928031440 tx=0x7fb928032c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc558ccf77ce209 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.867+0000 7fb9277fe640 1 -- 172.21.15.59:0/2479146257 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fb92c0e24d0 con 0x7fb938103380 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.867+0000 7fb924ff9640 1 -- 172.21.15.59:0/310203048 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92803f070 con 0x55f41293d830 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.867+0000 7fb924ff9640 1 -- 172.21.15.59:0/310203048 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb928032e40 con 0x55f41293d830 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.867+0000 7fb924ff9640 1 -- 172.21.15.59:0/310203048 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928035070 con 0x55f41293d830 2024-05-08T05:39:20.873 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 -- 172.21.15.59:0/310203048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293d830 msgr2=0x55f412947bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb936d9d640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb9200b2a10 0x7fb9200b4ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb930005fd0 tx=0x7fb930009700 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 --2- 172.21.15.59:0/310203048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293d830 0x55f412947bb0 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fb928031440 tx=0x7fb928032c50 comp rx=0 tx=0).stop 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 -- 172.21.15.59:0/310203048 shutdown_connections 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 --2- 172.21.15.59:0/310203048 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41294a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 --2- 172.21.15.59:0/310203048 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293d830 0x55f412947bb0 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 --2- 172.21.15.59:0/310203048 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293cf10 0x55f41293d2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 -- 172.21.15.59:0/310203048 >> 172.21.15.59:0/310203048 conn(0x55f412939420 msgr2=0x55f412939800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 -- 172.21.15.59:0/310203048 shutdown_connections 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 -- 172.21.15.59:0/310203048 wait complete. 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 -- start start 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.868+0000 7fb93f815740 1 Processor -- start 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb93f815740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f41293cf10 0x55f412944f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb93f815740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f4129454b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.874 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb93f815740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4129480f0 0x55f4129459f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb93f815740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55f41294acc0 con 0x55f4129480f0 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb93f815740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55f41285f370 con 0x55f41293d830 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb93f815740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55f41294ae40 con 0x55f41293cf10 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb93f815740 10 client.0 _ll_register_callbacks cb 0x55f4125b3a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4129480f0 0x55f4129459f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb936d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4129480f0 0x55f4129459f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39834/0 (socket says 172.21.15.59:39834) 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb936d9d640 1 -- 172.21.15.59:0/4119565716 learned_addr learned my addr 172.21.15.59:0/4119565716 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb93659c640 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f41293cf10 0x55f412944f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb935d9b640 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f4129454b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb936d9d640 1 -- 172.21.15.59:0/4119565716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f41293cf10 msgr2=0x55f412944f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb936d9d640 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f41293cf10 0x55f412944f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb936d9d640 1 -- 172.21.15.59:0/4119565716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 msgr2=0x55f4129454b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb936d9d640 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f4129454b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.875 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb936d9d640 1 -- 172.21.15.59:0/4119565716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f412680620 con 0x55f4129480f0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb935d9b640 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f4129454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb936d9d640 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4129480f0 0x55f4129459f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb9300096d0 tx=0x7fb930004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=610424caaa07edcb server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.869+0000 7fb91dffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb91dffb640 1 -- 172.21.15.59:0/4119565716 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93002a060 con 0x55f4129480f0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb91dffb640 1 -- 172.21.15.59:0/4119565716 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fb93001db90 con 0x55f4129480f0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb93f815740 1 -- 172.21.15.59:0/4119565716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f4128b01e0 con 0x55f4129480f0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb91dffb640 1 -- 172.21.15.59:0/4119565716 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb930031050 con 0x55f4129480f0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb93f815740 10 client.16629 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb93f815740 1 -- 172.21.15.59:0/4119565716 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f412956ae0 con 0x55f4129480f0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb91dffb640 1 -- 172.21.15.59:0/4119565716 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f4128b01e0 con 0x55f4129480f0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb91dffb640 1 -- 172.21.15.59:0/4119565716 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fb930019040 con 0x55f4129480f0 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb93f815740 10 client.16629 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb93f815740 10 client.16629 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb93f815740 20 client.16629 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb93f815740 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f412957fa0 0x55f41295a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.876 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb93f815740 4 client.16629 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:39:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.870+0000 7fb93f815740 1 -- 172.21.15.59:0/4119565716 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f4128aa050 con 0x55f412957fa0 2024-05-08T05:39:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.871+0000 7fb93659c640 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f412957fa0 0x55f41295a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.871+0000 7fb93659c640 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f412957fa0 0x55f41295a390 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=19409d885d4abe83 server_cookie=8593ed1ccf1a744c in_seq=0 out_seq=0 2024-05-08T05:39:20.877 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.871+0000 7fb91dffb640 10 client.16629 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb91dffb640 1 -- 172.21.15.59:0/4119565716 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55f4128aa050 con 0x55f412957fa0 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb91dffb640 10 client.16629 handle_command_reply: tid=0 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 client.16629 shutdown 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 -- 172.21.15.59:0/4119565716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4129480f0 msgr2=0x55f4129459f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4129480f0 0x55f4129459f0 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7fb9300096d0 tx=0x7fb930004a40 comp rx=0 tx=0).stop 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 -- 172.21.15.59:0/4119565716 shutdown_connections 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f412957fa0 0x55f41295a390 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f4129480f0 0x55f4129459f0 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f4129454b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.878 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 --2- 172.21.15.59:0/4119565716 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f41293cf10 0x55f412944f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb91dffb640 0 client.16629 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 -- 172.21.15.59:0/4119565716 >> 172.21.15.59:0/4119565716 conn(0x55f412939420 msgr2=0x55f41293f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 -- 172.21.15.59:0/4119565716 shutdown_connections 2024-05-08T05:39:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 1 -- 172.21.15.59:0/4119565716 wait complete. 2024-05-08T05:39:20.879 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.872+0000 7fb93f815740 20 client.16629 trim_cache size 0 max 16384 2024-05-08T05:39:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93f815740 1 Processor -- start 2024-05-08T05:39:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93f815740 1 -- start start 2024-05-08T05:39:20.892 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93f815740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f41293f920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93f815740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f412944850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93f815740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41295fca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93f815740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381afe40 con 0x55f41293cf10 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93f815740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b19b0 con 0x55f41293d830 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93f815740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b0510 con 0x55f4129480f0 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f41293f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f41293f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39840/0 (socket says 172.21.15.59:39840) 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93659c640 1 -- 172.21.15.59:0/1830848981 learned_addr learned my addr 172.21.15.59:0/1830848981 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb935d9b640 1 --2- 172.21.15.59:0/1830848981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f412944850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93659c640 1 -- 172.21.15.59:0/1830848981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 msgr2=0x55f41295fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93659c640 1 --2- 172.21.15.59:0/1830848981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41295fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93659c640 1 -- 172.21.15.59:0/1830848981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 msgr2=0x55f412944850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93659c640 1 --2- 172.21.15.59:0/1830848981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f412944850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.893 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93659c640 1 -- 172.21.15.59:0/1830848981 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f41295a700 con 0x55f41293cf10 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb935d9b640 1 --2- 172.21.15.59:0/1830848981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f412944850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.888+0000 7fb93659c640 1 --2- 172.21.15.59:0/1830848981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f41293f920 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb928033000 tx=0x7fb92802a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b35f39a9940b2cad server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb91ffff640 1 -- 172.21.15.59:0/1830848981 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb928055070 con 0x55f41293cf10 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb91ffff640 1 -- 172.21.15.59:0/1830848981 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9280317a0 con 0x55f41293cf10 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb91ffff640 1 -- 172.21.15.59:0/1830848981 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92803f040 con 0x55f41293cf10 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb93f815740 1 -- 172.21.15.59:0/1830848981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 msgr2=0x55f41293f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb93f815740 1 --2- 172.21.15.59:0/1830848981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f41293f920 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb928033000 tx=0x7fb92802a420 comp rx=0 tx=0).stop 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb93f815740 1 -- 172.21.15.59:0/1830848981 shutdown_connections 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb93f815740 1 --2- 172.21.15.59:0/1830848981 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41295fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb93f815740 1 --2- 172.21.15.59:0/1830848981 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f412944850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb93f815740 1 --2- 172.21.15.59:0/1830848981 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f41293f920 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb93f815740 1 -- 172.21.15.59:0/1830848981 >> 172.21.15.59:0/1830848981 conn(0x55f412939420 msgr2=0x55f412947030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb93f815740 1 -- 172.21.15.59:0/1830848981 shutdown_connections 2024-05-08T05:39:20.894 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.889+0000 7fb93f815740 1 -- 172.21.15.59:0/1830848981 wait complete. 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93f815740 1 -- start start 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93f815740 1 Processor -- start 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93f815740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f412950250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93f815740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f412950790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93f815740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41294b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93f815740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b12e0 con 0x55f41293cf10 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93f815740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381b0c10 con 0x55f41293d830 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93f815740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb9381ade40 con 0x55f4129480f0 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93f815740 10 client.0 _ll_register_callbacks cb 0x7fb930009bc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f412950250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93659c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f412950250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:39852/0 (socket says 172.21.15.59:39852) 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93659c640 1 -- 172.21.15.59:0/1433966372 learned_addr learned my addr 172.21.15.59:0/1433966372 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb935d9b640 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f412950790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.895 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93659c640 1 -- 172.21.15.59:0/1433966372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 msgr2=0x55f41294b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93659c640 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41294b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93659c640 1 -- 172.21.15.59:0/1433966372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 msgr2=0x55f412950790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93659c640 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f412950790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.890+0000 7fb93659c640 1 -- 172.21.15.59:0/1433966372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f41294bff0 con 0x55f41293cf10 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93659c640 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f412950250 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb928053040 tx=0x7fb928005950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4da3e2ff4ec572b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb91dffb640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb91dffb640 1 -- 172.21.15.59:0/1433966372 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb92805d060 con 0x55f41293cf10 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb91dffb640 1 -- 172.21.15.59:0/1433966372 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fb92804f110 con 0x55f41293cf10 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93f815740 1 -- 172.21.15.59:0/1433966372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9280317a0 con 0x55f41293cf10 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb91dffb640 1 -- 172.21.15.59:0/1433966372 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fb928064050 con 0x55f41293cf10 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93f815740 10 client.16635 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93f815740 1 -- 172.21.15.59:0/1433966372 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55f4129c2430 con 0x55f41293cf10 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb91dffb640 1 -- 172.21.15.59:0/1433966372 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9280317a0 con 0x55f41293cf10 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb91dffb640 1 -- 172.21.15.59:0/1433966372 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fb92804c070 con 0x55f41293cf10 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93f815740 10 client.16635 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:20.896 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93f815740 10 client.16635 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:39:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93f815740 20 client.16635 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93f815740 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4129c2650 0x55f4129c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93f815740 4 client.16635 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:39:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.891+0000 7fb93f815740 1 -- 172.21.15.59:0/1433966372 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f4128255f0 con 0x55f4129c2650 2024-05-08T05:39:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.892+0000 7fb935d9b640 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4129c2650 0x55f4129c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.892+0000 7fb935d9b640 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4129c2650 0x55f4129c4a40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44981b53a9f4ed98 server_cookie=44841fd4ea7c3dcd in_seq=0 out_seq=0 2024-05-08T05:39:20.897 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.892+0000 7fb91dffb640 10 client.16635 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb91dffb640 1 -- 172.21.15.59:0/1433966372 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f4128255f0 con 0x55f4129c2650 2024-05-08T05:39:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb91dffb640 10 client.16635 handle_command_reply: tid=0 2024-05-08T05:39:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb93f815740 1 client.16635 shutdown 2024-05-08T05:39:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb93f815740 1 -- 172.21.15.59:0/1433966372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 msgr2=0x55f412950250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:20.914 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb93f815740 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f412950250 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb928053040 tx=0x7fb928005950 comp rx=0 tx=0).stop 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb93f815740 1 -- 172.21.15.59:0/1433966372 shutdown_connections 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb93f815740 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55f4129c2650 0x55f4129c4a40 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb93f815740 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55f4129480f0 0x55f41294b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb91dffb640 0 client.16635 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb93f815740 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55f41293d830 0x55f412950790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb93f815740 1 --2- 172.21.15.59:0/1433966372 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55f41293cf10 0x55f412950250 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.913+0000 7fb93f815740 1 -- 172.21.15.59:0/1433966372 >> 172.21.15.59:0/1433966372 conn(0x55f412939420 msgr2=0x55f4129467a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93f815740 1 -- 172.21.15.59:0/1433966372 shutdown_connections 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93f815740 1 -- 172.21.15.59:0/1433966372 wait complete. 2024-05-08T05:39:20.915 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93f815740 20 client.16635 trim_cache size 0 max 16384 2024-05-08T05:39:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:39:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:39:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:39:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:39:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 -- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb9200b2a10 msgr2=0x7fb9200b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:20.916 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb9200b2a10 0x7fb9200b4ed0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb930005fd0 tx=0x7fb930009700 comp rx=0 tx=0).stop 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 -- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb938103380 msgr2=0x7fb938199680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb938103380 0x7fb938199680 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fb92c00d950 tx=0x7fb92c00de10 comp rx=0 tx=0).stop 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 -- 172.21.15.59:0/2479146257 shutdown_connections 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb9200b2a10 0x7fb9200b4ed0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb9381089d0 0x7fb93819da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb938103380 0x7fb938199680 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 --2- 172.21.15.59:0/2479146257 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb9381029c0 0x7fb938199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 -- 172.21.15.59:0/2479146257 >> 172.21.15.59:0/2479146257 conn(0x7fb938079e40 msgr2=0x7fb9381003e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 -- 172.21.15.59:0/2479146257 shutdown_connections 2024-05-08T05:39:20.917 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:20.914+0000 7fb93ca20640 1 -- 172.21.15.59:0/2479146257 wait complete. 2024-05-08T05:39:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:21 smithi096 ceph-mon[34253]: pgmap v305: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 29 op/s 2024-05-08T05:39:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:21 smithi059 ceph-mon[30304]: pgmap v305: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 29 op/s 2024-05-08T05:39:21.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:21 smithi143 ceph-mon[34337]: pgmap v305: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 29 op/s 2024-05-08T05:39:23.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:23 smithi096 ceph-mon[34253]: pgmap v306: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 26 op/s 2024-05-08T05:39:23.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:23 smithi059 ceph-mon[30304]: pgmap v306: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 26 op/s 2024-05-08T05:39:23.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:23 smithi143 ceph-mon[34337]: pgmap v306: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 18 KiB/s rd, 14 KiB/s wr, 26 op/s 2024-05-08T05:39:25.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:25 smithi096 ceph-mon[34253]: pgmap v307: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 24 op/s 2024-05-08T05:39:25.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:25 smithi059 ceph-mon[30304]: pgmap v307: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 24 op/s 2024-05-08T05:39:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:25 smithi143 ceph-mon[34337]: pgmap v307: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 24 op/s 2024-05-08T05:39:27.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:27 smithi096 ceph-mon[34253]: pgmap v308: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 34 op/s 2024-05-08T05:39:27.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:27 smithi059 ceph-mon[30304]: pgmap v308: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 34 op/s 2024-05-08T05:39:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:27 smithi143 ceph-mon[34337]: pgmap v308: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 22 KiB/s wr, 34 op/s 2024-05-08T05:39:29.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:29 smithi096 ceph-mon[34253]: pgmap v309: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-05-08T05:39:29.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:29 smithi059 ceph-mon[30304]: pgmap v309: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-05-08T05:39:29.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:29 smithi143 ceph-mon[34337]: pgmap v309: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-05-08T05:39:31.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:31 smithi096 ceph-mon[34253]: pgmap v310: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 17 op/s 2024-05-08T05:39:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:31 smithi059 ceph-mon[30304]: pgmap v310: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 17 op/s 2024-05-08T05:39:31.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:31 smithi143 ceph-mon[34337]: pgmap v310: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 9.5 KiB/s wr, 17 op/s 2024-05-08T05:39:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:33 smithi096 ceph-mon[34253]: pgmap v311: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-05-08T05:39:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:33 smithi059 ceph-mon[30304]: pgmap v311: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-05-08T05:39:33.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:33 smithi143 ceph-mon[34337]: pgmap v311: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-05-08T05:39:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.015+0000 7fe69f220640 1 -- 172.21.15.59:0/875641212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981018e0 msgr2=0x7fe698101d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.015+0000 7fe69f220640 1 --2- 172.21.15.59:0/875641212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981018e0 0x7fe698101d60 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7fe688009930 tx=0x7fe68802a420 comp rx=0 tx=0).stop 2024-05-08T05:39:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.015+0000 7fe69f220640 1 -- 172.21.15.59:0/875641212 shutdown_connections 2024-05-08T05:39:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.015+0000 7fe69f220640 1 --2- 172.21.15.59:0/875641212 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981022a0 0x7fe69810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.015+0000 7fe69f220640 1 --2- 172.21.15.59:0/875641212 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981018e0 0x7fe698101d60 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.019 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.015+0000 7fe69f220640 1 --2- 172.21.15.59:0/875641212 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6981078a0 0x7fe698107c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.015+0000 7fe69f220640 1 -- 172.21.15.59:0/875641212 >> 172.21.15.59:0/875641212 conn(0x7fe698078a30 msgr2=0x7fe698078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.015+0000 7fe69f220640 1 -- 172.21.15.59:0/875641212 shutdown_connections 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.015+0000 7fe69f220640 1 -- 172.21.15.59:0/875641212 wait complete. 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe69f220640 1 Processor -- start 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe69f220640 1 -- start start 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe69f220640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981018e0 0x7fe6981991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe69f220640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981022a0 0x7fe6981996f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe69f220640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6981078a0 0x7fe69819da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe69f220640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe698111b20 con 0x7fe6981022a0 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe69f220640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981119a0 con 0x7fe6981018e0 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe69f220640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe698111ca0 con 0x7fe6981078a0 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe697fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981022a0 0x7fe6981996f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe697fff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981022a0 0x7fe6981996f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45796/0 (socket says 172.21.15.59:45796) 2024-05-08T05:39:34.020 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe697fff640 1 -- 172.21.15.59:0/3903368098 learned_addr learned my addr 172.21.15.59:0/3903368098 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.016+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981018e0 0x7fe6981991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6981078a0 0x7fe69819da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe697fff640 1 -- 172.21.15.59:0/3903368098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6981078a0 msgr2=0x7fe69819da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe697fff640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6981078a0 0x7fe69819da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe697fff640 1 -- 172.21.15.59:0/3903368098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981018e0 msgr2=0x7fe6981991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe697fff640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981018e0 0x7fe6981991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe697fff640 1 -- 172.21.15.59:0/3903368098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe69819e1a0 con 0x7fe6981022a0 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe697fff640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981022a0 0x7fe6981996f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe688005940 tx=0x7fe6880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe695ffb640 1 -- 172.21.15.59:0/3903368098 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68803f030 con 0x7fe6981022a0 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe695ffb640 1 -- 172.21.15.59:0/3903368098 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe688003cb0 con 0x7fe6981022a0 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe695ffb640 1 -- 172.21.15.59:0/3903368098 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe688035070 con 0x7fe6981022a0 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe69f220640 1 -- 172.21.15.59:0/3903368098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe69819e430 con 0x7fe6981022a0 2024-05-08T05:39:34.021 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.017+0000 7fe69f220640 1 -- 172.21.15.59:0/3903368098 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6981a5d20 con 0x7fe6981022a0 2024-05-08T05:39:34.024 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.020+0000 7fe695ffb640 1 -- 172.21.15.59:0/3903368098 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fe688042030 con 0x7fe6981022a0 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.020+0000 7fe6a2013740 1 Processor -- start 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.020+0000 7fe6a2013740 1 -- start start 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.020+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 0x5597cf6722f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.020+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf672830 0x5597cf67cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.020+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 0x5597cf67f4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.020+0000 7fe6a2013740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5597cf5d75d0 con 0x5597cf672830 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.020+0000 7fe6a2013740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5597cf590890 con 0x5597cf671f10 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.020+0000 7fe6a2013740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5597cf5d3590 con 0x5597cf67d0f0 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69cfbc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf672830 0x5597cf67cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69cfbc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf672830 0x5597cf67cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45806/0 (socket says 172.21.15.59:45806) 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69cfbc640 1 -- 172.21.15.59:0/99910865 learned_addr learned my addr 172.21.15.59:0/99910865 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:34.025 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe697fff640 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 0x5597cf67f4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 0x5597cf6722f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69cfbc640 1 -- 172.21.15.59:0/99910865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 msgr2=0x5597cf67f4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 0x5597cf67f4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69cfbc640 1 -- 172.21.15.59:0/99910865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 msgr2=0x5597cf6722f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 0x5597cf6722f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69cfbc640 1 -- 172.21.15.59:0/99910865 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597cf2e9210 con 0x5597cf672830 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 0x5597cf6722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe695ffb640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe6840b2a10 0x7fe6840b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe6840b2a10 0x7fe6840b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf672830 0x5597cf67cbb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe68000bc90 tx=0x7fe6800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb05cd6607cb5e83 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.021+0000 7fe695ffb640 1 -- 172.21.15.59:0/3903368098 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fe688103220 con 0x7fe6981022a0 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.022+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe6840b2a10 0x7fe6840b4ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe690005fd0 tx=0x7fe690009500 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.022+0000 7fe68effd640 1 -- 172.21.15.59:0/99910865 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68001c480 con 0x5597cf672830 2024-05-08T05:39:34.026 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.022+0000 7fe68effd640 1 -- 172.21.15.59:0/99910865 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68001cad0 con 0x5597cf672830 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.022+0000 7fe68effd640 1 -- 172.21.15.59:0/99910865 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe680011b50 con 0x5597cf672830 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 -- 172.21.15.59:0/99910865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf672830 msgr2=0x5597cf67cbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf672830 0x5597cf67cbb0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7fe68000bc90 tx=0x7fe6800107a0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 -- 172.21.15.59:0/99910865 shutdown_connections 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 0x5597cf67f4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf672830 0x5597cf67cbb0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 --2- 172.21.15.59:0/99910865 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 0x5597cf6722f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 -- 172.21.15.59:0/99910865 >> 172.21.15.59:0/99910865 conn(0x5597cf66e420 msgr2=0x5597cf66e800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 -- 172.21.15.59:0/99910865 shutdown_connections 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 -- 172.21.15.59:0/99910865 wait complete. 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 -- start start 2024-05-08T05:39:34.027 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.023+0000 7fe6a2013740 1 Processor -- start 2024-05-08T05:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf67a070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 0x5597cf67a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.031 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf67d0f0 0x5597cf67aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe6a2013740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x5597cf67fd20 con 0x5597cf671f10 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe6a2013740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x5597cf594370 con 0x5597cf67d0f0 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe6a2013740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x5597cf67fea0 con 0x5597cf672830 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe6a2013740 10 client.0 _ll_register_callbacks cb 0x5597cf2e8a00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe69cfbc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf67a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe69cfbc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf67a070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45816/0 (socket says 172.21.15.59:45816) 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe69cfbc640 1 -- 172.21.15.59:0/3304250564 learned_addr learned my addr 172.21.15.59:0/3304250564 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf67d0f0 0x5597cf67aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe697fff640 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 0x5597cf67a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe69cfbc640 1 -- 172.21.15.59:0/3304250564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 msgr2=0x5597cf67a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 0x5597cf67a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe69cfbc640 1 -- 172.21.15.59:0/3304250564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf67d0f0 msgr2=0x5597cf67aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:34.032 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf67d0f0 0x5597cf67aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe69cfbc640 1 -- 172.21.15.59:0/3304250564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597cf3b5620 con 0x5597cf671f10 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.024+0000 7fe697fff640 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 0x5597cf67a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf67a070 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe680011010 tx=0x7fe68001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ec3162a38bc88fe server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe677fff640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe677fff640 1 -- 172.21.15.59:0/3304250564 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68001c690 con 0x5597cf671f10 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe677fff640 1 -- 172.21.15.59:0/3304250564 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fe68001c830 con 0x5597cf671f10 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe6a2013740 1 -- 172.21.15.59:0/3304250564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe68001cad0 con 0x5597cf671f10 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe677fff640 1 -- 172.21.15.59:0/3304250564 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe68002f050 con 0x5597cf671f10 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe6a2013740 10 client.16647 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe6a2013740 1 -- 172.21.15.59:0/3304250564 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5597cf68bbd0 con 0x5597cf671f10 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe677fff640 1 -- 172.21.15.59:0/3304250564 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe68001cad0 con 0x5597cf671f10 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe677fff640 1 -- 172.21.15.59:0/3304250564 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fe680005420 con 0x5597cf671f10 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe6a2013740 10 client.16647 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe6a2013740 10 client.16647 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:39:34.033 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe6a2013740 20 client.16647 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe6a2013740 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5597cf68d030 0x5597cf68f420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe6a2013740 4 client.16647 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.025+0000 7fe6a2013740 1 -- 172.21.15.59:0/3304250564 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5597cf5e51e0 con 0x5597cf68d030 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.026+0000 7fe697fff640 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5597cf68d030 0x5597cf68f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.026+0000 7fe677fff640 10 client.16647 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.026+0000 7fe697fff640 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5597cf68d030 0x5597cf68f420 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eab6b72cd890bce7 server_cookie=f469ce1e4b12fd29 in_seq=0 out_seq=0 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe677fff640 1 -- 172.21.15.59:0/3304250564 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x5597cf5e51e0 con 0x5597cf68d030 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe677fff640 10 client.16647 handle_command_reply: tid=0 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 client.16647 shutdown 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 -- 172.21.15.59:0/3304250564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 msgr2=0x5597cf67a070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf67a070 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7fe680011010 tx=0x7fe68001ce90 comp rx=0 tx=0).stop 2024-05-08T05:39:34.034 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 -- 172.21.15.59:0/3304250564 shutdown_connections 2024-05-08T05:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5597cf68d030 0x5597cf68f420 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf67d0f0 0x5597cf67aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 0x5597cf67a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 --2- 172.21.15.59:0/3304250564 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf67a070 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe677fff640 0 client.16647 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 -- 172.21.15.59:0/3304250564 >> 172.21.15.59:0/3304250564 conn(0x5597cf66e420 msgr2=0x5597cf6741a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.027+0000 7fe6a2013740 1 -- 172.21.15.59:0/3304250564 shutdown_connections 2024-05-08T05:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.028+0000 7fe6a2013740 1 -- 172.21.15.59:0/3304250564 wait complete. 2024-05-08T05:39:34.035 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.028+0000 7fe6a2013740 20 client.16647 trim_cache size 0 max 16384 2024-05-08T05:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe6a2013740 1 Processor -- start 2024-05-08T05:39:34.045 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe6a2013740 1 -- start start 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 0x5597cf67a910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 0x5597cf6949f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf67d0f0 0x5597cf694f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe6a2013740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981afd20 con 0x5597cf67d0f0 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe6a2013740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981b1890 con 0x5597cf671f10 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe6a2013740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981b03f0 con 0x5597cf672830 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe69d7bd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf67d0f0 0x5597cf694f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe69d7bd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf67d0f0 0x5597cf694f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45826/0 (socket says 172.21.15.59:45826) 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe69d7bd640 1 -- 172.21.15.59:0/1578621690 learned_addr learned my addr 172.21.15.59:0/1578621690 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.043+0000 7fe697fff640 1 --2- 172.21.15.59:0/1578621690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 0x5597cf6949f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe69d7bd640 1 -- 172.21.15.59:0/1578621690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 msgr2=0x5597cf6949f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/1578621690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 0x5597cf6949f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.046 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe69d7bd640 1 -- 172.21.15.59:0/1578621690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 msgr2=0x5597cf67a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/1578621690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 0x5597cf67a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/1578621690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 0x5597cf67a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe69d7bd640 1 -- 172.21.15.59:0/1578621690 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5597cf67f4e0 con 0x5597cf67d0f0 2024-05-08T05:39:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/1578621690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf67d0f0 0x5597cf694f30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe6900106e0 tx=0x7fe690015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26823d2ec6f525bd server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe68e7fc640 1 -- 172.21.15.59:0/1578621690 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69002c070 con 0x5597cf67d0f0 2024-05-08T05:39:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe68e7fc640 1 -- 172.21.15.59:0/1578621690 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe690010e70 con 0x5597cf67d0f0 2024-05-08T05:39:34.047 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.044+0000 7fe68e7fc640 1 -- 172.21.15.59:0/1578621690 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe690027a60 con 0x5597cf67d0f0 2024-05-08T05:39:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 -- 172.21.15.59:0/1578621690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf67d0f0 msgr2=0x5597cf694f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:34.050 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1578621690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf67d0f0 0x5597cf694f30 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7fe6900106e0 tx=0x7fe690015000 comp rx=0 tx=0).stop 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 -- 172.21.15.59:0/1578621690 shutdown_connections 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1578621690 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf67d0f0 0x5597cf694f30 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1578621690 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf672830 0x5597cf6949f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1578621690 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf671f10 0x5597cf67a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 -- 172.21.15.59:0/1578621690 >> 172.21.15.59:0/1578621690 conn(0x5597cf66e420 msgr2=0x5597cf6693e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 -- 172.21.15.59:0/1578621690 shutdown_connections 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 -- 172.21.15.59:0/1578621690 wait complete. 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 -- start start 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.045+0000 7fe6a2013740 1 Processor -- start 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf676e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf672830 0x5597cf677390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe6a2013740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 0x5597cf6778d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe6a2013740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981b11c0 con 0x5597cf671f10 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe6a2013740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981b0af0 con 0x5597cf672830 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe6a2013740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fe6981add20 con 0x5597cf67d0f0 2024-05-08T05:39:34.051 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe6a2013740 10 client.0 _ll_register_callbacks cb 0x7fe688106c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69cfbc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf676e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69cfbc640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf676e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45834/0 (socket says 172.21.15.59:45834) 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69cfbc640 1 -- 172.21.15.59:0/1803435707 learned_addr learned my addr 172.21.15.59:0/1803435707 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69d7bd640 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 0x5597cf6778d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe697fff640 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf672830 0x5597cf677390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69cfbc640 1 -- 172.21.15.59:0/1803435707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 msgr2=0x5597cf6778d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 0x5597cf6778d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69cfbc640 1 -- 172.21.15.59:0/1803435707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf672830 msgr2=0x5597cf677390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf672830 0x5597cf677390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69cfbc640 1 -- 172.21.15.59:0/1803435707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5597cf6f65a0 con 0x5597cf671f10 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe69cfbc640 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf676e50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe68000b730 tx=0x7fe680004980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69084227963f09e6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe677fff640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.046+0000 7fe677fff640 1 -- 172.21.15.59:0/1803435707 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe68001c690 con 0x5597cf671f10 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe6a2013740 1 -- 172.21.15.59:0/1803435707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe690010e70 con 0x5597cf671f10 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe677fff640 1 -- 172.21.15.59:0/1803435707 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fe68001c830 con 0x5597cf671f10 2024-05-08T05:39:34.052 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe677fff640 1 -- 172.21.15.59:0/1803435707 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7fe680041050 con 0x5597cf671f10 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe6a2013740 10 client.16653 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe6a2013740 1 -- 172.21.15.59:0/1803435707 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x5597cf6f7830 con 0x5597cf671f10 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe677fff640 1 -- 172.21.15.59:0/1803435707 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe690010e70 con 0x5597cf671f10 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe677fff640 1 -- 172.21.15.59:0/1803435707 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7fe680004ca0 con 0x5597cf671f10 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe6a2013740 10 client.16653 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe6a2013740 10 client.16653 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe6a2013740 20 client.16653 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5597cf6f7f70 0x5597cf6fa390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe6a2013740 4 client.16653 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe6a2013740 1 -- 172.21.15.59:0/1803435707 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5597cf5df050 con 0x5597cf6f7f70 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.047+0000 7fe697fff640 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5597cf6f7f70 0x5597cf6fa390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.048+0000 7fe677fff640 10 client.16653 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:34.053 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.048+0000 7fe697fff640 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5597cf6f7f70 0x5597cf6fa390 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b53aa4476510574 server_cookie=189a0070eabe4e8d in_seq=0 out_seq=0 2024-05-08T05:39:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe677fff640 1 -- 172.21.15.59:0/1803435707 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5597cf5df050 con 0x5597cf6f7f70 2024-05-08T05:39:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe677fff640 10 client.16653 handle_command_reply: tid=0 2024-05-08T05:39:34.064 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe6a2013740 1 client.16653 shutdown 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe6a2013740 1 -- 172.21.15.59:0/1803435707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 msgr2=0x5597cf676e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf676e50 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fe68000b730 tx=0x7fe680004980 comp rx=0 tx=0).stop 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe6a2013740 1 -- 172.21.15.59:0/1803435707 shutdown_connections 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x5597cf6f7f70 0x5597cf6fa390 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x5597cf67d0f0 0x5597cf6778d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x5597cf672830 0x5597cf677390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe6a2013740 1 --2- 172.21.15.59:0/1803435707 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x5597cf671f10 0x5597cf676e50 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe677fff640 0 client.16653 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.062+0000 7fe6a2013740 1 -- 172.21.15.59:0/1803435707 >> 172.21.15.59:0/1803435707 conn(0x5597cf66e420 msgr2=0x5597cf66ccf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe6a2013740 1 -- 172.21.15.59:0/1803435707 shutdown_connections 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe6a2013740 1 -- 172.21.15.59:0/1803435707 wait complete. 2024-05-08T05:39:34.065 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe6a2013740 20 client.16653 trim_cache size 0 max 16384 2024-05-08T05:39:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:39:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:39:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:39:34.066 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 -- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe6840b2a10 msgr2=0x7fe6840b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe6840b2a10 0x7fe6840b4ed0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe690005fd0 tx=0x7fe690009500 comp rx=0 tx=0).stop 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 -- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981022a0 msgr2=0x7fe6981996f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981022a0 0x7fe6981996f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7fe688005940 tx=0x7fe6880042f0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 -- 172.21.15.59:0/3903368098 shutdown_connections 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fe6840b2a10 0x7fe6840b4ed0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fe6981078a0 0x7fe69819da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fe6981022a0 0x7fe6981996f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 --2- 172.21.15.59:0/3903368098 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fe6981018e0 0x7fe6981991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.063+0000 7fe69f220640 1 -- 172.21.15.59:0/3903368098 >> 172.21.15.59:0/3903368098 conn(0x7fe698078a30 msgr2=0x7fe69806a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:34.067 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.064+0000 7fe69f220640 1 -- 172.21.15.59:0/3903368098 shutdown_connections 2024-05-08T05:39:34.068 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:34.064+0000 7fe69f220640 1 -- 172.21.15.59:0/3903368098 wait complete. 2024-05-08T05:39:35.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:35 smithi096 ceph-mon[34253]: pgmap v312: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-05-08T05:39:35.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:35 smithi059 ceph-mon[30304]: pgmap v312: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-05-08T05:39:35.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:35 smithi143 ceph-mon[34337]: pgmap v312: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 9.5 KiB/s wr, 13 op/s 2024-05-08T05:39:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:37 smithi059 ceph-mon[30304]: pgmap v313: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-05-08T05:39:37.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:37 smithi143 ceph-mon[34337]: pgmap v313: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-05-08T05:39:37.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:37 smithi096 ceph-mon[34253]: pgmap v313: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 17 KiB/s wr, 20 op/s 2024-05-08T05:39:38.268 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:39:38.540 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:39:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:39 smithi059 ceph-mon[30304]: pgmap v314: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-05-08T05:39:39.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:39 smithi143 ceph-mon[34337]: pgmap v314: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-05-08T05:39:39.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:39 smithi096 ceph-mon[34253]: pgmap v314: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-05-08T05:39:40.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.657+0000 7f65e29a0700 1 -- 172.21.15.59:0/537924635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc103fc0 msgr2=0x7f65dc108010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.657+0000 7f65e29a0700 1 --2- 172.21.15.59:0/537924635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc103fc0 0x7f65dc108010 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f65cc0099c0 tx=0x7f65cc009cd0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.657+0000 7f65e29a0700 1 -- 172.21.15.59:0/537924635 shutdown_connections 2024-05-08T05:39:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.657+0000 7f65e29a0700 1 --2- 172.21.15.59:0/537924635 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65dc1086f0 0x7f65dc1102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.657+0000 7f65e29a0700 1 --2- 172.21.15.59:0/537924635 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc103610 0x7f65dc1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.657+0000 7f65e29a0700 1 --2- 172.21.15.59:0/537924635 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc103fc0 0x7f65dc108010 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.657+0000 7f65e29a0700 1 -- 172.21.15.59:0/537924635 >> 172.21.15.59:0/537924635 conn(0x7f65dc0feec0 msgr2=0x7f65dc1012e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.657+0000 7f65e29a0700 1 -- 172.21.15.59:0/537924635 shutdown_connections 2024-05-08T05:39:40.659 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.657+0000 7f65e29a0700 1 -- 172.21.15.59:0/537924635 wait complete. 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e29a0700 1 Processor -- start 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e29a0700 1 -- start start 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e29a0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65dc103610 0x7f65dc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e29a0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc103fc0 0x7f65dc075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e29a0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc1086f0 0x7f65dc076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e29a0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc1136b0 con 0x7f65dc1086f0 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e29a0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc113530 con 0x7f65dc103610 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e29a0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f65dc113830 con 0x7f65dc103fc0 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e219f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc1086f0 0x7f65dc076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:40.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e199e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65dc103610 0x7f65dc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e219f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc1086f0 0x7f65dc076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55676/0 (socket says 172.21.15.59:55676) 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e219f700 1 -- 172.21.15.59:0/2841016634 learned_addr learned my addr 172.21.15.59:0/2841016634 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e199e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65dc103610 0x7f65dc0757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:37114/0 (socket says 172.21.15.59:37114) 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e119d700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc103fc0 0x7f65dc075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e219f700 1 -- 172.21.15.59:0/2841016634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc103fc0 msgr2=0x7f65dc075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e219f700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc103fc0 0x7f65dc075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e219f700 1 -- 172.21.15.59:0/2841016634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65dc103610 msgr2=0x7f65dc0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e219f700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65dc103610 0x7f65dc0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.658+0000 7f65e219f700 1 -- 172.21.15.59:0/2841016634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65dc1a8100 con 0x7f65dc1086f0 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.659+0000 7f65e219f700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc1086f0 0x7f65dc076220 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f65d400b970 tx=0x7f65d400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.659+0000 7f65d2ffd700 1 -- 172.21.15.59:0/2841016634 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d400d860 con 0x7f65dc1086f0 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.659+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65dc1a83f0 con 0x7f65dc1086f0 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.659+0000 7f65d2ffd700 1 -- 172.21.15.59:0/2841016634 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65d400e430 con 0x7f65dc1086f0 2024-05-08T05:39:40.662 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.659+0000 7f65d2ffd700 1 -- 172.21.15.59:0/2841016634 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65d4017660 con 0x7f65dc1086f0 2024-05-08T05:39:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.659+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65dc1a8900 con 0x7f65dc1086f0 2024-05-08T05:39:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.660+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65dc104af0 con 0x7f65dc1086f0 2024-05-08T05:39:40.663 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.662+0000 7f65d2ffd700 1 -- 172.21.15.59:0/2841016634 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f65d400da00 con 0x7f65dc1086f0 2024-05-08T05:39:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.663+0000 7f65d2ffd700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f65c80b29f0 0x7f65c80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.663+0000 7f65d2ffd700 1 -- 172.21.15.59:0/2841016634 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f65d40e2810 con 0x7f65dc1086f0 2024-05-08T05:39:40.664 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.663+0000 7f65e199e700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f65c80b29f0 0x7f65c80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:40.665 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.663+0000 7f65e199e700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f65c80b29f0 0x7f65c80b4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009a10 tx=0x7f65d8009190 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:40.667 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.666+0000 7f65d2ffd700 1 -- 172.21.15.59:0/2841016634 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f65d4053df0 con 0x7f65dc1086f0 2024-05-08T05:39:40.940 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.939+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f65dc076c80 con 0x7f65dc1086f0 2024-05-08T05:39:40.941 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.940+0000 7f65d2ffd700 1 -- 172.21.15.59:0/2841016634 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) ==== 94+0+11930 (secure 0 0 0) 0x7f65d4013030 con 0x7f65dc1086f0 2024-05-08T05:39:40.941 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:39:40.942 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":39,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":39,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:39:03.820486+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25876":{"gid":25876,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2351774265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2351774265},{"type":"v1","addr":"172.21.15.143:6835","nonce":2351774265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24433,24419,24421,24473,24443]},"id":1}]} 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f65c80b29f0 msgr2=0x7f65c80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f65c80b29f0 0x7f65c80b4eb0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f65d8009a10 tx=0x7f65d8009190 comp rx=0 tx=0).stop 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc1086f0 msgr2=0x7f65dc076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc1086f0 0x7f65dc076220 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f65d400b970 tx=0x7f65d400bc80 comp rx=0 tx=0).stop 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 shutdown_connections 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f65c80b29f0 0x7f65c80b4eb0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f65dc1086f0 0x7f65dc076220 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f65dc103610 0x7f65dc0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 --2- 172.21.15.59:0/2841016634 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f65dc103fc0 0x7f65dc075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 >> 172.21.15.59:0/2841016634 conn(0x7f65dc0feec0 msgr2=0x7f65dc101260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 shutdown_connections 2024-05-08T05:39:40.944 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:40.942+0000 7f65e29a0700 1 -- 172.21.15.59:0/2841016634 wait complete. 2024-05-08T05:39:40.945 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 39 2024-05-08T05:39:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:41 smithi059 ceph-mon[30304]: pgmap v315: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 14 op/s 2024-05-08T05:39:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:41 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2841016634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:39:41.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:41 smithi143 ceph-mon[34337]: pgmap v315: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 14 op/s 2024-05-08T05:39:41.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:41 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2841016634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:39:41.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:41 smithi096 ceph-mon[34253]: pgmap v315: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 7.8 KiB/s wr, 14 op/s 2024-05-08T05:39:41.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:41 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2841016634' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:39:41.795 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db --set-id df019e95 --release --await' 2024-05-08T05:39:41.795 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db --set-id df019e95 --release --await 2024-05-08T05:39:42.062 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:39:43.520 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:43 smithi059 ceph-mon[30304]: pgmap v316: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-05-08T05:39:43.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:43 smithi143 ceph-mon[34337]: pgmap v316: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-05-08T05:39:43.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:43 smithi096 ceph-mon[34253]: pgmap v316: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-05-08T05:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.919+0000 7f2c2cda5700 1 -- 172.21.15.59:0/3928694227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806c630 msgr2=0x7f2c28110360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.919+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/3928694227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806c630 0x7f2c28110360 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000b0d0 tx=0x7f2c2000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.919+0000 7f2c2cda5700 1 -- 172.21.15.59:0/3928694227 shutdown_connections 2024-05-08T05:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.919+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/3928694227 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806c630 0x7f2c28110360 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.919+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/3928694227 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c28109590 0x7f2c28109970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.919+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/3928694227 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c2806bc70 0x7f2c2806c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.919+0000 7f2c2cda5700 1 -- 172.21.15.59:0/3928694227 >> 172.21.15.59:0/3928694227 conn(0x7f2c28079440 msgr2=0x7f2c28079850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.919+0000 7f2c2cda5700 1 -- 172.21.15.59:0/3928694227 shutdown_connections 2024-05-08T05:39:43.921 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.919+0000 7f2c2cda5700 1 -- 172.21.15.59:0/3928694227 wait complete. 2024-05-08T05:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.920+0000 7f2c2cda5700 1 Processor -- start 2024-05-08T05:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.920+0000 7f2c2cda5700 1 -- start start 2024-05-08T05:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.920+0000 7f2c2cda5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806bc70 0x7f2c2819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.923 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.920+0000 7f2c2cda5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c2806c630 0x7f2c2819fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.920+0000 7f2c2cda5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c28109590 0x7f2c281998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.920+0000 7f2c2cda5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281136b0 con 0x7f2c2806bc70 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.920+0000 7f2c2cda5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28113530 con 0x7f2c2806c630 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.920+0000 7f2c2cda5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2c28113830 con 0x7f2c28109590 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.920+0000 7f2c2659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806bc70 0x7f2c2819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2659c700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806bc70 0x7f2c2819f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55694/0 (socket says 172.21.15.59:55694) 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2659c700 1 -- 172.21.15.59:0/1208433695 learned_addr learned my addr 172.21.15.59:0/1208433695 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c25d9b700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c2806c630 0x7f2c2819fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c28109590 0x7f2c281998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2659c700 1 -- 172.21.15.59:0/1208433695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c28109590 msgr2=0x7f2c281998f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2659c700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c28109590 0x7f2c281998f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2659c700 1 -- 172.21.15.59:0/1208433695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c2806c630 msgr2=0x7f2c2819fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2659c700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c2806c630 0x7f2c2819fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2659c700 1 -- 172.21.15.59:0/1208433695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c2819a050 con 0x7f2c2806bc70 2024-05-08T05:39:43.924 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c28109590 0x7f2c281998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:39:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2659c700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806bc70 0x7f2c2819f730 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800d950 tx=0x7f2c1800dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c177fe700 1 -- 172.21.15.59:0/1208433695 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c18016070 con 0x7f2c2806bc70 2024-05-08T05:39:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c177fe700 1 -- 172.21.15.59:0/1208433695 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c18009df0 con 0x7f2c2806bc70 2024-05-08T05:39:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2cda5700 1 -- 172.21.15.59:0/1208433695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c2819a340 con 0x7f2c2806bc70 2024-05-08T05:39:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c177fe700 1 -- 172.21.15.59:0/1208433695 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1800f3c0 con 0x7f2c2806bc70 2024-05-08T05:39:43.925 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.921+0000 7f2c2cda5700 1 -- 172.21.15.59:0/1208433695 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c281a6600 con 0x7f2c2806bc70 2024-05-08T05:39:43.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.924+0000 7f2c177fe700 1 -- 172.21.15.59:0/1208433695 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f2c18019070 con 0x7f2c2806bc70 2024-05-08T05:39:43.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.924+0000 7f2c177fe700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2c100b2af0 0x7f2c100b4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c25d9b700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2c100b2af0 0x7f2c100b4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.926 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c177fe700 1 -- 172.21.15.59:0/1208433695 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2c180e2630 con 0x7f2c2806bc70 2024-05-08T05:39:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c37638b80 1 Processor -- start 2024-05-08T05:39:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c25d9b700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2c100b2af0 0x7f2c100b4fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2c28079940 tx=0x7f2c1c00b3c0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c37638b80 1 -- start start 2024-05-08T05:39:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa63380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa5e7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa5f220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c37638b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fc9f9b0960 con 0x55fc9fa638c0 2024-05-08T05:39:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c37638b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc9f94f8c0 con 0x55fc9fa5edc0 2024-05-08T05:39:43.928 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.925+0000 7f2c37638b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fc9f9ac920 con 0x55fc9fa62fa0 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c25d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa5e7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c25d9b700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa5e7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55702/0 (socket says 172.21.15.59:55702) 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c25d9b700 1 -- 172.21.15.59:0/1878943683 learned_addr learned my addr 172.21.15.59:0/1878943683 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c2659c700 1 --2- 172.21.15.59:0/1878943683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa63380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/1878943683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa5f220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c25d9b700 1 -- 172.21.15.59:0/1878943683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa62fa0 msgr2=0x55fc9fa63380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c25d9b700 1 --2- 172.21.15.59:0/1878943683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa63380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c25d9b700 1 -- 172.21.15.59:0/1878943683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa5edc0 msgr2=0x55fc9fa5f220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c25d9b700 1 --2- 172.21.15.59:0/1878943683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa5f220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c25d9b700 1 -- 172.21.15.59:0/1878943683 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc9f8fd9d0 con 0x55fc9fa638c0 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c25d9b700 1 --2- 172.21.15.59:0/1878943683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa5e7a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b670 tx=0x7f2c1c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64a0dff0e91a7c7b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c14ff9700 1 -- 172.21.15.59:0/1878943683 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c0118d0 con 0x55fc9fa638c0 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c14ff9700 1 -- 172.21.15.59:0/1878943683 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c1c004830 con 0x55fc9fa638c0 2024-05-08T05:39:43.929 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.926+0000 7f2c14ff9700 1 -- 172.21.15.59:0/1878943683 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c1c005400 con 0x55fc9fa638c0 2024-05-08T05:39:43.933 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.927+0000 7f2c37638b80 1 -- 172.21.15.59:0/1878943683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 msgr2=0x55fc9fa5e7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.927+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1878943683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa5e7a0 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f2c1c00b670 tx=0x7f2c1c013000 comp rx=0 tx=0).stop 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.927+0000 7f2c37638b80 1 -- 172.21.15.59:0/1878943683 shutdown_connections 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.927+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1878943683 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa5f220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.927+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1878943683 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa63380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.927+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1878943683 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa5e7a0 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.927+0000 7f2c37638b80 1 -- 172.21.15.59:0/1878943683 >> 172.21.15.59:0/1878943683 conn(0x55fc9f98a200 msgr2=0x55fc9fa58fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.927+0000 7f2c37638b80 1 -- 172.21.15.59:0/1878943683 shutdown_connections 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.927+0000 7f2c37638b80 1 -- 172.21.15.59:0/1878943683 wait complete. 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c37638b80 1 -- start start 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c37638b80 1 Processor -- start 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa6a630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa67a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa67fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.934 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c37638b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55fc9fa6b490 con 0x55fc9fa638c0 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c37638b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55fc9f916d30 con 0x55fc9fa62fa0 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c37638b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55fc9fa6b610 con 0x55fc9fa5edc0 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c37638b80 10 client.0 _ll_register_callbacks cb 0x55fc9f948a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c26d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa67fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c25d9b700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa67a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c26d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa67fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55714/0 (socket says 172.21.15.59:55714) 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c26d9d700 1 -- 172.21.15.59:0/909425306 learned_addr learned my addr 172.21.15.59:0/909425306 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c2659c700 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa6a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c26d9d700 1 -- 172.21.15.59:0/909425306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 msgr2=0x55fc9fa6a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa6a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c26d9d700 1 -- 172.21.15.59:0/909425306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 msgr2=0x55fc9fa67a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa67a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.928+0000 7f2c26d9d700 1 -- 172.21.15.59:0/909425306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc9f94c790 con 0x55fc9fa638c0 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa67fd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2c200095b0 tx=0x7f2c20004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95ec21822b69dc8d server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:43.935 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c0dffb700 1 -- 172.21.15.59:0/909425306 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c200076e0 con 0x55fc9fa638c0 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c0dffb700 1 -- 172.21.15.59:0/909425306 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2c20007880 con 0x55fc9fa638c0 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c37638b80 1 -- 172.21.15.59:0/909425306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c1c004830 con 0x55fc9fa638c0 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c0dffb700 1 -- 172.21.15.59:0/909425306 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2c20007a80 con 0x55fc9fa638c0 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c37638b80 10 client.16671 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c37638b80 1 -- 172.21.15.59:0/909425306 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fc9fa75c90 con 0x55fc9fa638c0 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c0dffb700 1 -- 172.21.15.59:0/909425306 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c1c004830 con 0x55fc9fa638c0 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c0dffb700 1 -- 172.21.15.59:0/909425306 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f2c20011070 con 0x55fc9fa638c0 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c37638b80 10 client.16671 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c37638b80 10 client.16671 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c37638b80 20 client.16671 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.929+0000 7f2c37638b80 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc9fa78370 0x55fc9fa7a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.936 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.930+0000 7f2c37638b80 4 client.16671 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.930+0000 7f2c37638b80 1 -- 172.21.15.59:0/909425306 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fc9fa2a750 con 0x55fc9fa78370 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.930+0000 7f2c2659c700 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc9fa78370 0x55fc9fa7a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.930+0000 7f2c2659c700 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc9fa78370 0x55fc9fa7a790 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce03c30d79e9ae36 server_cookie=68fefef8d63321d in_seq=0 out_seq=0 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.930+0000 7f2c0dffb700 10 client.16671 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c0dffb700 1 -- 172.21.15.59:0/909425306 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55fc9fa2a750 con 0x55fc9fa78370 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c0dffb700 10 client.16671 handle_command_reply: tid=0 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 client.16671 shutdown 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 -- 172.21.15.59:0/909425306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 msgr2=0x55fc9fa67fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa67fd0 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f2c200095b0 tx=0x7f2c20004a50 comp rx=0 tx=0).stop 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 -- 172.21.15.59:0/909425306 shutdown_connections 2024-05-08T05:39:43.937 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc9fa78370 0x55fc9fa7a790 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c0dffb700 0 client.16671 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa67fd0 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa6a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 --2- 172.21.15.59:0/909425306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa67a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 -- 172.21.15.59:0/909425306 >> 172.21.15.59:0/909425306 conn(0x55fc9f98a200 msgr2=0x55fc9fa57da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 -- 172.21.15.59:0/909425306 shutdown_connections 2024-05-08T05:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 1 -- 172.21.15.59:0/909425306 wait complete. 2024-05-08T05:39:43.938 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.931+0000 7f2c37638b80 20 client.16671 trim_cache size 0 max 16384 2024-05-08T05:39:43.957 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.951+0000 7f2c37638b80 1 Processor -- start 2024-05-08T05:39:43.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.951+0000 7f2c37638b80 1 -- start start 2024-05-08T05:39:43.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.951+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa74520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.951+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.951+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa80040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.951+0000 7f2c37638b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281af5f0 con 0x55fc9fa638c0 2024-05-08T05:39:43.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.951+0000 7f2c37638b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b1160 con 0x55fc9fa62fa0 2024-05-08T05:39:43.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.951+0000 7f2c37638b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281afcc0 con 0x55fc9fa5edc0 2024-05-08T05:39:43.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c26d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa80040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.958 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c26d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa80040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55718/0 (socket says 172.21.15.59:55718) 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c26d9d700 1 -- 172.21.15.59:0/3398745648 learned_addr learned my addr 172.21.15.59:0/3398745648 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c2659c700 1 --2- 172.21.15.59:0/3398745648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa74520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c25d9b700 1 --2- 172.21.15.59:0/3398745648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c26d9d700 1 -- 172.21.15.59:0/3398745648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 msgr2=0x55fc9fa74520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/3398745648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa74520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c26d9d700 1 -- 172.21.15.59:0/3398745648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 msgr2=0x55fc9fa6a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/3398745648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa6a2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c26d9d700 1 -- 172.21.15.59:0/3398745648 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fc9fa53f40 con 0x55fc9fa638c0 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/3398745648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa80040 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009a20 tx=0x7f2c20009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fdfb56fd0874f9 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c0ffff700 1 -- 172.21.15.59:0/3398745648 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20005c60 con 0x55fc9fa638c0 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c0ffff700 1 -- 172.21.15.59:0/3398745648 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c20037070 con 0x55fc9fa638c0 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.952+0000 7f2c0ffff700 1 -- 172.21.15.59:0/3398745648 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20032480 con 0x55fc9fa638c0 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 -- 172.21.15.59:0/3398745648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 msgr2=0x55fc9fa80040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 --2- 172.21.15.59:0/3398745648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa80040 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f2c20009a20 tx=0x7f2c20009780 comp rx=0 tx=0).stop 2024-05-08T05:39:43.959 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 -- 172.21.15.59:0/3398745648 shutdown_connections 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 --2- 172.21.15.59:0/3398745648 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa80040 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 --2- 172.21.15.59:0/3398745648 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa74520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 --2- 172.21.15.59:0/3398745648 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 -- 172.21.15.59:0/3398745648 >> 172.21.15.59:0/3398745648 conn(0x55fc9f98a200 msgr2=0x55fc9fa58630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 -- 172.21.15.59:0/3398745648 shutdown_connections 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 -- 172.21.15.59:0/3398745648 wait complete. 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 -- start start 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.953+0000 7f2c37638b80 1 Processor -- start 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa70a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa70f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c37638b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa6d9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c37638b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b0a90 con 0x55fc9fa638c0 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c37638b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281b03c0 con 0x55fc9fa62fa0 2024-05-08T05:39:43.960 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c37638b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2c281ad5b0 con 0x55fc9fa5edc0 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c37638b80 10 client.0 _ll_register_callbacks cb 0x55fc9f948a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c26d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa6d9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c26d9d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa6d9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55730/0 (socket says 172.21.15.59:55730) 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c26d9d700 1 -- 172.21.15.59:0/1202772017 learned_addr learned my addr 172.21.15.59:0/1202772017 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c2659c700 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa70a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c25d9b700 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa70f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c26d9d700 1 -- 172.21.15.59:0/1202772017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 msgr2=0x55fc9fa70a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa70a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c26d9d700 1 -- 172.21.15.59:0/1202772017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 msgr2=0x55fc9fa70f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa70f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c26d9d700 1 -- 172.21.15.59:0/1202772017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fc9fa6e200 con 0x55fc9fa638c0 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c25d9b700 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa70f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:39:43.961 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c26d9d700 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa6d9f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000ba60 tx=0x7f2c20009010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a40b4fad3523524 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c0dffb700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c0dffb700 1 -- 172.21.15.59:0/1202772017 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c20005660 con 0x55fc9fa638c0 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.954+0000 7f2c0dffb700 1 -- 172.21.15.59:0/1202772017 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2c2002b020 con 0x55fc9fa638c0 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c37638b80 1 -- 172.21.15.59:0/1202772017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c20037070 con 0x55fc9fa638c0 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c0dffb700 1 -- 172.21.15.59:0/1202772017 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f2c20048050 con 0x55fc9fa638c0 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c37638b80 10 client.16677 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c37638b80 1 -- 172.21.15.59:0/1202772017 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x55fc9fae38f0 con 0x55fc9fa638c0 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c0dffb700 1 -- 172.21.15.59:0/1202772017 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c20037070 con 0x55fc9fa638c0 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c0dffb700 1 -- 172.21.15.59:0/1202772017 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f2c20011030 con 0x55fc9fa638c0 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c37638b80 10 client.16677 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c37638b80 10 client.16677 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c37638b80 20 client.16677 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc9fa78370 0x55fc9fae5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:43.962 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c37638b80 4 client.16677 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "df019e95", "release": true, "await": true}] 2024-05-08T05:39:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c37638b80 1 -- 172.21.15.59:0/1202772017 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "set_id": "df019e95", "release": true, "await": true}) -- 0x55fc9f9be340 con 0x55fc9fa78370 2024-05-08T05:39:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.955+0000 7f2c2659c700 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc9fa78370 0x55fc9fae5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.956+0000 7f2c2659c700 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc9fa78370 0x55fc9fae5ef0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b279de53f41895a server_cookie=8ebd0104fa2a3867 in_seq=0 out_seq=0 2024-05-08T05:39:43.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:43.956+0000 7f2c0dffb700 10 client.16677 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:44.122 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c0dffb700 1 -- 172.21.15.59:0/1202772017 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55fc9f9be340 con 0x55fc9fa78370 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c0dffb700 10 client.16677 handle_command_reply: tid=0 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 client.16677 shutdown 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 -- 172.21.15.59:0/1202772017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 msgr2=0x55fc9fa6d9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa6d9f0 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f2c2000ba60 tx=0x7f2c20009010 comp rx=0 tx=0).stop 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 -- 172.21.15.59:0/1202772017 shutdown_connections 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55fc9fa78370 0x55fc9fae5ef0 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55fc9fa638c0 0x55fc9fa6d9f0 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55fc9fa5edc0 0x55fc9fa70a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 --2- 172.21.15.59:0/1202772017 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55fc9fa62fa0 0x55fc9fa70f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 -- 172.21.15.59:0/1202772017 >> 172.21.15.59:0/1202772017 conn(0x55fc9f98a200 msgr2=0x55fc9fa58630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c0dffb700 0 client.16677 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 -- 172.21.15.59:0/1202772017 shutdown_connections 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 1 -- 172.21.15.59:0/1202772017 wait complete. 2024-05-08T05:39:44.123 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c37638b80 20 client.16677 trim_cache size 0 max 16384 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 39, 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 32, 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": { 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "df019e95": { 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "version": 32, 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "age_ref": 0.0, 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:39:44.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "timeout": 60.0, 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "expiration": 90.0, 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "members": { 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "file:/": { 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "excluded": false, 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "state": { 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RELEASED", 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: "age": 0.0 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:39:44.125 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:39:44.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c2cda5700 1 -- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2c100b2af0 msgr2=0x7f2c100b4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:44.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2c100b2af0 0x7f2c100b4fb0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2c28079940 tx=0x7f2c1c00b3c0 comp rx=0 tx=0).stop 2024-05-08T05:39:44.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c2cda5700 1 -- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806bc70 msgr2=0x7f2c2819f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:44.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.121+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806bc70 0x7f2c2819f730 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f2c1800d950 tx=0x7f2c1800dc60 comp rx=0 tx=0).stop 2024-05-08T05:39:44.126 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.122+0000 7f2c2cda5700 1 -- 172.21.15.59:0/1208433695 shutdown_connections 2024-05-08T05:39:44.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.122+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f2c100b2af0 0x7f2c100b4fb0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:44.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.122+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2c28109590 0x7f2c281998f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:44.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.122+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2c2806bc70 0x7f2c2819f730 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:44.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.122+0000 7f2c2cda5700 1 --2- 172.21.15.59:0/1208433695 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2c2806c630 0x7f2c2819fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:44.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.122+0000 7f2c2cda5700 1 -- 172.21.15.59:0/1208433695 >> 172.21.15.59:0/1208433695 conn(0x7f2c28079440 msgr2=0x7f2c28100e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:44.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.122+0000 7f2c2cda5700 1 -- 172.21.15.59:0/1208433695 shutdown_connections 2024-05-08T05:39:44.127 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:44.122+0000 7f2c2cda5700 1 -- 172.21.15.59:0/1208433695 wait complete. 2024-05-08T05:39:44.831 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set 'df019e95', seconds elapsed: 44.8 2024-05-08T05:39:45.545 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:45 smithi143 ceph-mon[34337]: pgmap v317: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-05-08T05:39:45.581 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:45 smithi096 ceph-mon[34253]: pgmap v317: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-05-08T05:39:45.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:45 smithi059 ceph-mon[30304]: pgmap v317: 129 pgs: 129 active+clean; 122 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 7.8 KiB/s wr, 10 op/s 2024-05-08T05:39:46.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:46 smithi143 ceph-mon[34337]: pgmap v318: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 133 KiB/s wr, 15 op/s 2024-05-08T05:39:46.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:46 smithi096 ceph-mon[34253]: pgmap v318: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 133 KiB/s wr, 15 op/s 2024-05-08T05:39:46.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:46 smithi059 ceph-mon[30304]: pgmap v318: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 133 KiB/s wr, 15 op/s 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.157+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3440217568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781089d0 msgr2=0x7f8c78108db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.157+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3440217568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781089d0 0x7f8c78108db0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f8c74009930 tx=0x7f8c7402a420 comp rx=0 tx=0).stop 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.157+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3440217568 shutdown_connections 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.157+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3440217568 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c78103380 0x7f8c7810f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.157+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3440217568 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c781029c0 0x7f8c78102e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.157+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3440217568 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781089d0 0x7f8c78108db0 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.157+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3440217568 >> 172.21.15.59:0/3440217568 conn(0x7f8c780fe4b0 msgr2=0x7f8c781008d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.157+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3440217568 shutdown_connections 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.157+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3440217568 wait complete. 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7fc93640 1 Processor -- start 2024-05-08T05:39:47.160 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7fc93640 1 -- start start 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7fc93640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781029c0 0x7f8c78074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7fc93640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78103380 0x7f8c780751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7fc93640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c781089d0 0x7f8c78075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7fc93640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78112c90 con 0x7f8c781029c0 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7fc93640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78112b10 con 0x7f8c781089d0 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7fc93640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8c78112e10 con 0x7f8c78103380 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7da2f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781029c0 0x7f8c78074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7da2f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781029c0 0x7f8c78074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55758/0 (socket says 172.21.15.59:55758) 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7da2f640 1 -- 172.21.15.59:0/3913552228 learned_addr learned my addr 172.21.15.59:0/3913552228 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7e230640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c781089d0 0x7f8c78075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78103380 0x7f8c780751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7da2f640 1 -- 172.21.15.59:0/3913552228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78103380 msgr2=0x7f8c780751f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:47.161 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78103380 0x7f8c780751f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7da2f640 1 -- 172.21.15.59:0/3913552228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c781089d0 msgr2=0x7f8c78075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c781089d0 0x7f8c78075730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.158+0000 7f8c7da2f640 1 -- 172.21.15.59:0/3913552228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c781a7690 con 0x7f8c781029c0 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.159+0000 7f8c7e230640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c781089d0 0x7f8c78075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.159+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78103380 0x7f8c780751f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.159+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781029c0 0x7f8c78074cb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005940 tx=0x7f8c740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.159+0000 7f8c6effd640 1 -- 172.21.15.59:0/3913552228 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c7403f030 con 0x7f8c781029c0 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.159+0000 7f8c6effd640 1 -- 172.21.15.59:0/3913552228 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c74003cb0 con 0x7f8c781029c0 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.159+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3913552228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c781a7920 con 0x7f8c781029c0 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.159+0000 7f8c6effd640 1 -- 172.21.15.59:0/3913552228 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74035070 con 0x7f8c781029c0 2024-05-08T05:39:47.162 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.159+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3913552228 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c781a7e10 con 0x7f8c781029c0 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.161+0000 7f8c6effd640 1 -- 172.21.15.59:0/3913552228 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f8c74042030 con 0x7f8c781029c0 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c829b4740 1 Processor -- start 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c829b4740 1 -- start start 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffea942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea9f0f0 0x556ffeaa14e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c829b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556ffe9f95d0 con 0x556ffea94830 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c829b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556ffe9b2890 con 0x556ffea93f10 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c829b4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556ffe9f5590 con 0x556ffea9f0f0 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c7da2f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.168 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c7e230640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffea942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c7da2f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55766/0 (socket says 172.21.15.59:55766) 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c7e230640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffea942f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.96:3300/0 says I am v2:172.21.15.59:37188/0 (socket says 172.21.15.59:37188) 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.162+0000 7f8c7da2f640 1 -- 172.21.15.59:0/2910519054 learned_addr learned my addr 172.21.15.59:0/2910519054 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/2910519054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea9f0f0 0x556ffeaa14e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c6effd640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8c640b2a10 0x7f8c640b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7da2f640 1 -- 172.21.15.59:0/2910519054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea9f0f0 msgr2=0x556ffeaa14e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/2910519054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea9f0f0 0x556ffeaa14e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7da2f640 1 -- 172.21.15.59:0/2910519054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 msgr2=0x556ffea942f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/2910519054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffea942f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7da2f640 1 -- 172.21.15.59:0/2910519054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ffe70b210 con 0x556ffea94830 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7e230640 1 --2- 172.21.15.59:0/2910519054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffea942f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7e230640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8c640b2a10 0x7f8c640b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c6effd640 1 -- 172.21.15.59:0/3913552228 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8c74103230 con 0x7f8c781029c0 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/2910519054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9ebb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8c74100490 tx=0x7f8c7408f680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c88d64ba55087b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:47.169 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c7e230640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8c640b2a10 0x7f8c640b4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009fd0 tx=0x7f8c70004a40 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c63fff640 1 -- 172.21.15.59:0/2910519054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c740c9eb0 con 0x556ffea94830 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c63fff640 1 -- 172.21.15.59:0/2910519054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c7403b460 con 0x556ffea94830 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.163+0000 7f8c63fff640 1 -- 172.21.15.59:0/2910519054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c740044e0 con 0x556ffea94830 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 -- 172.21.15.59:0/2910519054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 msgr2=0x556ffea9ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2910519054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9ebb0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f8c74100490 tx=0x7f8c7408f680 comp rx=0 tx=0).stop 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 -- 172.21.15.59:0/2910519054 shutdown_connections 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2910519054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea9f0f0 0x556ffeaa14e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2910519054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9ebb0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2910519054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffea942f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 -- 172.21.15.59:0/2910519054 >> 172.21.15.59:0/2910519054 conn(0x556ffea90420 msgr2=0x556ffea90800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 -- 172.21.15.59:0/2910519054 shutdown_connections 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 -- 172.21.15.59:0/2910519054 wait complete. 2024-05-08T05:39:47.170 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 -- start start 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.164+0000 7f8c829b4740 1 Processor -- start 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea93f10 0x556ffea9c000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9c540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 0x556ffea9ca80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c829b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x556ffeaa1ae0 con 0x556ffea94830 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c829b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x556ffe9b6370 con 0x556ffea9f0f0 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c829b4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x556ffeaa1c60 con 0x556ffea93f10 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c829b4740 10 client.0 _ll_register_callbacks cb 0x556ffe70aa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7d22e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9c540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7d22e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9c540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55780/0 (socket says 172.21.15.59:55780) 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7d22e640 1 -- 172.21.15.59:0/2760885181 learned_addr learned my addr 172.21.15.59:0/2760885181 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7e230640 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 0x556ffea9ca80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea93f10 0x556ffea9c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7d22e640 1 -- 172.21.15.59:0/2760885181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea93f10 msgr2=0x556ffea9c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:47.171 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea93f10 0x556ffea9c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7d22e640 1 -- 172.21.15.59:0/2760885181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 msgr2=0x556ffea9ca80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 0x556ffea9ca80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7d22e640 1 -- 172.21.15.59:0/2760885181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffe7d7620 con 0x556ffea94830 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.165+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea93f10 0x556ffea9c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9c540 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800e300 tx=0x7f8c6800e7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f974794ab5d6922 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c617fa640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c617fa640 1 -- 172.21.15.59:0/2760885181 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68018030 con 0x556ffea94830 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c617fa640 1 -- 172.21.15.59:0/2760885181 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8c6800f040 con 0x556ffea94830 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c829b4740 1 -- 172.21.15.59:0/2760885181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ffea071e0 con 0x556ffea94830 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c617fa640 1 -- 172.21.15.59:0/2760885181 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c68022050 con 0x556ffea94830 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c829b4740 10 client.16689 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c829b4740 1 -- 172.21.15.59:0/2760885181 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556ffeaad8c0 con 0x556ffea94830 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c617fa640 1 -- 172.21.15.59:0/2760885181 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556ffea071e0 con 0x556ffea94830 2024-05-08T05:39:47.172 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c617fa640 1 -- 172.21.15.59:0/2760885181 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f8c6801d8f0 con 0x556ffea94830 2024-05-08T05:39:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c829b4740 10 client.16689 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c829b4740 10 client.16689 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:39:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c829b4740 20 client.16689 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556ffeaaed80 0x556ffeab11f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c829b4740 4 client.16689 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:39:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.166+0000 7f8c829b4740 1 -- 172.21.15.59:0/2760885181 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ffea01050 con 0x556ffeaaed80 2024-05-08T05:39:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.167+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556ffeaaed80 0x556ffeab11f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.167+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556ffeaaed80 0x556ffeab11f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62da5160cac19509 server_cookie=d3d5a9ec91ba8da4 in_seq=0 out_seq=0 2024-05-08T05:39:47.173 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.167+0000 7f8c617fa640 10 client.16689 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c617fa640 1 -- 172.21.15.59:0/2760885181 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x556ffea01050 con 0x556ffeaaed80 2024-05-08T05:39:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c617fa640 10 client.16689 handle_command_reply: tid=0 2024-05-08T05:39:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 client.16689 shutdown 2024-05-08T05:39:47.174 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 -- 172.21.15.59:0/2760885181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 msgr2=0x556ffea9c540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9c540 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800e300 tx=0x7f8c6800e7c0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 -- 172.21.15.59:0/2760885181 shutdown_connections 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556ffeaaed80 0x556ffeab11f0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 0x556ffea9ca80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffea9c540 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c617fa640 0 client.16689 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2760885181 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea93f10 0x556ffea9c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 -- 172.21.15.59:0/2760885181 >> 172.21.15.59:0/2760885181 conn(0x556ffea90420 msgr2=0x556ffea961a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 -- 172.21.15.59:0/2760885181 shutdown_connections 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 1 -- 172.21.15.59:0/2760885181 wait complete. 2024-05-08T05:39:47.175 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.169+0000 7f8c829b4740 20 client.16689 trim_cache size 0 max 16384 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.184+0000 7f8c829b4740 1 Processor -- start 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.184+0000 7f8c829b4740 1 -- start start 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.184+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffeab6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.184+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffeab7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.184+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea9f0f0 0x556ffea8ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.184+0000 7f8c829b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781b1fe0 con 0x556ffea94830 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.184+0000 7f8c829b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781b3b50 con 0x556ffea93f10 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.184+0000 7f8c829b4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781b26b0 con 0x556ffea9f0f0 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7d22e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffeab7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.189 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7d22e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffeab7040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55788/0 (socket says 172.21.15.59:55788) 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7d22e640 1 -- 172.21.15.59:0/2575618226 learned_addr learned my addr 172.21.15.59:0/2575618226 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/2575618226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffeab6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7d22e640 1 -- 172.21.15.59:0/2575618226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea9f0f0 msgr2=0x556ffea8ecf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/2575618226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea9f0f0 0x556ffea8ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7d22e640 1 -- 172.21.15.59:0/2575618226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 msgr2=0x556ffeab6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/2575618226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffeab6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7d22e640 1 -- 172.21.15.59:0/2575618226 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ffea8a960 con 0x556ffea94830 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/2575618226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffeab7040 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c9c0 tx=0x7f8c6800ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b1f956fe09bdca0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c637fe640 1 -- 172.21.15.59:0/2575618226 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68018040 con 0x556ffea94830 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c637fe640 1 -- 172.21.15.59:0/2575618226 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c68014400 con 0x556ffea94830 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.185+0000 7f8c637fe640 1 -- 172.21.15.59:0/2575618226 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c68013b80 con 0x556ffea94830 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 -- 172.21.15.59:0/2575618226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 msgr2=0x556ffeab7040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2575618226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffeab7040 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800c9c0 tx=0x7f8c6800ecc0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 -- 172.21.15.59:0/2575618226 shutdown_connections 2024-05-08T05:39:47.190 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2575618226 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea9f0f0 0x556ffea8ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2575618226 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea94830 0x556ffeab7040 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 --2- 172.21.15.59:0/2575618226 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea93f10 0x556ffeab6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 -- 172.21.15.59:0/2575618226 >> 172.21.15.59:0/2575618226 conn(0x556ffea90420 msgr2=0x556ffea8b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 -- 172.21.15.59:0/2575618226 shutdown_connections 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 -- 172.21.15.59:0/2575618226 wait complete. 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 -- start start 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.186+0000 7f8c829b4740 1 Processor -- start 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea93f10 0x556ffea98e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea94830 0x556ffea99360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c829b4740 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 0x556ffea998a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c829b4740 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781b3480 con 0x556ffea93f10 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c829b4740 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781b2db0 con 0x556ffea9f0f0 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c829b4740 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f8c781affe0 con 0x556ffea94830 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c829b4740 10 client.0 _ll_register_callbacks cb 0x7f8c6800c190 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7da2f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea93f10 0x556ffea98e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.191 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7da2f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea93f10 0x556ffea98e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:55794/0 (socket says 172.21.15.59:55794) 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7da2f640 1 -- 172.21.15.59:0/3142385613 learned_addr learned my addr 172.21.15.59:0/3142385613 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7e230640 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 0x556ffea998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea94830 0x556ffea99360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7da2f640 1 -- 172.21.15.59:0/3142385613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea94830 msgr2=0x556ffea99360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea94830 0x556ffea99360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7da2f640 1 -- 172.21.15.59:0/3142385613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 msgr2=0x556ffea998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 0x556ffea998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7da2f640 1 -- 172.21.15.59:0/3142385613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ffeb186b0 con 0x556ffea93f10 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c7da2f640 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea93f10 0x556ffea98e20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8c74003f80 tx=0x7f8c7402a420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a33f78aeab374c6 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c617fa640 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c617fa640 1 -- 172.21.15.59:0/3142385613 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c74049070 con 0x556ffea93f10 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c617fa640 1 -- 172.21.15.59:0/3142385613 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8c740ce120 con 0x556ffea93f10 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.187+0000 7f8c829b4740 1 -- 172.21.15.59:0/3142385613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8c68014400 con 0x556ffea93f10 2024-05-08T05:39:47.192 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c617fa640 1 -- 172.21.15.59:0/3142385613 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) ==== 24+0+0 (secure 0 0 0) 0x7f8c74106dd0 con 0x556ffea93f10 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c829b4740 10 client.16695 fetch_fsmap learned FSMap version 39 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c829b4740 1 -- 172.21.15.59:0/3142385613 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=4+}) -- 0x556ffeb19970 con 0x556ffea93f10 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c617fa640 1 -- 172.21.15.59:0/3142385613 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c68014400 con 0x556ffea93f10 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c617fa640 1 -- 172.21.15.59:0/3142385613 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 39) ==== 8366+0+0 (secure 0 0 0) 0x7f8c741007d0 con 0x556ffea93f10 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c829b4740 10 client.16695 fetch_fsmap finished waiting for FSMap version 39 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c829b4740 10 client.16695 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c829b4740 20 client.16695 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c829b4740 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556ffeb19e70 0x556ffeb1c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c829b4740 4 client.16695 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c829b4740 1 -- 172.21.15.59:0/3142385613 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556ffe97c5f0 con 0x556ffeb19e70 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556ffeb19e70 0x556ffeb1c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c7d22e640 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556ffeb19e70 0x556ffeb1c290 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=74df1c0173d49d0f server_cookie=49fe08b061e22704 in_seq=0 out_seq=0 2024-05-08T05:39:47.193 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:47.188+0000 7f8c617fa640 10 client.16695 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:47.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:47 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:39:47.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:47 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:39:47.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:47 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c617fa640 1 -- 172.21.15.59:0/3142385613 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556ffe97c5f0 con 0x556ffeb19e70 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c617fa640 10 client.16695 handle_command_reply: tid=0 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 client.16695 shutdown 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 -- 172.21.15.59:0/3142385613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea93f10 msgr2=0x556ffea98e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea93f10 0x556ffea98e20 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f8c74003f80 tx=0x7f8c7402a420 comp rx=0 tx=0).stop 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 -- 172.21.15.59:0/3142385613 shutdown_connections 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x556ffeb19e70 0x556ffeb1c290 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x556ffea9f0f0 0x556ffea998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x556ffea94830 0x556ffea99360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 --2- 172.21.15.59:0/3142385613 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x556ffea93f10 0x556ffea98e20 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:48.273 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 -- 172.21.15.59:0/3142385613 >> 172.21.15.59:0/3142385613 conn(0x556ffea90420 msgr2=0x556ffea9bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c617fa640 0 client.16695 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 -- 172.21.15.59:0/3142385613 shutdown_connections 2024-05-08T05:39:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.272+0000 7f8c829b4740 1 -- 172.21.15.59:0/3142385613 wait complete. 2024-05-08T05:39:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.273+0000 7f8c829b4740 20 client.16695 trim_cache size 0 max 16384 2024-05-08T05:39:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:{ 2024-05-08T05:39:48.274 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "message": "", 2024-05-08T05:39:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout: "return_code": 0 2024-05-08T05:39:48.275 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stdout:} 2024-05-08T05:39:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8c640b2a10 msgr2=0x7f8c640b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8c640b2a10 0x7f8c640b4ed0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009fd0 tx=0x7f8c70004a40 comp rx=0 tx=0).stop 2024-05-08T05:39:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781029c0 msgr2=0x7f8c78074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781029c0 0x7f8c78074cb0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f8c74005940 tx=0x7f8c740042f0 comp rx=0 tx=0).stop 2024-05-08T05:39:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3913552228 shutdown_connections 2024-05-08T05:39:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f8c640b2a10 0x7f8c640b4ed0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:48.276 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f8c781089d0 0x7f8c78075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f8c78103380 0x7f8c780751f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 --2- 172.21.15.59:0/3913552228 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f8c781029c0 0x7f8c78074cb0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3913552228 >> 172.21.15.59:0/3913552228 conn(0x7f8c780fe4b0 msgr2=0x7f8c780feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3913552228 shutdown_connections 2024-05-08T05:39:48.277 INFO:teuthology.task.background_exec.ubuntu@smithi059.front.sepia.ceph.com.smithi059.stderr:2024-05-08T05:39:48.274+0000 7f8c7fc93640 1 -- 172.21.15.59:0/3913552228 wait complete. 2024-05-08T05:39:49.231 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:48 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.232 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:48 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.232 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:48 smithi059 ceph-mon[30304]: pgmap v319: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-05-08T05:39:49.232 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:48 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.232 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:48 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.232 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:48 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.232 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:48 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:48 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:48 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:48 smithi143 ceph-mon[34337]: pgmap v319: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-05-08T05:39:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:48 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:48 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:48 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:48 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:48 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:48 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:48 smithi096 ceph-mon[34253]: pgmap v319: 129 pgs: 129 active+clean; 124 MiB data, 2.6 GiB used, 1.0 TiB / 1.0 TiB avail; 9.4 KiB/s rd, 125 KiB/s wr, 8 op/s 2024-05-08T05:39:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:48 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:48 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:48 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:49.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:48 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: pgmap v320: 129 pgs: 129 active+clean; 125 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 979 KiB/s wr, 356 op/s 2024-05-08T05:39:51.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:39:51.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:39:51.495 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:51 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: pgmap v320: 129 pgs: 129 active+clean; 125 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 979 KiB/s wr, 356 op/s 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:39:51.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:39:51.531 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:51 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: pgmap v320: 129 pgs: 129 active+clean; 125 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 716 KiB/s rd, 979 KiB/s wr, 356 op/s 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-05-08T05:39:51.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:39:51.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:39:51.588 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:51 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:39:52.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:52 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi143 to 176.2M 2024-05-08T05:39:52.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:52 smithi143 ceph-mon[34337]: Unable to set osd_memory_target on smithi143 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:39:52.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:52 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-08T05:39:52.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:52 smithi143 ceph-mon[34337]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:39:52.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:52 smithi143 ceph-mon[34337]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-08T05:39:52.494 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:52 smithi143 ceph-mon[34337]: Unable to set osd_memory_target on smithi059 to 185879859: error parsing value: Value '185879859' is below minimum 939524096 2024-05-08T05:39:52.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:52 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi143 to 176.2M 2024-05-08T05:39:52.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:52 smithi096 ceph-mon[34253]: Unable to set osd_memory_target on smithi143 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:39:52.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:52 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-08T05:39:52.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:52 smithi096 ceph-mon[34253]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:39:52.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:52 smithi096 ceph-mon[34253]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-08T05:39:52.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:52 smithi096 ceph-mon[34253]: Unable to set osd_memory_target on smithi059 to 185879859: error parsing value: Value '185879859' is below minimum 939524096 2024-05-08T05:39:52.578 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:52 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi143 to 176.2M 2024-05-08T05:39:52.579 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:52 smithi059 ceph-mon[30304]: Unable to set osd_memory_target on smithi143 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:39:52.579 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:52 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi096 to 176.2M 2024-05-08T05:39:52.579 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:52 smithi059 ceph-mon[30304]: Unable to set osd_memory_target on smithi096 to 184785305: error parsing value: Value '184785305' is below minimum 939524096 2024-05-08T05:39:52.579 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:52 smithi059 ceph-mon[30304]: Adjusting osd_memory_target on smithi059 to 177.2M 2024-05-08T05:39:52.579 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:52 smithi059 ceph-mon[30304]: Unable to set osd_memory_target on smithi059 to 185879859: error parsing value: Value '185879859' is below minimum 939524096 2024-05-08T05:39:53.253 INFO:tasks.workunit:Stopping ['suites/fsx.sh'] on client.0... 2024-05-08T05:39:53.253 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-05-08T05:39:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:53 smithi143 ceph-mon[34337]: pgmap v321: 129 pgs: 129 active+clean; 125 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 979 KiB/s wr, 352 op/s 2024-05-08T05:39:53.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:53 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:39:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:53 smithi096 ceph-mon[34253]: pgmap v321: 129 pgs: 129 active+clean; 125 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 979 KiB/s wr, 352 op/s 2024-05-08T05:39:53.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:53 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:39:53.536 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:53 smithi059 ceph-mon[30304]: pgmap v321: 129 pgs: 129 active+clean; 125 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 979 KiB/s wr, 352 op/s 2024-05-08T05:39:53.536 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:53 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:39:53.538 DEBUG:teuthology.parallel:result is None 2024-05-08T05:39:53.539 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T05:39:53.575 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-05-08T05:39:53.575 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-05-08T05:39:53.591 INFO:tasks.quiescer:joining Quiescers 2024-05-08T05:39:53.591 WARNING:tasks.quiescer:The quiescer is requested to stop, running cancel all 2024-05-08T05:39:53.591 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:39:53.867 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:39:55.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:55 smithi143 ceph-mon[34337]: pgmap v322: 129 pgs: 129 active+clean; 125 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 979 KiB/s wr, 352 op/s 2024-05-08T05:39:55.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:55 smithi096 ceph-mon[34253]: pgmap v322: 129 pgs: 129 active+clean; 125 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 979 KiB/s wr, 352 op/s 2024-05-08T05:39:55.540 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:55 smithi059 ceph-mon[30304]: pgmap v322: 129 pgs: 129 active+clean; 125 MiB data, 2.7 GiB used, 1.0 TiB / 1.0 TiB avail; 709 KiB/s rd, 979 KiB/s wr, 352 op/s 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.982+0000 7f39f8283700 1 -- 172.21.15.59:0/1264858988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f0109510 msgr2=0x7f39f01098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.982+0000 7f39f8283700 1 --2- 172.21.15.59:0/1264858988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f0109510 0x7f39f01098f0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f39ec009990 tx=0x7f39ec009ca0 comp rx=0 tx=0).stop 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.982+0000 7f39f8283700 1 -- 172.21.15.59:0/1264858988 shutdown_connections 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.982+0000 7f39f8283700 1 --2- 172.21.15.59:0/1264858988 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39f006bb40 0x7f39f0110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.982+0000 7f39f8283700 1 --2- 172.21.15.59:0/1264858988 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f0109510 0x7f39f01098f0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.982+0000 7f39f8283700 1 --2- 172.21.15.59:0/1264858988 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f006b180 0x7f39f006b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.982+0000 7f39f8283700 1 -- 172.21.15.59:0/1264858988 >> 172.21.15.59:0/1264858988 conn(0x7f39f0079350 msgr2=0x7f39f0079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.982+0000 7f39f8283700 1 -- 172.21.15.59:0/1264858988 shutdown_connections 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.982+0000 7f39f8283700 1 -- 172.21.15.59:0/1264858988 wait complete. 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f8283700 1 Processor -- start 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f8283700 1 -- start start 2024-05-08T05:39:55.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f8283700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f006b180 0x7f39f0102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f8283700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39f006bb40 0x7f39f01029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f8283700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f0109510 0x7f39f0102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f8283700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f39f0113670 con 0x7f39f006b180 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f8283700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f39f01134f0 con 0x7f39f006bb40 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f8283700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f39f01137f0 con 0x7f39f0109510 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f601f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f006b180 0x7f39f0102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f601f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f006b180 0x7f39f0102470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53794/0 (socket says 172.21.15.59:53794) 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.983+0000 7f39f601f700 1 -- 172.21.15.59:0/1867852097 learned_addr learned my addr 172.21.15.59:0/1867852097 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f581e700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39f006bb40 0x7f39f01029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f6820700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f0109510 0x7f39f0102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f601f700 1 -- 172.21.15.59:0/1867852097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f0109510 msgr2=0x7f39f0102ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f601f700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f0109510 0x7f39f0102ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:55.986 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f601f700 1 -- 172.21.15.59:0/1867852097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39f006bb40 msgr2=0x7f39f01029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f601f700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39f006bb40 0x7f39f01029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f601f700 1 -- 172.21.15.59:0/1867852097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39f01a80d0 con 0x7f39f006b180 2024-05-08T05:39:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f601f700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f006b180 0x7f39f0102470 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f39ec005e80 tx=0x7f39ec005eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39e77fe700 1 -- 172.21.15.59:0/1867852097 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec027070 con 0x7f39f006b180 2024-05-08T05:39:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39e77fe700 1 -- 172.21.15.59:0/1867852097 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39ec003fb0 con 0x7f39f006b180 2024-05-08T05:39:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39f01a8360 con 0x7f39f006b180 2024-05-08T05:39:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39e77fe700 1 -- 172.21.15.59:0/1867852097 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39ec022c60 con 0x7f39f006b180 2024-05-08T05:39:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.984+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39f01a87b0 con 0x7f39f006b180 2024-05-08T05:39:55.987 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.985+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39f010b9f0 con 0x7f39f006b180 2024-05-08T05:39:55.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.988+0000 7f39e77fe700 1 -- 172.21.15.59:0/1867852097 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f39ec02a070 con 0x7f39f006b180 2024-05-08T05:39:55.988 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.988+0000 7f39e77fe700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f39dc0b29f0 0x7f39dc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:55.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.988+0000 7f39f581e700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f39dc0b29f0 0x7f39dc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:55.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.988+0000 7f39e77fe700 1 -- 172.21.15.59:0/1867852097 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f39ec022e00 con 0x7f39f006b180 2024-05-08T05:39:55.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.989+0000 7f39f581e700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f39dc0b29f0 0x7f39dc0b4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39f0079a30 tx=0x7f39e000b410 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:55.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:55.992+0000 7f39e77fe700 1 -- 172.21.15.59:0/1867852097 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f39ec031070 con 0x7f39f006b180 2024-05-08T05:39:56.253 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.253+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f39f010b3d0 con 0x7f39f006b180 2024-05-08T05:39:56.254 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.254+0000 7f39e77fe700 1 -- 172.21.15.59:0/1867852097 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11932 (secure 0 0 0) 0x7f39ec013070 con 0x7f39f006b180 2024-05-08T05:39:56.254 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:39:56.254 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:39:55.325460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25876":{"gid":25876,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2351774265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2351774265},{"type":"v1","addr":"172.21.15.143:6835","nonce":2351774265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:39:56.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f39dc0b29f0 msgr2=0x7f39dc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:56.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f39dc0b29f0 0x7f39dc0b4eb0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f39f0079a30 tx=0x7f39e000b410 comp rx=0 tx=0).stop 2024-05-08T05:39:56.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f006b180 msgr2=0x7f39f0102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:56.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f006b180 0x7f39f0102470 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7f39ec005e80 tx=0x7f39ec005eb0 comp rx=0 tx=0).stop 2024-05-08T05:39:56.257 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 shutdown_connections 2024-05-08T05:39:56.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f39dc0b29f0 0x7f39dc0b4eb0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:56.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f39f0109510 0x7f39f0102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:56.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f39f006b180 0x7f39f0102470 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:56.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 --2- 172.21.15.59:0/1867852097 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f39f006bb40 0x7f39f01029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:56.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 >> 172.21.15.59:0/1867852097 conn(0x7f39f0079350 msgr2=0x7f39f01011a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:56.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 shutdown_connections 2024-05-08T05:39:56.258 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:56.257+0000 7f39f8283700 1 -- 172.21.15.59:0/1867852097 wait complete. 2024-05-08T05:39:56.258 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 41 2024-05-08T05:39:57.159 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24473 quiesce db --cancel --all' 2024-05-08T05:39:57.159 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- timeout 75 ceph tell mds.24473 quiesce db --cancel --all 2024-05-08T05:39:57.420 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:39:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:57 smithi143 ceph-mon[34337]: pgmap v323: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 581 op/s 2024-05-08T05:39:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:57 smithi143 ceph-mon[34337]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:39:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:57 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1867852097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:39:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:57 smithi096 ceph-mon[34253]: pgmap v323: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 581 op/s 2024-05-08T05:39:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:57 smithi096 ceph-mon[34253]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:39:57.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:57 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1867852097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:39:57.550 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:57 smithi059 ceph-mon[30304]: pgmap v323: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.4 MiB/s wr, 581 op/s 2024-05-08T05:39:57.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:57 smithi059 ceph-mon[30304]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-05-08T05:39:57.551 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:57 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1867852097' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:39:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 -- 172.21.15.59:0/255985704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 msgr2=0x7f23e4103b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 --2- 172.21.15.59:0/255985704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 0x7f23e4103b30 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7f23d40099c0 tx=0x7f23d4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 -- 172.21.15.59:0/255985704 shutdown_connections 2024-05-08T05:39:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 --2- 172.21.15.59:0/255985704 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e4108850 0x7f23e41103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.282 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 --2- 172.21.15.59:0/255985704 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 0x7f23e4103b30 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 --2- 172.21.15.59:0/255985704 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23e4104100 0x7f23e4108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 -- 172.21.15.59:0/255985704 >> 172.21.15.59:0/255985704 conn(0x7f23e407a600 msgr2=0x7f23e407aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 -- 172.21.15.59:0/255985704 shutdown_connections 2024-05-08T05:39:59.283 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 -- 172.21.15.59:0/255985704 wait complete. 2024-05-08T05:39:59.283 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:39:59 smithi059 ceph-mon[30304]: pgmap v324: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 577 op/s 2024-05-08T05:39:59.284 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.281+0000 7f23ea272700 1 Processor -- start 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23ea272700 1 -- start start 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23ea272700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 0x7f23e419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23ea272700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e4104100 0x7f23e419fd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23ea272700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23e4108850 0x7f23e41999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23ea272700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4113690 con 0x7f23e4103750 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23ea272700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4113510 con 0x7f23e4104100 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23ea272700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f23e4113810 con 0x7f23e4108850 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 0x7f23e419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 0x7f23e419f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43970/0 (socket says 172.21.15.59:43970) 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23e3fff700 1 -- 172.21.15.59:0/20222645 learned_addr learned my addr 172.21.15.59:0/20222645 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:59.285 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23e37fe700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e4104100 0x7f23e419fd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23e880f700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23e4108850 0x7f23e41999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23e3fff700 1 -- 172.21.15.59:0/20222645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23e4108850 msgr2=0x7f23e41999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23e3fff700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23e4108850 0x7f23e41999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.282+0000 7f23e3fff700 1 -- 172.21.15.59:0/20222645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e4104100 msgr2=0x7f23e419fd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.283+0000 7f23e3fff700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e4104100 0x7f23e419fd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.283+0000 7f23e3fff700 1 -- 172.21.15.59:0/20222645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23e419a130 con 0x7f23e4103750 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.283+0000 7f23e3fff700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 0x7f23e419f810 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f23d40098e0 tx=0x7f23d4003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.283+0000 7f23e17fa700 1 -- 172.21.15.59:0/20222645 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d4027070 con 0x7f23e4103750 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.283+0000 7f23ea272700 1 -- 172.21.15.59:0/20222645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23e419a3c0 con 0x7f23e4103750 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.283+0000 7f23e17fa700 1 -- 172.21.15.59:0/20222645 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d40040a0 con 0x7f23e4103750 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.283+0000 7f23e17fa700 1 -- 172.21.15.59:0/20222645 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d400f460 con 0x7f23e4103750 2024-05-08T05:39:59.286 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.283+0000 7f23ea272700 1 -- 172.21.15.59:0/20222645 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23e41a66e0 con 0x7f23e4103750 2024-05-08T05:39:59.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.285+0000 7f23e17fa700 1 -- 172.21.15.59:0/20222645 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f23d402a070 con 0x7f23e4103750 2024-05-08T05:39:59.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.286+0000 7f23e17fa700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f23cc0b29f0 0x7f23cc0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.289 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.286+0000 7f23e37fe700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f23cc0b29f0 0x7f23cc0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.286+0000 7f23e17fa700 1 -- 172.21.15.59:0/20222645 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f23d40e3c90 con 0x7f23e4103750 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23f4b05b80 1 Processor -- start 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23f4b05b80 1 -- start start 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e37fe700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f23cc0b29f0 0x7f23cc0b4eb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f23d8006fd0 tx=0x7f23d8008040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecface2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 0x55aecfac9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfaca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55aecfa1b8d0 con 0x55aecfacdf10 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aecf9ba830 con 0x55aecface830 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55aecfa17890 con 0x55aecfac9d30 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecface2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.290 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecface2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43982/0 (socket says 172.21.15.59:43982) 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e3fff700 1 -- 172.21.15.59:0/1054175946 learned_addr learned my addr 172.21.15.59:0/1054175946 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e37fe700 1 --2- 172.21.15.59:0/1054175946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 0x55aecfac9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e3fff700 1 -- 172.21.15.59:0/1054175946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 msgr2=0x55aecfaca190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e3fff700 1 --2- 172.21.15.59:0/1054175946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfaca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e3fff700 1 -- 172.21.15.59:0/1054175946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 msgr2=0x55aecfac9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e3fff700 1 --2- 172.21.15.59:0/1054175946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 0x55aecfac9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.287+0000 7f23e3fff700 1 -- 172.21.15.59:0/1054175946 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aecf968940 con 0x55aecfacdf10 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.288+0000 7f23e3fff700 1 --2- 172.21.15.59:0/1054175946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecface2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f23d400b580 tx=0x7f23d40a6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a78b263ba2cc38bf server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.288+0000 7f23ca7fc700 1 -- 172.21.15.59:0/1054175946 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d40b1040 con 0x55aecfacdf10 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.288+0000 7f23ca7fc700 1 -- 172.21.15.59:0/1054175946 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d40cbe50 con 0x55aecfacdf10 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.288+0000 7f23ca7fc700 1 -- 172.21.15.59:0/1054175946 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d40b3570 con 0x55aecfacdf10 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.288+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1054175946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 msgr2=0x55aecface2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.288+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1054175946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecface2f0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7f23d400b580 tx=0x7f23d40a6c00 comp rx=0 tx=0).stop 2024-05-08T05:39:59.291 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1054175946 shutdown_connections 2024-05-08T05:39:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1054175946 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfaca190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1054175946 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecface2f0 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1054175946 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 0x55aecfac9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1054175946 >> 172.21.15.59:0/1054175946 conn(0x55aecf9f5170 msgr2=0x55aecfac3f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1054175946 shutdown_connections 2024-05-08T05:39:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1054175946 wait complete. 2024-05-08T05:39:59.292 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 -- start start 2024-05-08T05:39:59.295 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 Processor -- start 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfac9d30 0x55aecfad5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 0x55aecfad2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.289+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecface830 0x55aecfad2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x55aecfa1b8d0 con 0x55aecfac9d30 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x55aecf981ca0 con 0x55aecfacdf10 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x55aecfad61c0 con 0x55aecface830 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfac9d30 0x55aecfad5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e3fff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfac9d30 0x55aecfad5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43984/0 (socket says 172.21.15.59:43984) 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e3fff700 1 -- 172.21.15.59:0/1704998976 learned_addr learned my addr 172.21.15.59:0/1704998976 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23f4b05b80 10 client.0 _ll_register_callbacks cb 0x55aecf9b3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e37fe700 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 0x55aecfad2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e880f700 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecface830 0x55aecfad2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e3fff700 1 -- 172.21.15.59:0/1704998976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecface830 msgr2=0x55aecfad2f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:59.296 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e3fff700 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecface830 0x55aecfad2f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e3fff700 1 -- 172.21.15.59:0/1704998976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 msgr2=0x55aecfad2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e3fff700 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 0x55aecfad2a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e3fff700 1 -- 172.21.15.59:0/1704998976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aecf9b7700 con 0x55aecfac9d30 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23e3fff700 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfac9d30 0x55aecfad5620 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f23d40aabc0 tx=0x7f23d4009f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f745ddba3ac1e1b server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23c37fe700 1 -- 172.21.15.59:0/1704998976 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d406e070 con 0x55aecfac9d30 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23c37fe700 1 -- 172.21.15.59:0/1704998976 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f23d40a7220 con 0x55aecfac9d30 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1704998976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23d40cbe50 con 0x55aecfac9d30 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23c37fe700 1 -- 172.21.15.59:0/1704998976 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f23d40b1040 con 0x55aecfac9d30 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23f4b05b80 10 client.16713 fetch_fsmap learned FSMap version 41 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.290+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1704998976 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55aecfae07c0 con 0x55aecfac9d30 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.291+0000 7f23c37fe700 1 -- 172.21.15.59:0/1704998976 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23d40cbe50 con 0x55aecfac9d30 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.291+0000 7f23c37fe700 1 -- 172.21.15.59:0/1704998976 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f23d40b11e0 con 0x55aecfac9d30 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.291+0000 7f23f4b05b80 10 client.16713 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.291+0000 7f23f4b05b80 10 client.16713 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:39:59.297 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.291+0000 7f23f4b05b80 20 client.16713 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.291+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aecfae2c90 0x55aecfae5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.291+0000 7f23f4b05b80 4 client.16713 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.291+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1704998976 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55aecfa956c0 con 0x55aecfae2c90 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.291+0000 7f23e37fe700 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aecfae2c90 0x55aecfae5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.292+0000 7f23e37fe700 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aecfae2c90 0x55aecfae5080 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e584ac433c8ed8d server_cookie=c5af80e973b61b43 in_seq=0 out_seq=0 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.292+0000 7f23c37fe700 10 client.16713 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.292+0000 7f23c37fe700 1 -- 172.21.15.59:0/1704998976 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23648 (crc 0 0 0) 0x55aecfa956c0 con 0x55aecfae2c90 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.292+0000 7f23c37fe700 10 client.16713 handle_command_reply: tid=0 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.292+0000 7f23f4b05b80 1 client.16713 shutdown 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.292+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1704998976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfac9d30 msgr2=0x55aecfad5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.292+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfac9d30 0x55aecfad5620 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f23d40aabc0 tx=0x7f23d4009f10 comp rx=0 tx=0).stop 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1704998976 shutdown_connections 2024-05-08T05:39:59.298 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aecfae2c90 0x55aecfae5080 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecface830 0x55aecfad2f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23c37fe700 0 client.16713 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfac9d30 0x55aecfad5620 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/1704998976 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 0x55aecfad2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1704998976 >> 172.21.15.59:0/1704998976 conn(0x55aecf9f5170 msgr2=0x55aecfac2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1704998976 shutdown_connections 2024-05-08T05:39:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23f4b05b80 1 -- 172.21.15.59:0/1704998976 wait complete. 2024-05-08T05:39:59.299 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.293+0000 7f23f4b05b80 20 client.16713 trim_cache size 0 max 16384 2024-05-08T05:39:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23f4b05b80 1 Processor -- start 2024-05-08T05:39:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23f4b05b80 1 -- start start 2024-05-08T05:39:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfadefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 0x55aecfad52d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecface830 0x55aecfacd260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23d000ed60 con 0x55aecface830 2024-05-08T05:39:59.315 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23d00108d0 con 0x55aecfacdf10 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f23d000f430 con 0x55aecfac9d30 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e880f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecface830 0x55aecfacd260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e880f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecface830 0x55aecfacd260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43986/0 (socket says 172.21.15.59:43986) 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e880f700 1 -- 172.21.15.59:0/2295766952 learned_addr learned my addr 172.21.15.59:0/2295766952 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e37fe700 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 0x55aecfad52d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e3fff700 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfadefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e880f700 1 -- 172.21.15.59:0/2295766952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 msgr2=0x55aecfadefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e880f700 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfadefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e880f700 1 -- 172.21.15.59:0/2295766952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 msgr2=0x55aecfad52d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e880f700 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 0x55aecfad52d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e880f700 1 -- 172.21.15.59:0/2295766952 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aecfad1a00 con 0x55aecface830 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.313+0000 7f23e3fff700 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfadefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23e880f700 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecface830 0x55aecfacd260 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f23dc008470 tx=0x7f23dc008780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac0f5c06988aab0f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23c9ffb700 1 -- 172.21.15.59:0/2295766952 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc015610 con 0x55aecface830 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23c9ffb700 1 -- 172.21.15.59:0/2295766952 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23dc01a070 con 0x55aecface830 2024-05-08T05:39:59.316 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23c9ffb700 1 -- 172.21.15.59:0/2295766952 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23dc014ab0 con 0x55aecface830 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2295766952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecface830 msgr2=0x55aecfacd260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecface830 0x55aecfacd260 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f23dc008470 tx=0x7f23dc008780 comp rx=0 tx=0).stop 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2295766952 shutdown_connections 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecface830 0x55aecfacd260 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfadefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2295766952 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecfacdf10 0x55aecfad52d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.314+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2295766952 >> 172.21.15.59:0/2295766952 conn(0x55aecf9f5170 msgr2=0x55aecfaccb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.315+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2295766952 shutdown_connections 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.315+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2295766952 wait complete. 2024-05-08T05:39:59.317 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.315+0000 7f23f4b05b80 1 -- start start 2024-05-08T05:39:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.315+0000 7f23f4b05b80 1 Processor -- start 2024-05-08T05:39:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.315+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfadb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.315+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecfadba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.315+0000 7f23f4b05b80 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 0x55aecfad7a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f23d0010200 con 0x55aecfacdf10 2024-05-08T05:39:59.320 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f23d000fb30 con 0x55aecface830 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23f4b05b80 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f23d000cd20 con 0x55aecfac9d30 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23f4b05b80 10 client.0 _ll_register_callbacks cb 0x55aecf9b3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecfadba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e37fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecfadba20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:43988/0 (socket says 172.21.15.59:43988) 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e37fe700 1 -- 172.21.15.59:0/2211533301 learned_addr learned my addr 172.21.15.59:0/2211533301 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e880f700 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 0x55aecfad7a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e3fff700 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfadb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e37fe700 1 -- 172.21.15.59:0/2211533301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 msgr2=0x55aecfadb4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e37fe700 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfadb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e37fe700 1 -- 172.21.15.59:0/2211533301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 msgr2=0x55aecfad7a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e37fe700 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 0x55aecfad7a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e37fe700 1 -- 172.21.15.59:0/2211533301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aecfad82b0 con 0x55aecfacdf10 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23e37fe700 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecfadba20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f23d80092d0 tx=0x7f23d8004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a8190c136ab40f server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23c37fe700 10 client.0 ms_handle_connect on v2:172.21.15.59:3300/0 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23c37fe700 1 -- 172.21.15.59:0/2211533301 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23d8034060 con 0x55aecfacdf10 2024-05-08T05:39:59.321 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2211533301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23dc01a070 con 0x55aecfacdf10 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.316+0000 7f23c37fe700 1 -- 172.21.15.59:0/2211533301 <== mon.0 v2:172.21.15.59:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f23d800f040 con 0x55aecfacdf10 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23c37fe700 1 -- 172.21.15.59:0/2211533301 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_get_version_reply(handle=1 version=41) ==== 24+0+0 (secure 0 0 0) 0x7f23d803b050 con 0x55aecfacdf10 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23f4b05b80 10 client.16719 fetch_fsmap learned FSMap version 41 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2211533301 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,fsmap=41,monmap=4+}) -- 0x55aecfb525e0 con 0x55aecfacdf10 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23c37fe700 1 -- 172.21.15.59:0/2211533301 <== mon.0 v2:172.21.15.59:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23dc01a070 con 0x55aecfacdf10 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23c37fe700 1 -- 172.21.15.59:0/2211533301 <== mon.0 v2:172.21.15.59:3300/0 5 ==== fsmap(e 41) ==== 8374+0+0 (secure 0 0 0) 0x7f23d8013070 con 0x55aecfacdf10 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23f4b05b80 10 client.16719 fetch_fsmap finished waiting for FSMap version 41 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23f4b05b80 10 client.16719 resolve_mds: validated gid 24473 aka daemon mds.b 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23f4b05b80 20 client.16719 populate_metadata read hostname 'smithi059' 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aecfb52d00 0x55aecfb550f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23f4b05b80 4 client.16719 mds_command: new command op to 24473 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2211533301 --> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55aecfa292b0 con 0x55aecfb52d00 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.317+0000 7f23e3fff700 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aecfb52d00 0x55aecfb550f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23c37fe700 10 client.16719 ms_handle_connect on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23e3fff700 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aecfb52d00 0x55aecfb550f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba86f8d3fc5b35cd server_cookie=ec4154b4b93e9e26 in_seq=0 out_seq=0 2024-05-08T05:39:59.322 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23c37fe700 1 -- 172.21.15.59:0/2211533301 <== mds.0 v2:172.21.15.96:6838/126053836 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55aecfa292b0 con 0x55aecfb52d00 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23c37fe700 10 client.16719 handle_command_reply: tid=0 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 client.16719 shutdown 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2211533301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 msgr2=0x55aecfadba20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecfadba20 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f23d80092d0 tx=0x7f23d8004750 comp rx=0 tx=0).stop 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2211533301 shutdown_connections 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.96:6838/126053836,v1:172.21.15.96:6839/126053836] conn(0x55aecfb52d00 0x55aecfb550f0 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x55aecface830 0x55aecfad7a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x55aecfac9d30 0x55aecfadb4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 --2- 172.21.15.59:0/2211533301 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x55aecfacdf10 0x55aecfadba20 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23c37fe700 0 client.16719 ms_handle_reset on v2:172.21.15.96:6838/126053836 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2211533301 >> 172.21.15.59:0/2211533301 conn(0x55aecf9f5170 msgr2=0x55aecfacd3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2211533301 shutdown_connections 2024-05-08T05:39:59.323 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 1 -- 172.21.15.59:0/2211533301 wait complete. 2024-05-08T05:39:59.324 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.318+0000 7f23f4b05b80 20 client.16719 trim_cache size 0 max 16384 2024-05-08T05:39:59.324 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:39:59.324 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 41, 2024-05-08T05:39:59.324 INFO:teuthology.orchestra.run.smithi059.stdout: "leader": 24473, 2024-05-08T05:39:59.324 INFO:teuthology.orchestra.run.smithi059.stdout: "set_version": 32, 2024-05-08T05:39:59.324 INFO:teuthology.orchestra.run.smithi059.stdout: "sets": {} 2024-05-08T05:39:59.324 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:39:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 -- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f23cc0b29f0 msgr2=0x7f23cc0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f23cc0b29f0 0x7f23cc0b4eb0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f23d8006fd0 tx=0x7f23d8008040 comp rx=0 tx=0).stop 2024-05-08T05:39:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 -- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 msgr2=0x7f23e419f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:39:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 0x7f23e419f810 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7f23d40098e0 tx=0x7f23d4003680 comp rx=0 tx=0).stop 2024-05-08T05:39:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 -- 172.21.15.59:0/20222645 shutdown_connections 2024-05-08T05:39:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f23cc0b29f0 0x7f23cc0b4eb0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f23e4108850 0x7f23e41999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f23e4103750 0x7f23e419f810 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.325 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 --2- 172.21.15.59:0/20222645 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f23e4104100 0x7f23e419fd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:39:59.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 -- 172.21.15.59:0/20222645 >> 172.21.15.59:0/20222645 conn(0x7f23e407a600 msgr2=0x7f23e41018a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:39:59.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 -- 172.21.15.59:0/20222645 shutdown_connections 2024-05-08T05:39:59.326 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:39:59.319+0000 7f23ea272700 1 -- 172.21.15.59:0/20222645 wait complete. 2024-05-08T05:39:59.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:39:59 smithi143 ceph-mon[34337]: pgmap v324: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 577 op/s 2024-05-08T05:39:59.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:39:59 smithi096 ceph-mon[34253]: pgmap v324: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 577 op/s 2024-05-08T05:40:00.047 INFO:tasks.quiescer:done joining Quiescers 2024-05-08T05:40:00.048 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-05-08T05:40:00.059 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:40:00.067 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-05-08T05:40:00.101 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-05-08T05:40:00.101 INFO:teuthology.task.exec:Executing custom commands... 2024-05-08T05:40:00.102 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:40:00.102 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-05-08T05:40:00.138 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-05-08T05:40:00.207 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-05-08T05:40:00.269 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:00 smithi059 ceph-mon[30304]: overall HEALTH_OK 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.353+0000 7feb0701d640 1 -- 172.21.15.59:0/364865868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb001018d0 msgr2=0x7feb00101d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.353+0000 7feb0701d640 1 --2- 172.21.15.59:0/364865868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb001018d0 0x7feb00101d50 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7feaf4009930 tx=0x7feaf402a420 comp rx=0 tx=0).stop 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.353+0000 7feb0701d640 1 -- 172.21.15.59:0/364865868 shutdown_connections 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.353+0000 7feb0701d640 1 --2- 172.21.15.59:0/364865868 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb00102290 0x7feb0010e7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.353+0000 7feb0701d640 1 --2- 172.21.15.59:0/364865868 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb001018d0 0x7feb00101d50 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.353+0000 7feb0701d640 1 --2- 172.21.15.59:0/364865868 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb001078e0 0x7feb00107cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.353+0000 7feb0701d640 1 -- 172.21.15.59:0/364865868 >> 172.21.15.59:0/364865868 conn(0x7feb00077c30 msgr2=0x7feb00078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.353+0000 7feb0701d640 1 -- 172.21.15.59:0/364865868 shutdown_connections 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.353+0000 7feb0701d640 1 -- 172.21.15.59:0/364865868 wait complete. 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb0701d640 1 Processor -- start 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb0701d640 1 -- start start 2024-05-08T05:40:00.356 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb0701d640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb001018d0 0x7feb00199210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb0701d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb00102290 0x7feb00199750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb0701d640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb001078e0 0x7feb0019da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb0701d640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7feb00111b60 con 0x7feb00102290 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb0701d640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7feb001119e0 con 0x7feb001018d0 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb0701d640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7feb00111ce0 con 0x7feb001078e0 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feafffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb00102290 0x7feb00199750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feafffff640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb00102290 0x7feb00199750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44002/0 (socket says 172.21.15.59:44002) 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feafffff640 1 -- 172.21.15.59:0/4134424159 learned_addr learned my addr 172.21.15.59:0/4134424159 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb04db9640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb001018d0 0x7feb00199210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feb055ba640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb001078e0 0x7feb0019da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feafffff640 1 -- 172.21.15.59:0/4134424159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb001078e0 msgr2=0x7feb0019da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feafffff640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb001078e0 0x7feb0019da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feafffff640 1 -- 172.21.15.59:0/4134424159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb001018d0 msgr2=0x7feb00199210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.357 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feafffff640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb001018d0 0x7feb00199210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feafffff640 1 -- 172.21.15.59:0/4134424159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb0019e1b0 con 0x7feb00102290 2024-05-08T05:40:00.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.354+0000 7feafffff640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb00102290 0x7feb00199750 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7feaf4004770 tx=0x7feaf40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:00.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.355+0000 7feafdffb640 1 -- 172.21.15.59:0/4134424159 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf403f070 con 0x7feb00102290 2024-05-08T05:40:00.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.355+0000 7feafdffb640 1 -- 172.21.15.59:0/4134424159 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feaf403b460 con 0x7feb00102290 2024-05-08T05:40:00.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.355+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb0019e440 con 0x7feb00102290 2024-05-08T05:40:00.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.355+0000 7feafdffb640 1 -- 172.21.15.59:0/4134424159 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feaf4035070 con 0x7feb00102290 2024-05-08T05:40:00.358 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.355+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb001a5d30 con 0x7feb00102290 2024-05-08T05:40:00.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.356+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb00050740 con 0x7feb00102290 2024-05-08T05:40:00.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.357+0000 7feafdffb640 1 -- 172.21.15.59:0/4134424159 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7feaf403b8d0 con 0x7feb00102290 2024-05-08T05:40:00.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.358+0000 7feafdffb640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7feae80b2a90 0x7feae80b4f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:00.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.358+0000 7feafdffb640 1 -- 172.21.15.59:0/4134424159 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7feaf4102f90 con 0x7feb00102290 2024-05-08T05:40:00.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.358+0000 7feb04db9640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7feae80b2a90 0x7feae80b4f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:00.359 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.359+0000 7feb04db9640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7feae80b2a90 0x7feae80b4f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005d00 tx=0x7feaf000b410 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:00.362 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.362+0000 7feafdffb640 1 -- 172.21.15.59:0/4134424159 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7feaf40ca5a0 con 0x7feb00102290 2024-05-08T05:40:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:00 smithi143 ceph-mon[34337]: overall HEALTH_OK 2024-05-08T05:40:00.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:00 smithi096 ceph-mon[34253]: overall HEALTH_OK 2024-05-08T05:40:00.576 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.575+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7feb0010aea0 con 0x7feae80b2a90 2024-05-08T05:40:00.584 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.584+0000 7feafdffb640 1 -- 172.21.15.59:0/4134424159 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7feb0010aea0 con 0x7feae80b2a90 2024-05-08T05:40:00.584 INFO:teuthology.orchestra.run.smithi059.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-05-08T00:00:00", "created": "2024-05-08T05:29:36", "first": "2024-05-08T05:30:00", "last": "2024-05-08T05:39:00", "last_pruned": "2024-05-08T05:39:00", "created_count": 10, "pruned_count": 4, "active": true} 2024-05-08T05:40:00.586 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7feae80b2a90 msgr2=0x7feae80b4f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7feae80b2a90 0x7feae80b4f50 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7feaf0005d00 tx=0x7feaf000b410 comp rx=0 tx=0).stop 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb00102290 msgr2=0x7feb00199750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb00102290 0x7feb00199750 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7feaf4004770 tx=0x7feaf40042f0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 shutdown_connections 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7feae80b2a90 0x7feae80b4f50 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7feb001078e0 0x7feb0019da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7feb00102290 0x7feb00199750 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 --2- 172.21.15.59:0/4134424159 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7feb001018d0 0x7feb00199210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 >> 172.21.15.59:0/4134424159 conn(0x7feb00077c30 msgr2=0x7feb0006b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 shutdown_connections 2024-05-08T05:40:00.587 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.586+0000 7feb0701d640 1 -- 172.21.15.59:0/4134424159 wait complete. 2024-05-08T05:40:00.598 DEBUG:teuthology.orchestra.run.smithi059:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-05-08T05:40:00.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.744+0000 7faf1c796640 1 -- 172.21.15.59:0/27243838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf1410cfa0 msgr2=0x7faf1410f410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.744+0000 7faf1c796640 1 --2- 172.21.15.59:0/27243838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf1410cfa0 0x7faf1410f410 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7faf0c009900 tx=0x7faf0c02a480 comp rx=0 tx=0).stop 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.744+0000 7faf1c796640 1 -- 172.21.15.59:0/27243838 shutdown_connections 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.744+0000 7faf1c796640 1 --2- 172.21.15.59:0/27243838 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf1410cfa0 0x7faf1410f410 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.744+0000 7faf1c796640 1 --2- 172.21.15.59:0/27243838 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf14104580 0x7faf1410ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.744+0000 7faf1c796640 1 --2- 172.21.15.59:0/27243838 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf14103c60 0x7faf14104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.744+0000 7faf1c796640 1 -- 172.21.15.59:0/27243838 >> 172.21.15.59:0/27243838 conn(0x7faf14078a30 msgr2=0x7faf14078e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.744+0000 7faf1c796640 1 -- 172.21.15.59:0/27243838 shutdown_connections 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.744+0000 7faf1c796640 1 -- 172.21.15.59:0/27243838 wait complete. 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf1c796640 1 Processor -- start 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf1c796640 1 -- start start 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf1c796640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf14103c60 0x7faf141990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:00.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf1c796640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf14104580 0x7faf14199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf1c796640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf1410cfa0 0x7faf1419d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf1c796640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7faf14112c20 con 0x7faf14104580 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf1c796640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7faf14112aa0 con 0x7faf1410cfa0 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf1c796640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7faf14112da0 con 0x7faf14103c60 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf19d31640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf14104580 0x7faf14199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.745+0000 7faf19d31640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf14104580 0x7faf14199600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44018/0 (socket says 172.21.15.59:44018) 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf19d31640 1 -- 172.21.15.59:0/3127138538 learned_addr learned my addr 172.21.15.59:0/3127138538 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf1ad33640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf1410cfa0 0x7faf1419d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf19d31640 1 -- 172.21.15.59:0/3127138538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf14103c60 msgr2=0x7faf141990c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf1a532640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf14103c60 0x7faf141990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf19d31640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf14103c60 0x7faf141990c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf19d31640 1 -- 172.21.15.59:0/3127138538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf1410cfa0 msgr2=0x7faf1419d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf19d31640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf1410cfa0 0x7faf1419d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.749 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf19d31640 1 -- 172.21.15.59:0/3127138538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf1419e0b0 con 0x7faf14104580 2024-05-08T05:40:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf1ad33640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf1410cfa0 0x7faf1419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:40:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf1a532640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf14103c60 0x7faf141990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:40:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf19d31640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf14104580 0x7faf14199600 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e9f0 tx=0x7faf0400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf0b7fe640 1 -- 172.21.15.59:0/3127138538 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf0400cd80 con 0x7faf14104580 2024-05-08T05:40:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf0b7fe640 1 -- 172.21.15.59:0/3127138538 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf040108e0 con 0x7faf14104580 2024-05-08T05:40:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf1419e3a0 con 0x7faf14104580 2024-05-08T05:40:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf0b7fe640 1 -- 172.21.15.59:0/3127138538 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf04018980 con 0x7faf14104580 2024-05-08T05:40:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.746+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf141a5c30 con 0x7faf14104580 2024-05-08T05:40:00.750 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.747+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faf14050740 con 0x7faf14104580 2024-05-08T05:40:00.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.749+0000 7faf0b7fe640 1 -- 172.21.15.59:0/3127138538 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7faf04010430 con 0x7faf14104580 2024-05-08T05:40:00.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.750+0000 7faf0b7fe640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faf000b2a10 0x7faf000b4ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:00.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.750+0000 7faf1a532640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faf000b2a10 0x7faf000b4ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:00.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.750+0000 7faf0b7fe640 1 -- 172.21.15.59:0/3127138538 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7faf040e3340 con 0x7faf14104580 2024-05-08T05:40:00.751 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.750+0000 7faf1a532640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faf000b2a10 0x7faf000b4ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faf10005fd0 tx=0x7faf10009500 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:00.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.753+0000 7faf0b7fe640 1 -- 172.21.15.59:0/3127138538 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7faf040aa8f0 con 0x7faf14104580 2024-05-08T05:40:00.971 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.970+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 --> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7faf14062df0 con 0x7faf000b2a10 2024-05-08T05:40:00.989 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.989+0000 7faf0b7fe640 1 -- 172.21.15.59:0/3127138538 <== mgr.24962 v2:172.21.15.59:6800/1895772199 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7faf14062df0 con 0x7faf000b2a10 2024-05-08T05:40:00.989 INFO:teuthology.orchestra.run.smithi059.stdout:/ 1m 6m3h 2024-05-08T05:40:00.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faf000b2a10 msgr2=0x7faf000b4ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faf000b2a10 0x7faf000b4ed0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7faf10005fd0 tx=0x7faf10009500 comp rx=0 tx=0).stop 2024-05-08T05:40:00.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf14104580 msgr2=0x7faf14199600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:00.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf14104580 0x7faf14199600 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7faf0400e9f0 tx=0x7faf0400eeb0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 shutdown_connections 2024-05-08T05:40:00.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7faf000b2a10 0x7faf000b4ed0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7faf1410cfa0 0x7faf1419d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7faf14104580 0x7faf14199600 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.992 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 --2- 172.21.15.59:0/3127138538 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7faf14103c60 0x7faf141990c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:00.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 >> 172.21.15.59:0/3127138538 conn(0x7faf14078a30 msgr2=0x7faf140fe9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:00.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 shutdown_connections 2024-05-08T05:40:00.993 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:00.991+0000 7faf1c796640 1 -- 172.21.15.59:0/3127138538 wait complete. 2024-05-08T05:40:01.004 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-05-08T05:40:01.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-05-08T05:40:01.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.128+0000 7f298e39e640 1 -- 172.21.15.59:0/2155484923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2988109eb0 msgr2=0x7f2988111a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.128+0000 7f298e39e640 1 --2- 172.21.15.59:0/2155484923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2988109eb0 0x7f2988111a80 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7f2980009ca0 tx=0x7f298002a8c0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.128+0000 7f298e39e640 1 -- 172.21.15.59:0/2155484923 shutdown_connections 2024-05-08T05:40:01.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.128+0000 7f298e39e640 1 --2- 172.21.15.59:0/2155484923 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f2988109eb0 0x7f2988111a80 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.129 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.128+0000 7f298e39e640 1 --2- 172.21.15.59:0/2155484923 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2988105790 0x7f29881097d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.128+0000 7f298e39e640 1 --2- 172.21.15.59:0/2155484923 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2988104de0 0x7f29881051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.128+0000 7f298e39e640 1 -- 172.21.15.59:0/2155484923 >> 172.21.15.59:0/2155484923 conn(0x7f2988100660 msgr2=0x7f2988102a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.128+0000 7f298e39e640 1 -- 172.21.15.59:0/2155484923 shutdown_connections 2024-05-08T05:40:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.128+0000 7f298e39e640 1 -- 172.21.15.59:0/2155484923 wait complete. 2024-05-08T05:40:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298e39e640 1 Processor -- start 2024-05-08T05:40:01.130 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298e39e640 1 -- start start 2024-05-08T05:40:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298e39e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2988104de0 0x7f298819b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298e39e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2988109eb0 0x7f298819b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.132 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298e39e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f298819fb80 0x7f29881a0030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298e39e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f2988114e60 con 0x7f298819fb80 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298e39e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f2988114ce0 con 0x7f2988109eb0 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298e39e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f2988114fe0 con 0x7f2988104de0 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298c93b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f298819fb80 0x7f29881a0030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298c93b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f298819fb80 0x7f29881a0030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44040/0 (socket says 172.21.15.59:44040) 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.129+0000 7f298c93b640 1 -- 172.21.15.59:0/898277042 learned_addr learned my addr 172.21.15.59:0/898277042 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f29877fe640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2988109eb0 0x7f298819b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f2987fff640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2988104de0 0x7f298819b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f298c93b640 1 -- 172.21.15.59:0/898277042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2988104de0 msgr2=0x7f298819b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f298c93b640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2988104de0 0x7f298819b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f298c93b640 1 -- 172.21.15.59:0/898277042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2988109eb0 msgr2=0x7f298819b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f298c93b640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2988109eb0 0x7f298819b800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f298c93b640 1 -- 172.21.15.59:0/898277042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29881a0670 con 0x7f298819fb80 2024-05-08T05:40:01.133 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f29877fe640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2988109eb0 0x7f298819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:40:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f298c93b640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f298819fb80 0x7f29881a0030 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2980009430 tx=0x7f29800319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f29857fa640 1 -- 172.21.15.59:0/898277042 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2980003c30 con 0x7f298819fb80 2024-05-08T05:40:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29881a0900 con 0x7f298819fb80 2024-05-08T05:40:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f29857fa640 1 -- 172.21.15.59:0/898277042 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2980003dd0 con 0x7f298819fb80 2024-05-08T05:40:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f29857fa640 1 -- 172.21.15.59:0/898277042 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298002faa0 con 0x7f298819fb80 2024-05-08T05:40:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.130+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29881a8150 con 0x7f298819fb80 2024-05-08T05:40:01.134 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.131+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2988069c90 con 0x7f298819fb80 2024-05-08T05:40:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.133+0000 7f29857fa640 1 -- 172.21.15.59:0/898277042 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f298002fc40 con 0x7f298819fb80 2024-05-08T05:40:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.133+0000 7f29857fa640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f29700b2960 0x7f29700b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.133+0000 7f2987fff640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f29700b2960 0x7f29700b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.133+0000 7f29857fa640 1 -- 172.21.15.59:0/898277042 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f29801040d0 con 0x7f298819fb80 2024-05-08T05:40:01.135 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.134+0000 7f2987fff640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f29700b2960 0x7f29700b4e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2978005fd0 tx=0x7f2978005850 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:01.137 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.137+0000 7f29857fa640 1 -- 172.21.15.59:0/898277042 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f29800cb790 con 0x7f298819fb80 2024-05-08T05:40:01.332 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.332+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f29881057c0 con 0x7f298819fb80 2024-05-08T05:40:01.334 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.333+0000 7f29857fa640 1 -- 172.21.15.59:0/898277042 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v100) ==== 74+0+25253 (secure 0 0 0) 0x7f29800cb090 con 0x7f298819fb80 2024-05-08T05:40:01.334 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:40:01.334 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":100,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:39:10.706457+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"99","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":15,"snap_epoch":99,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/2839794658":"2024-05-09T05:29:33.818933+0000","172.21.15.59:6801/546809822":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/3042855743":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/1220365613":"2024-05-09T05:29:33.818933+0000","172.21.15.59:6800/546809822":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3190007358":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]},{"pool":4,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:40:01.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f29700b2960 msgr2=0x7f29700b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f29700b2960 0x7f29700b4e20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2978005fd0 tx=0x7f2978005850 comp rx=0 tx=0).stop 2024-05-08T05:40:01.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f298819fb80 msgr2=0x7f29881a0030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.336 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f298819fb80 0x7f29881a0030 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7f2980009430 tx=0x7f29800319f0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:01 smithi059 ceph-mon[30304]: pgmap v325: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 584 op/s 2024-05-08T05:40:01.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 shutdown_connections 2024-05-08T05:40:01.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f29700b2960 0x7f29700b4e20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f298819fb80 0x7f29881a0030 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f2988109eb0 0x7f298819b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 --2- 172.21.15.59:0/898277042 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f2988104de0 0x7f298819b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 >> 172.21.15.59:0/898277042 conn(0x7f2988100660 msgr2=0x7f29881010f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:01.337 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 shutdown_connections 2024-05-08T05:40:01.338 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.336+0000 7f298e39e640 1 -- 172.21.15.59:0/898277042 wait complete. 2024-05-08T05:40:01.349 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-05-08T05:20:31.976415+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-05-08T05:25:21.249621+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-05-08T05:25:24.480989+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-05-08T05:25:35.040484+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '99', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 15, 'snap_epoch': 99, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-05-08T05:40:01.349 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-05-08T05:40:01.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 -- 172.21.15.59:0/1751608945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa864103c30 msgr2=0x7fa864104010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.459 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 --2- 172.21.15.59:0/1751608945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa864103c30 0x7fa864104010 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7fa854009900 tx=0x7fa85402a510 comp rx=0 tx=0).stop 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 -- 172.21.15.59:0/1751608945 shutdown_connections 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 --2- 172.21.15.59:0/1751608945 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa86410cf70 0x7fa86410f410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 --2- 172.21.15.59:0/1751608945 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa864104550 0x7fa86410ca30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 --2- 172.21.15.59:0/1751608945 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa864103c30 0x7fa864104010 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 -- 172.21.15.59:0/1751608945 >> 172.21.15.59:0/1751608945 conn(0x7fa8640789b0 msgr2=0x7fa864078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 -- 172.21.15.59:0/1751608945 shutdown_connections 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 -- 172.21.15.59:0/1751608945 wait complete. 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.457+0000 7fa868c3e640 1 Processor -- start 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa868c3e640 1 -- start start 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa868c3e640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa864103c30 0x7fa864199250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa868c3e640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa864104550 0x7fa864199790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa868c3e640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86410cf70 0x7fa86419db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa868c3e640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa864112c20 con 0x7fa86410cf70 2024-05-08T05:40:01.460 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa868c3e640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa864112aa0 con 0x7fa864104550 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa868c3e640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa864112da0 con 0x7fa864103c30 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa862d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86410cf70 0x7fa86419db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa862d9d640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86410cf70 0x7fa86419db10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44054/0 (socket says 172.21.15.59:44054) 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa862d9d640 1 -- 172.21.15.59:0/4106734782 learned_addr learned my addr 172.21.15.59:0/4106734782 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa86259c640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa864103c30 0x7fa864199250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa861d9b640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa864104550 0x7fa864199790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa862d9d640 1 -- 172.21.15.59:0/4106734782 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa864103c30 msgr2=0x7fa864199250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa862d9d640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa864103c30 0x7fa864199250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa862d9d640 1 -- 172.21.15.59:0/4106734782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa864104550 msgr2=0x7fa864199790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa862d9d640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa864104550 0x7fa864199790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa862d9d640 1 -- 172.21.15.59:0/4106734782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa86419e240 con 0x7fa86410cf70 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.458+0000 7fa862d9d640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86410cf70 0x7fa86419db10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00d750 tx=0x7fa85c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.459+0000 7fa8537fe640 1 -- 172.21.15.59:0/4106734782 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c017070 con 0x7fa86410cf70 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.459+0000 7fa8537fe640 1 -- 172.21.15.59:0/4106734782 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa85c0054d0 con 0x7fa86410cf70 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.459+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa86419e530 con 0x7fa86410cf70 2024-05-08T05:40:01.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.459+0000 7fa8537fe640 1 -- 172.21.15.59:0/4106734782 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa85c004c60 con 0x7fa86410cf70 2024-05-08T05:40:01.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.459+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8641a5dc0 con 0x7fa86410cf70 2024-05-08T05:40:01.462 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.460+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8640697d0 con 0x7fa86410cf70 2024-05-08T05:40:01.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.461+0000 7fa8537fe640 1 -- 172.21.15.59:0/4106734782 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa85c009bd0 con 0x7fa86410cf70 2024-05-08T05:40:01.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.462+0000 7fa8537fe640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa84c0b29e0 0x7fa84c0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.462+0000 7fa8537fe640 1 -- 172.21.15.59:0/4106734782 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fa85c0e2650 con 0x7fa86410cf70 2024-05-08T05:40:01.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.462+0000 7fa86259c640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa84c0b29e0 0x7fa84c0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.463 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.462+0000 7fa86259c640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa84c0b29e0 0x7fa84c0b4ea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa854031440 tx=0x7fa854005b50 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:01.465 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.465+0000 7fa8537fe640 1 -- 172.21.15.59:0/4106734782 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa85c0a9ce0 con 0x7fa86410cf70 2024-05-08T05:40:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:01 smithi143 ceph-mon[34337]: pgmap v325: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 584 op/s 2024-05-08T05:40:01.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:01 smithi096 ceph-mon[34253]: pgmap v325: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 1.3 MiB/s wr, 584 op/s 2024-05-08T05:40:01.657 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.657+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fa864108150 con 0x7fa86410cf70 2024-05-08T05:40:01.658 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.658+0000 7fa8537fe640 1 -- 172.21.15.59:0/4106734782 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v100) ==== 93+0+10 (secure 0 0 0) 0x7fa85c0a95e0 con 0x7fa86410cf70 2024-05-08T05:40:01.658 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 1 2024-05-08T05:40:01.660 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa84c0b29e0 msgr2=0x7fa84c0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa84c0b29e0 0x7fa84c0b4ea0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fa854031440 tx=0x7fa854005b50 comp rx=0 tx=0).stop 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86410cf70 msgr2=0x7fa86419db10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86410cf70 0x7fa86419db10 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7fa85c00d750 tx=0x7fa85c00dc10 comp rx=0 tx=0).stop 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 shutdown_connections 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa84c0b29e0 0x7fa84c0b4ea0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa86410cf70 0x7fa86419db10 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa864104550 0x7fa864199790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 --2- 172.21.15.59:0/4106734782 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa864103c30 0x7fa864199250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 >> 172.21.15.59:0/4106734782 conn(0x7fa8640789b0 msgr2=0x7fa8640ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 shutdown_connections 2024-05-08T05:40:01.661 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.660+0000 7fa868c3e640 1 -- 172.21.15.59:0/4106734782 wait complete. 2024-05-08T05:40:01.672 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-05-08T05:40:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.779+0000 7ff7085af640 1 -- 172.21.15.59:0/1521829194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700103340 msgr2=0x7ff70010f850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.782 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.779+0000 7ff7085af640 1 --2- 172.21.15.59:0/1521829194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700103340 0x7ff70010f850 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff6f8009cd0 tx=0x7ff6f802a8e0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.779+0000 7ff7085af640 1 -- 172.21.15.59:0/1521829194 shutdown_connections 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.779+0000 7ff7085af640 1 --2- 172.21.15.59:0/1521829194 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700103340 0x7ff70010f850 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.779+0000 7ff7085af640 1 --2- 172.21.15.59:0/1521829194 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff700102980 0x7ff700102e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.779+0000 7ff7085af640 1 --2- 172.21.15.59:0/1521829194 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff700108990 0x7ff700108d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.779+0000 7ff7085af640 1 -- 172.21.15.59:0/1521829194 >> 172.21.15.59:0/1521829194 conn(0x7ff7000fe3f0 msgr2=0x7ff700100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 -- 172.21.15.59:0/1521829194 shutdown_connections 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 -- 172.21.15.59:0/1521829194 wait complete. 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 Processor -- start 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 -- start start 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700102980 0x7ff700074cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff700103340 0x7ff7000751f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff700108990 0x7ff700075730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff700112c20 con 0x7ff700102980 2024-05-08T05:40:01.783 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff700112aa0 con 0x7ff700103340 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff7085af640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff700112da0 con 0x7ff700108990 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.780+0000 7ff70634b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700102980 0x7ff700074cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff70634b640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700102980 0x7ff700074cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44060/0 (socket says 172.21.15.59:44060) 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff70634b640 1 -- 172.21.15.59:0/1412464054 learned_addr learned my addr 172.21.15.59:0/1412464054 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff705b4a640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff700103340 0x7ff7000751f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff706b4c640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff700108990 0x7ff700075730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff70634b640 1 -- 172.21.15.59:0/1412464054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff700108990 msgr2=0x7ff700075730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff70634b640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff700108990 0x7ff700075730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff70634b640 1 -- 172.21.15.59:0/1412464054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff700103340 msgr2=0x7ff7000751f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff70634b640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff700103340 0x7ff7000751f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff70634b640 1 -- 172.21.15.59:0/1412464054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7001a7660 con 0x7ff700102980 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff70634b640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700102980 0x7ff700074cb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00d950 tx=0x7ff6fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff6f77fe640 1 -- 172.21.15.59:0/1412464054 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc017030 con 0x7ff700102980 2024-05-08T05:40:01.784 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff6f77fe640 1 -- 172.21.15.59:0/1412464054 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff6fc004020 con 0x7ff700102980 2024-05-08T05:40:01.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7001a7950 con 0x7ff700102980 2024-05-08T05:40:01.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff6f77fe640 1 -- 172.21.15.59:0/1412464054 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff6fc0047a0 con 0x7ff700102980 2024-05-08T05:40:01.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.781+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7001a7ef0 con 0x7ff700102980 2024-05-08T05:40:01.785 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.782+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff700069c20 con 0x7ff700102980 2024-05-08T05:40:01.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.784+0000 7ff6f77fe640 1 -- 172.21.15.59:0/1412464054 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff6fc009a30 con 0x7ff700102980 2024-05-08T05:40:01.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.784+0000 7ff6f77fe640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6ec0b29e0 0x7ff6ec0b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:01.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.785+0000 7ff705b4a640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6ec0b29e0 0x7ff6ec0b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:01.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.785+0000 7ff6f77fe640 1 -- 172.21.15.59:0/1412464054 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7ff6fc0e2a30 con 0x7ff700102980 2024-05-08T05:40:01.786 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.785+0000 7ff705b4a640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6ec0b29e0 0x7ff6ec0b4ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff700076740 tx=0x7ff6f0005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:01.788 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.788+0000 7ff6f77fe640 1 -- 172.21.15.59:0/1412464054 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff6fc0aa0c0 con 0x7ff700102980 2024-05-08T05:40:01.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.979+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7ff70010af70 con 0x7ff700102980 2024-05-08T05:40:01.979 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.979+0000 7ff6f77fe640 1 -- 172.21.15.59:0/1412464054 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v100) ==== 104+0+11 (secure 0 0 0) 0x7ff6fc0a99c0 con 0x7ff700102980 2024-05-08T05:40:01.980 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6ec0b29e0 msgr2=0x7ff6ec0b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6ec0b29e0 0x7ff6ec0b4ea0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff700076740 tx=0x7ff6f0005f90 comp rx=0 tx=0).stop 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700102980 msgr2=0x7ff700074cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700102980 0x7ff700074cb0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff6fc00d950 tx=0x7ff6fc00de10 comp rx=0 tx=0).stop 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 shutdown_connections 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff6ec0b29e0 0x7ff6ec0b4ea0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff700108990 0x7ff700075730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff700103340 0x7ff7000751f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 --2- 172.21.15.59:0/1412464054 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff700102980 0x7ff700074cb0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:01.982 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 >> 172.21.15.59:0/1412464054 conn(0x7ff7000fe3f0 msgr2=0x7ff7000feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:01.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 shutdown_connections 2024-05-08T05:40:01.983 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:01.981+0000 7ff7085af640 1 -- 172.21.15.59:0/1412464054 wait complete. 2024-05-08T05:40:01.993 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-05-08T05:40:02.103 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 -- 172.21.15.59:0/22766018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70102980 msgr2=0x7f1c7010e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 --2- 172.21.15.59:0/22766018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70102980 0x7f1c7010e970 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f1c68009cd0 tx=0x7f1c6802a8e0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 -- 172.21.15.59:0/22766018 shutdown_connections 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 --2- 172.21.15.59:0/22766018 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70102980 0x7f1c7010e970 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 --2- 172.21.15.59:0/22766018 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c70101fc0 0x7f1c70102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 --2- 172.21.15.59:0/22766018 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70107ab0 0x7f1c70107e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 -- 172.21.15.59:0/22766018 >> 172.21.15.59:0/22766018 conn(0x7f1c70077e70 msgr2=0x7f1c700ff1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 -- 172.21.15.59:0/22766018 shutdown_connections 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 -- 172.21.15.59:0/22766018 wait complete. 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.101+0000 7f1c7595f640 1 Processor -- start 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c7595f640 1 -- start start 2024-05-08T05:40:02.104 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c7595f640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70101fc0 0x7f1c7010f400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c7595f640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102980 0x7f1c7010f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c7595f640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c70107ab0 0x7f1c7010fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c7595f640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70112ec0 con 0x7f1c70101fc0 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c7595f640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70112d40 con 0x7f1c70102980 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c7595f640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f1c70113040 con 0x7f1c70107ab0 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70101fc0 0x7f1c7010f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6effd640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70101fc0 0x7f1c7010f400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44082/0 (socket says 172.21.15.59:44082) 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6effd640 1 -- 172.21.15.59:0/1739321547 learned_addr learned my addr 172.21.15.59:0/1739321547 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6e7fc640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102980 0x7f1c7010f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6f7fe640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c70107ab0 0x7f1c7010fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6effd640 1 -- 172.21.15.59:0/1739321547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c70107ab0 msgr2=0x7f1c7010fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6effd640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c70107ab0 0x7f1c7010fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6effd640 1 -- 172.21.15.59:0/1739321547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102980 msgr2=0x7f1c7010f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.105 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6effd640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102980 0x7f1c7010f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.102+0000 7f1c6effd640 1 -- 172.21.15.59:0/1739321547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1c701a7920 con 0x7f1c70101fc0 2024-05-08T05:40:02.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.103+0000 7f1c6effd640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70101fc0 0x7f1c7010f400 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000e9f0 tx=0x7f1c6000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:02.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.103+0000 7f1c7495d640 1 -- 172.21.15.59:0/1739321547 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c6000cd80 con 0x7f1c70101fc0 2024-05-08T05:40:02.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.103+0000 7f1c7495d640 1 -- 172.21.15.59:0/1739321547 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1c600108e0 con 0x7f1c70101fc0 2024-05-08T05:40:02.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.103+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1c701a7c10 con 0x7f1c70101fc0 2024-05-08T05:40:02.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.103+0000 7f1c7495d640 1 -- 172.21.15.59:0/1739321547 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1c60018980 con 0x7f1c70101fc0 2024-05-08T05:40:02.106 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.103+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1c701a8160 con 0x7f1c70101fc0 2024-05-08T05:40:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.104+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1c70069c20 con 0x7f1c70101fc0 2024-05-08T05:40:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.105+0000 7f1c7495d640 1 -- 172.21.15.59:0/1739321547 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f1c60010430 con 0x7f1c70101fc0 2024-05-08T05:40:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.106+0000 7f1c7495d640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1c580b2960 0x7f1c580b4e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.106+0000 7f1c6e7fc640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1c580b2960 0x7f1c580b4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.106+0000 7f1c7495d640 1 -- 172.21.15.59:0/1739321547 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f1c600e2de0 con 0x7f1c70101fc0 2024-05-08T05:40:02.107 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.106+0000 7f1c6e7fc640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1c580b2960 0x7f1c580b4e20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c70110e90 tx=0x7f1c64005f90 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:02.111 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.110+0000 7f1c7495d640 1 -- 172.21.15.59:0/1739321547 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f1c600aa3f0 con 0x7f1c70101fc0 2024-05-08T05:40:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.308+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f1c7006add0 con 0x7f1c70101fc0 2024-05-08T05:40:02.308 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.308+0000 7f1c7495d640 1 -- 172.21.15.59:0/1739321547 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v100) ==== 100+0+11 (secure 0 0 0) 0x7f1c600a9cf0 con 0x7f1c70101fc0 2024-05-08T05:40:02.309 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 64 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1c580b2960 msgr2=0x7f1c580b4e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1c580b2960 0x7f1c580b4e20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1c70110e90 tx=0x7f1c64005f90 comp rx=0 tx=0).stop 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70101fc0 msgr2=0x7f1c7010f400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70101fc0 0x7f1c7010f400 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f1c6000e9f0 tx=0x7f1c6000eeb0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 shutdown_connections 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f1c580b2960 0x7f1c580b4e20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f1c70107ab0 0x7f1c7010fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f1c70102980 0x7f1c7010f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 --2- 172.21.15.59:0/1739321547 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f1c70101fc0 0x7f1c7010f400 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 >> 172.21.15.59:0/1739321547 conn(0x7f1c70077e70 msgr2=0x7f1c70104a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 shutdown_connections 2024-05-08T05:40:02.311 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.310+0000 7f1c7595f640 1 -- 172.21.15.59:0/1739321547 wait complete. 2024-05-08T05:40:02.321 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:02 smithi059 ceph-mon[30304]: from='client.16725 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:40:02.321 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:02 smithi059 ceph-mon[30304]: from='client.16731 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:40:02.321 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:02 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/898277042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:40:02.321 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:02 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/4106734782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:40:02.321 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:02 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1412464054' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T05:40:02.322 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 -- 172.21.15.59:0/4157988599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a01017b0 msgr2=0x7ff1a0101b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/4157988599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a01017b0 0x7ff1a0101b90 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ff190009ca0 tx=0x7ff19002a8c0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 -- 172.21.15.59:0/4157988599 shutdown_connections 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/4157988599 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1a010f360 0x7ff1a0111750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/4157988599 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a01020d0 0x7ff1a010ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/4157988599 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a01017b0 0x7ff1a0101b90 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 -- 172.21.15.59:0/4157988599 >> 172.21.15.59:0/4157988599 conn(0x7ff1a00789b0 msgr2=0x7ff1a0078dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 -- 172.21.15.59:0/4157988599 shutdown_connections 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 -- 172.21.15.59:0/4157988599 wait complete. 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 Processor -- start 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.430+0000 7ff1a75b6640 1 -- start start 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a75b6640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a01017b0 0x7ff1a019b320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.433 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a75b6640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1a01020d0 0x7ff1a019b860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a75b6640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a010f360 0x7ff1a019fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a75b6640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0114e00 con 0x7ff1a010f360 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a75b6640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0114c80 con 0x7ff1a01020d0 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a75b6640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff1a0114f80 con 0x7ff1a01017b0 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5b53640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a010f360 0x7ff1a019fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5b53640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a010f360 0x7ff1a019fbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44102/0 (socket says 172.21.15.59:44102) 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5b53640 1 -- 172.21.15.59:0/1495915594 learned_addr learned my addr 172.21.15.59:0/1495915594 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5352640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a01017b0 0x7ff1a019b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a4b51640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1a01020d0 0x7ff1a019b860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5b53640 1 -- 172.21.15.59:0/1495915594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a01017b0 msgr2=0x7ff1a019b320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5b53640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a01017b0 0x7ff1a019b320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5b53640 1 -- 172.21.15.59:0/1495915594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1a01020d0 msgr2=0x7ff1a019b860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.434 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5b53640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1a01020d0 0x7ff1a019b860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5b53640 1 -- 172.21.15.59:0/1495915594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff1a01a0310 con 0x7ff1a010f360 2024-05-08T05:40:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff1a5b53640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a010f360 0x7ff1a019fbe0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff19800e970 tx=0x7ff19800ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff19e7fc640 1 -- 172.21.15.59:0/1495915594 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff19800ccc0 con 0x7ff1a010f360 2024-05-08T05:40:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.431+0000 7ff19e7fc640 1 -- 172.21.15.59:0/1495915594 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff1980054d0 con 0x7ff1a010f360 2024-05-08T05:40:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.432+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff1a01a0600 con 0x7ff1a010f360 2024-05-08T05:40:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.432+0000 7ff19e7fc640 1 -- 172.21.15.59:0/1495915594 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff198010430 con 0x7ff1a010f360 2024-05-08T05:40:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.432+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff1a01a7e90 con 0x7ff1a010f360 2024-05-08T05:40:02.435 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.432+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff1a00697d0 con 0x7ff1a010f360 2024-05-08T05:40:02.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.434+0000 7ff19e7fc640 1 -- 172.21.15.59:0/1495915594 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff198004fe0 con 0x7ff1a010f360 2024-05-08T05:40:02.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.435+0000 7ff19e7fc640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1880b29e0 0x7ff1880b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.435+0000 7ff1a5352640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1880b29e0 0x7ff1880b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.435+0000 7ff19e7fc640 1 -- 172.21.15.59:0/1495915594 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7ff1980e2fa0 con 0x7ff1a010f360 2024-05-08T05:40:02.436 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.435+0000 7ff1a5352640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1880b29e0 0x7ff1880b4ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff190000f80 tx=0x7ff190005ef0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:02.438 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.438+0000 7ff19e7fc640 1 -- 172.21.15.59:0/1495915594 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff198054df0 con 0x7ff1a010f360 2024-05-08T05:40:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:02 smithi143 ceph-mon[34337]: from='client.16725 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:40:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:02 smithi143 ceph-mon[34337]: from='client.16731 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:40:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:02 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/898277042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:40:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:02 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/4106734782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:40:02.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:02 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1412464054' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T05:40:02.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:02 smithi096 ceph-mon[34253]: from='client.16725 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:40:02.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:02 smithi096 ceph-mon[34253]: from='client.16731 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-05-08T05:40:02.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:02 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/898277042' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:40:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:02 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/4106734782' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-05-08T05:40:02.530 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:02 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1412464054' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-05-08T05:40:02.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.629+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ff1a002db50 con 0x7ff1a010f360 2024-05-08T05:40:02.630 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.629+0000 7ff19e7fc640 1 -- 172.21.15.59:0/1495915594 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v100) ==== 103+0+11 (secure 0 0 0) 0x7ff1980aa5b0 con 0x7ff1a010f360 2024-05-08T05:40:02.630 INFO:teuthology.orchestra.run.smithi059.stdout:pg_num: 32 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1880b29e0 msgr2=0x7ff1880b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1880b29e0 0x7ff1880b4ea0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7ff190000f80 tx=0x7ff190005ef0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a010f360 msgr2=0x7ff1a019fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a010f360 0x7ff1a019fbe0 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7ff19800e970 tx=0x7ff19800ee30 comp rx=0 tx=0).stop 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 shutdown_connections 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1880b29e0 0x7ff1880b4ea0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff1a010f360 0x7ff1a019fbe0 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff1a01020d0 0x7ff1a019b860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 --2- 172.21.15.59:0/1495915594 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff1a01017b0 0x7ff1a019b320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.632 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 >> 172.21.15.59:0/1495915594 conn(0x7ff1a00789b0 msgr2=0x7ff1a01109e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 shutdown_connections 2024-05-08T05:40:02.633 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.631+0000 7ff1a75b6640 1 -- 172.21.15.59:0/1495915594 wait complete. 2024-05-08T05:40:02.643 DEBUG:teuthology.orchestra.run.smithi059:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-05-08T05:40:02.752 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.750+0000 7fefb0771640 1 -- 172.21.15.59:0/2140642958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa810cfd0 msgr2=0x7fefa810f440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.750+0000 7fefb0771640 1 --2- 172.21.15.59:0/2140642958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa810cfd0 0x7fefa810f440 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009ca0 tx=0x7fefa002a950 comp rx=0 tx=0).stop 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.750+0000 7fefb0771640 1 -- 172.21.15.59:0/2140642958 shutdown_connections 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.750+0000 7fefb0771640 1 --2- 172.21.15.59:0/2140642958 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa810cfd0 0x7fefa810f440 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.750+0000 7fefb0771640 1 --2- 172.21.15.59:0/2140642958 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa81045b0 0x7fefa810ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.750+0000 7fefb0771640 1 --2- 172.21.15.59:0/2140642958 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa8103c90 0x7fefa8104070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.750+0000 7fefb0771640 1 -- 172.21.15.59:0/2140642958 >> 172.21.15.59:0/2140642958 conn(0x7fefa8078a10 msgr2=0x7fefa8078e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.750+0000 7fefb0771640 1 -- 172.21.15.59:0/2140642958 shutdown_connections 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.750+0000 7fefb0771640 1 -- 172.21.15.59:0/2140642958 wait complete. 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefb0771640 1 Processor -- start 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefb0771640 1 -- start start 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefb0771640 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa8103c90 0x7fefa81990f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.753 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefb0771640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa81045b0 0x7fefa8199630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefb0771640 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa810cfd0 0x7fefa819d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefb0771640 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8112c50 con 0x7fefa81045b0 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefb0771640 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8112ad0 con 0x7fefa8103c90 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefb0771640 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fefa8112dd0 con 0x7fefa810cfd0 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefadd0c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa81045b0 0x7fefa8199630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefadd0c640 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa81045b0 0x7fefa8199630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:44108/0 (socket says 172.21.15.59:44108) 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefadd0c640 1 -- 172.21.15.59:0/3591581453 learned_addr learned my addr 172.21.15.59:0/3591581453 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefae50d640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa8103c90 0x7fefa81990f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefaed0e640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa810cfd0 0x7fefa819d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefadd0c640 1 -- 172.21.15.59:0/3591581453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa810cfd0 msgr2=0x7fefa819d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefadd0c640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa810cfd0 0x7fefa819d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefadd0c640 1 -- 172.21.15.59:0/3591581453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa8103c90 msgr2=0x7fefa81990f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:02.754 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefadd0c640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa8103c90 0x7fefa81990f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.751+0000 7fefadd0c640 1 -- 172.21.15.59:0/3591581453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefa819e090 con 0x7fefa81045b0 2024-05-08T05:40:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.752+0000 7fefadd0c640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa81045b0 0x7fefa8199630 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fef9800d950 tx=0x7fef9800dd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.752+0000 7fef9f7fe640 1 -- 172.21.15.59:0/3591581453 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98017030 con 0x7fefa81045b0 2024-05-08T05:40:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.752+0000 7fef9f7fe640 1 -- 172.21.15.59:0/3591581453 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef98004020 con 0x7fefa81045b0 2024-05-08T05:40:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.752+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefa819e380 con 0x7fefa81045b0 2024-05-08T05:40:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.752+0000 7fef9f7fe640 1 -- 172.21.15.59:0/3591581453 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef98005560 con 0x7fefa81045b0 2024-05-08T05:40:02.755 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.752+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefa81a5c10 con 0x7fefa81045b0 2024-05-08T05:40:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.753+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fefa8069c20 con 0x7fefa81045b0 2024-05-08T05:40:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.754+0000 7fef9f7fe640 1 -- 172.21.15.59:0/3591581453 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fef98009a30 con 0x7fefa81045b0 2024-05-08T05:40:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.755+0000 7fef9f7fe640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fef940b29e0 0x7fef940b4ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.755+0000 7fefae50d640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fef940b29e0 0x7fef940b4ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.755+0000 7fef9f7fe640 1 -- 172.21.15.59:0/3591581453 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fef980e22b0 con 0x7fefa81045b0 2024-05-08T05:40:02.756 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.755+0000 7fefae50d640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fef940b29e0 0x7fef940b4ea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fefa400abe0 tx=0x7fefa400a2d0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:02.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:02.759+0000 7fef9f7fe640 1 -- 172.21.15.59:0/3591581453 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fef980a9860 con 0x7fefa81045b0 2024-05-08T05:40:03.007 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.007+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7fefa8107bc0 con 0x7fefa81045b0 2024-05-08T05:40:03.016 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.016+0000 7fef9f7fe640 1 -- 172.21.15.59:0/3591581453 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+816174 (secure 0 0 0) 0x7fef980a9160 con 0x7fefa81045b0 2024-05-08T05:40:03.027 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:40:03.027 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "epoch": 26, 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "active_gid": 24962, 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "active_name": "x", 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "active_addrs": { 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6800", 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1895772199 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v1", 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:6801", 2024-05-08T05:40:03.028 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1895772199 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: "active_addr": "172.21.15.59:6801/1895772199", 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: "active_change": "2024-05-08T05:29:33.819247+0000", 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: "active_mgr_features": 4540701547738038271, 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: "available": true, 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: "standbys": [ 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: "gid": 15270, 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "y", 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_features": 4540701547738038271, 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2024-05-08T05:40:03.029 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.030 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.031 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.033 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.034 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.035 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.037 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.038 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.040 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.042 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.044 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.045 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.046 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-05-08T05:40:03.047 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2024-05-08T05:40:03.048 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.049 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2024-05-08T05:40:03.050 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T05:40:03.051 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-05-08T05:40:03.052 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.053 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.055 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.056 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.057 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.058 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2024-05-08T05:40:03.059 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "read", 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap", 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap-read" 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2024-05-08T05:40:03.060 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.061 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.062 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2024-05-08T05:40:03.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.064 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2024-05-08T05:40:03.065 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.066 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T05:40:03.067 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T05:40:03.068 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T05:40:03.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T05:40:03.070 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.071 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2024-05-08T05:40:03.072 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.073 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.074 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.075 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.076 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2024-05-08T05:40:03.077 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.078 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.079 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.080 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.081 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.082 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T05:40:03.083 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.084 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.085 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.086 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.087 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.088 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2024-05-08T05:40:03.089 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_samba": { 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_samba", 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.090 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Samba/SMB container image", 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.091 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T05:40:03.092 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T05:40:03.093 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T05:40:03.094 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.095 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.096 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.097 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.098 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "grafana_dashboards_path": { 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "grafana_dashboards_path", 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.099 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "hw_monitoring": { 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hw_monitoring", 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.101 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.102 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.103 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.104 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.105 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.106 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.107 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.108 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T05:40:03.109 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2024-05-08T05:40:03.110 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2024-05-08T05:40:03.111 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.112 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.113 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.114 INFO:teuthology.orchestra.run.smithi059.stdout: "oob_default_addr": { 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "oob_default_addr", 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "169.254.1.1", 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2024-05-08T05:40:03.115 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2024-05-08T05:40:03.116 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.118 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.120 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2024-05-08T05:40:03.121 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.122 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.123 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.124 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.125 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.126 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.127 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.128 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.129 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.130 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.131 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.132 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.133 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.134 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.135 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.136 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.137 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.139 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.140 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.141 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.142 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T05:40:03.143 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T05:40:03.144 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.145 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.146 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.147 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.148 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.149 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.150 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-05-08T05:40:03.151 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.152 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.153 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-05-08T05:40:03.154 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.155 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.156 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.157 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.158 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "[]", 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.159 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "{}", 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.160 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.161 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.162 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.163 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.164 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.165 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.166 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.167 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.168 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.169 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.170 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.171 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.172 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.175 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.176 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T05:40:03.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.178 INFO:teuthology.orchestra.run.smithi059.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T05:40:03.179 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T05:40:03.180 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T05:40:03.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2024-05-08T05:40:03.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.183 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.184 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.185 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.186 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.187 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.188 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.189 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.190 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.191 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2024-05-08T05:40:03.192 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.195 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.196 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.197 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.198 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.199 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.200 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.201 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.202 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.203 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.204 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.205 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.206 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.207 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.209 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2024-05-08T05:40:03.210 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.211 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.212 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.213 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.214 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T05:40:03.215 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.216 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.217 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.219 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.220 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.221 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.222 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.223 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.224 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.225 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.226 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2024-05-08T05:40:03.227 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.229 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.230 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.231 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.232 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.233 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.234 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.235 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.236 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2024-05-08T05:40:03.237 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:03.238 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.239 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.241 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.242 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.243 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.244 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.245 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2024-05-08T05:40:03.247 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.248 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.249 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.250 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.252 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.254 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.255 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.256 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.258 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.259 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.260 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.261 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.262 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.263 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.264 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.266 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.267 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.268 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.269 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2024-05-08T05:40:03.270 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.271 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.272 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.273 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.274 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.275 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.276 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.277 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.279 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.280 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.281 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.282 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.283 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.284 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.285 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.286 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.287 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.288 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.289 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.290 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.291 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.292 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.293 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T05:40:03.294 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.295 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.296 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.297 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.298 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2024-05-08T05:40:03.299 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.300 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.301 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.302 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.304 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.305 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.306 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.307 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.308 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.309 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.310 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.311 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.312 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.313 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.314 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.315 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.316 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.317 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.318 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.319 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.320 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.321 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2024-05-08T05:40:03.322 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.323 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.324 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.325 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.326 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.327 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.328 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2024-05-08T05:40:03.329 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:03.330 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.331 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.333 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.334 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.335 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.336 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.337 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.338 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.339 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.340 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2024-05-08T05:40:03.341 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-05-08T05:40:03.342 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.343 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.344 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.345 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.346 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-05-08T05:40:03.347 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.348 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.349 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.350 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.351 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.352 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2024-05-08T05:40:03.353 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2024-05-08T05:40:03.354 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.355 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2024-05-08T05:40:03.356 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.358 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.359 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.360 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.361 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.362 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.363 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.364 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.365 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.366 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.368 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.369 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.370 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.371 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.372 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.373 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.374 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.375 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.376 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.377 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.378 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.379 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2024-05-08T05:40:03.380 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.381 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.382 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.383 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.384 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.385 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2024-05-08T05:40:03.386 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2024-05-08T05:40:03.387 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.388 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2024-05-08T05:40:03.389 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2024-05-08T05:40:03.390 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2024-05-08T05:40:03.391 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2024-05-08T05:40:03.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2024-05-08T05:40:03.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2024-05-08T05:40:03.394 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.395 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2024-05-08T05:40:03.396 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.397 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.398 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smb", 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.399 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.400 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.401 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.402 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.403 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.404 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "update_orchestration": { 2024-05-08T05:40:03.405 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "update_orchestration", 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.406 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.407 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2024-05-08T05:40:03.408 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.409 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.410 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.411 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.412 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.413 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2024-05-08T05:40:03.414 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.415 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.416 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.417 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.418 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.419 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.420 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.421 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.422 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.423 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.424 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.425 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.426 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.427 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.428 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.429 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.430 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.431 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.432 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.433 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.434 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.435 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.436 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.437 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.438 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.439 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.440 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-05-08T05:40:03.441 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.442 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.443 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.444 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.445 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.446 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.447 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.448 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.449 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.450 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.451 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.452 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2024-05-08T05:40:03.453 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.454 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.455 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.456 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.457 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.458 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.459 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.460 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.461 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.462 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.463 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.464 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.465 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T05:40:03.466 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.467 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.468 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_no_wait": { 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.469 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.470 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.471 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.472 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.473 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.474 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.475 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.476 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.477 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.478 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.479 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.480 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.481 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.482 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: "gid": 24949, 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "z", 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: "mgr_features": 4540701547738038271, 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.483 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.484 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.485 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.486 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.487 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.488 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2024-05-08T05:40:03.489 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2024-05-08T05:40:03.490 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2024-05-08T05:40:03.491 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2024-05-08T05:40:03.492 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2024-05-08T05:40:03.493 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2024-05-08T05:40:03.494 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.495 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.496 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.497 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2024-05-08T05:40:03.498 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.499 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.500 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2024-05-08T05:40:03.501 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2024-05-08T05:40:03.502 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2024-05-08T05:40:03.503 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2024-05-08T05:40:03.504 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2024-05-08T05:40:03.505 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.506 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.507 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.508 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.509 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.510 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2024-05-08T05:40:03.511 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.512 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "read", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap-read" 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.513 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.514 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.515 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.516 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.517 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.518 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-05-08T05:40:03.519 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2024-05-08T05:40:03.520 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.521 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.522 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T05:40:03.523 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.524 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.525 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.526 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.527 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.528 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.529 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.530 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2024-05-08T05:40:03.531 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.532 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.533 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T05:40:03.534 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T05:40:03.535 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.536 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.537 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.538 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.539 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.540 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.541 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_samba": { 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_samba", 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Samba/SMB container image", 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.542 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2024-05-08T05:40:03.543 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.544 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.545 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.546 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.547 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.548 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.549 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "grafana_dashboards_path": { 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "grafana_dashboards_path", 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.550 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.551 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "hw_monitoring": { 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hw_monitoring", 2024-05-08T05:40:03.552 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2024-05-08T05:40:03.553 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.554 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.555 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T05:40:03.556 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.557 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.558 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.559 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.560 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.561 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.562 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.563 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.564 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T05:40:03.565 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "oob_default_addr": { 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "oob_default_addr", 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "169.254.1.1", 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.566 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.568 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2024-05-08T05:40:03.569 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2024-05-08T05:40:03.570 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.571 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.572 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.573 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.574 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.575 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.576 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.577 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.578 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.579 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.580 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.581 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.582 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.583 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.584 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.585 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.586 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.587 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.588 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.589 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.590 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.591 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.593 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T05:40:03.594 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T05:40:03.595 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.596 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.597 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.598 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.599 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.600 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.601 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.602 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.603 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.604 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.605 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.606 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T05:40:03.607 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.608 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.609 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "[]", 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.610 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "{}", 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.611 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.612 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T05:40:03.613 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T05:40:03.614 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.615 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.616 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.617 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.618 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.619 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.620 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.621 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.622 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.623 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T05:40:03.624 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T05:40:03.625 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T05:40:03.626 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.627 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.628 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.629 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.630 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.631 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.632 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.633 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.634 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.635 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.636 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.637 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.638 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.639 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.640 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2024-05-08T05:40:03.641 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.642 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.643 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-05-08T05:40:03.644 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.645 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.646 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.647 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2024-05-08T05:40:03.648 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.649 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.650 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.651 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2024-05-08T05:40:03.652 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.653 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.654 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.655 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.656 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.657 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.658 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.658 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.658 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.658 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2024-05-08T05:40:03.658 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2024-05-08T05:40:03.658 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.658 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.659 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.659 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2024-05-08T05:40:03.659 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.659 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.659 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.659 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T05:40:03.659 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.659 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T05:40:03.660 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.661 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.662 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.663 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.664 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.665 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.666 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.667 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.668 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.669 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.670 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.671 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2024-05-08T05:40:03.672 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2024-05-08T05:40:03.673 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.674 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.675 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.676 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T05:40:03.677 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.678 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2024-05-08T05:40:03.679 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.680 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.681 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.682 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.683 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.684 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.685 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.686 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.687 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.688 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.689 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.690 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.691 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.692 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.693 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.694 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.695 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.696 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.697 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.698 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.699 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.700 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.701 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.702 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.703 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.704 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.705 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.706 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.707 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.708 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.709 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.710 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.711 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.712 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.713 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.714 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.715 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.716 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.717 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.718 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.719 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2024-05-08T05:40:03.720 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2024-05-08T05:40:03.721 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2024-05-08T05:40:03.722 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.723 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.724 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.725 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.726 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.727 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.728 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.729 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.730 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2024-05-08T05:40:03.731 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.732 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.733 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.734 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.735 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.736 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.737 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.738 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.739 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.740 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.741 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.742 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.743 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.744 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.745 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.746 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.747 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.748 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.749 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.750 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.751 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2024-05-08T05:40:03.752 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.753 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.754 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.755 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.756 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.757 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.758 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.759 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.760 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.761 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.762 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.763 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.764 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.765 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.766 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.767 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.768 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.769 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.770 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.771 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.772 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.773 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.774 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.775 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.776 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.777 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.778 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2024-05-08T05:40:03.779 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.780 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.781 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.782 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.783 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.784 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.785 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.786 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.787 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.788 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.789 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.790 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.791 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.792 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.793 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.794 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.795 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.796 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2024-05-08T05:40:03.797 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-05-08T05:40:03.798 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.799 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.800 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.801 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.802 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.803 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.804 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.805 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.806 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.807 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.808 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.809 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.810 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.811 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.812 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.813 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.814 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.815 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.816 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.817 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.818 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.819 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.820 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.821 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.822 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.823 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.824 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.825 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.826 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.827 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.828 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.829 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.830 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2024-05-08T05:40:03.831 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.832 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.833 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.834 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.835 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.836 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2024-05-08T05:40:03.837 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.838 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2024-05-08T05:40:03.839 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.840 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2024-05-08T05:40:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2024-05-08T05:40:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.841 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.842 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2024-05-08T05:40:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2024-05-08T05:40:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.843 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.844 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2024-05-08T05:40:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2024-05-08T05:40:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.845 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2024-05-08T05:40:03.846 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.847 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.848 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.849 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.850 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.851 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.852 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smb", 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.853 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.854 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.855 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.856 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.857 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.858 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.859 INFO:teuthology.orchestra.run.smithi059.stdout: "update_orchestration": { 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "update_orchestration", 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.860 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.861 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.862 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.863 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.864 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.865 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.866 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.867 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.868 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.869 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.870 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.871 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.872 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.873 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.874 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.875 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.876 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.877 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.878 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.879 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.880 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.881 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.882 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.883 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.884 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.885 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.886 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.887 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.888 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.889 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.890 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.891 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.892 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.893 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.894 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.895 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.896 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.897 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.898 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2024-05-08T05:40:03.899 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.900 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.901 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.902 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.903 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.904 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.905 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.906 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.907 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.908 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.909 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.910 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.911 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.912 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.913 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.914 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.915 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.916 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.917 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.918 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.919 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.920 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.921 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.922 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.923 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.924 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.925 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.926 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_no_wait": { 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.927 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.928 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.929 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.930 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.931 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.932 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.933 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.934 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.935 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.936 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.937 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.938 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.939 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.940 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: "modules": [ 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: "dashboard", 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: "iostat", 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: "nfs", 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: "restful", 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: "snap_schedule" 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.941 INFO:teuthology.orchestra.run.smithi059.stdout: "available_modules": [ 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "alerts", 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.942 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How frequently to reexamine health status", 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.943 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.944 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.945 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.946 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.947 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_destination": { 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_destination", 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.948 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email address to send alerts to", 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_from_name": { 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_from_name", 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "Ceph", 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.949 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Email From: name", 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_host": { 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_host", 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.950 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP server", 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_password": { 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_password", 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.951 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Password to authenticate with", 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_port": { 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_port", 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "465", 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.952 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP port", 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_sender": { 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_sender", 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.953 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SMTP envelope sender", 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_ssl": { 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_ssl", 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.954 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use SSL to connect to SMTP server", 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "smtp_user": { 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smtp_user", 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.955 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "User to authenticate as", 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.956 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "balancer", 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "active": { 2024-05-08T05:40:03.957 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "active", 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically balance PGs across cluster", 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_time": { 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_time", 2024-05-08T05:40:03.958 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0000", 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "beginning time of day to automatically balance", 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "begin_weekday": { 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "begin_weekday", 2024-05-08T05:40:03.959 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_max_iterations": { 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_max_iterations", 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.960 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "25", 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "250", 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_metrics": { 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_metrics", 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.961 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "pgs,objects,bytes", 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "crush_compat_step": { 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crush_compat_step", 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.962 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.5", 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0.001", 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "0.999", 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "aggressiveness of optimization", 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "end_time": { 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_time", 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.963 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2359", 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ending time of day to automatically balance", 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "end_weekday": { 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "end_weekday", 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.964 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "0", 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "6", 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.965 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.966 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.967 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.968 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.969 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "min_score": { 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_score", 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.970 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "upmap", 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "crush-compat", 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "none", 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "read", 2024-05-08T05:40:03.971 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap", 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap-read" 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Balancer mode", 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_ids": { 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_ids", 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.972 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.973 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.974 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_deviation": { 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_deviation", 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.975 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "deviation below which no optimization is attempted", 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "upmap_max_optimizations": { 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "upmap_max_optimizations", 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.976 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm", 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_down_multiplier": { 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_down_multiplier", 2024-05-08T05:40:03.977 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_refresh_rate": { 2024-05-08T05:40:03.978 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_refresh_rate", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "20", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "agent_starting_port": { 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "agent_starting_port", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.979 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4721", 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_ptrace": { 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_ptrace", 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.980 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_interval": { 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_interval", 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.981 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to autotune daemon memory", 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "autotune_memory_target_ratio": { 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "autotune_memory_target_ratio", 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.982 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0.7", 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm_log_destination": { 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cephadm_log_destination", 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.983 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "file", 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "file,syslog", 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "syslog" 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "cgroups_split": { 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cgroups_split", 2024-05-08T05:40:03.984 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "config_checks_enabled": { 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_checks_enabled", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.985 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "config_dashboard": { 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_dashboard", 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.986 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_alertmanager": { 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_alertmanager", 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.987 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_base": { 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_base", 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.988 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/ceph", 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Container image name, without the tag", 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_elasticsearch": { 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_elasticsearch", 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.989 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "elasticsearch container image", 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_grafana": { 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_grafana", 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.990 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_haproxy": { 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_haproxy", 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-05-08T05:40:03.991 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "HAproxy container image", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_agent": { 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_agent", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.992 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger agent container image", 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_collector": { 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_collector", 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.993 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger collector container image", 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_jaeger_query": { 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_jaeger_query", 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.994 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Jaeger query container image", 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_keepalived": { 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_keepalived", 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.995 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Keepalived container image", 2024-05-08T05:40:03.996 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.996 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.996 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.997 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.997 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_loki": { 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_loki", 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Loki container image", 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.998 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_node_exporter": { 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_node_exporter", 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_nvmeof": { 2024-05-08T05:40:03.999 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_nvmeof", 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Nvme-of container image", 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_prometheus": { 2024-05-08T05:40:04.000 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_prometheus", 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Prometheus container image", 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_promtail": { 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_promtail", 2024-05-08T05:40:04.001 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Promtail container image", 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_samba": { 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_samba", 2024-05-08T05:40:04.002 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Samba/SMB container image", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "container_image_snmp_gateway": { 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_image_snmp_gateway", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.003 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "SNMP Gateway container image", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "container_init": { 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "container_init", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.004 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Run podman/docker with `--init`", 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "daemon_cache_timeout": { 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "daemon_cache_timeout", 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.005 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "default_cephadm_command_timeout": { 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_cephadm_command_timeout", 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "900", 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-05-08T05:40:04.006 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "default_registry": { 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "default_registry", 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "docker.io", 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.007 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "device_cache_timeout": { 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_cache_timeout", 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1800", 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache device inventory", 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.008 INFO:teuthology.orchestra.run.smithi059.stdout: "device_enhanced_scan": { 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_enhanced_scan", 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use libstoragemgmt during device scans", 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.009 INFO:teuthology.orchestra.run.smithi059.stdout: "facts_cache_timeout": { 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "facts_cache_timeout", 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "seconds to cache host facts data", 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.010 INFO:teuthology.orchestra.run.smithi059.stdout: "grafana_dashboards_path": { 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "grafana_dashboards_path", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "host_check_interval": { 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "host_check_interval", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.011 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to perform a host check", 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "hw_monitoring": { 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hw_monitoring", 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.012 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "inventory_list_all": { 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "inventory_list_all", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.013 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.014 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "log_refresh_metadata": { 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_refresh_metadata", 2024-05-08T05:40:04.015 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.016 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.017 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.018 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.019 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf": { 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-05-08T05:40:04.020 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "*", 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "max_count_per_host": { 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_count_per_host", 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.021 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of daemons per service per host", 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.022 INFO:teuthology.orchestra.run.smithi059.stdout: "max_osd_draining_count": { 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_osd_draining_count", 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "migration_current": { 2024-05-08T05:40:04.023 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "migration_current", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "internal - do not modify", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "mode": { 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mode", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.024 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "root", 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm-package", 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "root" 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "mode for remote execution of cephadm", 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "oob_default_addr": { 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "oob_default_addr", 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.025 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "169.254.1.1", 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Default address for RedFish API (oob management).", 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "prometheus_alerts_path": { 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus_alerts_path", 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.026 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_insecure": { 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_insecure", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.027 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_password": { 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_password", 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.028 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_url": { 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_url", 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.029 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "registry_username": { 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "registry_username", 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-05-08T05:40:04.030 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "secure_monitoring_stack": { 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "secure_monitoring_stack", 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.031 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "service_discovery_port": { 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "service_discovery_port", 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8765", 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "cephadm service discovery port", 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.032 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.033 INFO:teuthology.orchestra.run.smithi059.stdout: "ssh_config_file": { 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssh_config_file", 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.034 INFO:teuthology.orchestra.run.smithi059.stdout: "use_agent": { 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_agent", 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "use_repo_digest": { 2024-05-08T05:40:04.035 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "use_repo_digest", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_failed_host_check": { 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_failed_host_check", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.036 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if the host check fails", 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_daemons": { 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_daemons", 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.037 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_on_stray_hosts": { 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_on_stray_hosts", 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.038 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crash", 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.039 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.040 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.041 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.042 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.043 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "retain_interval": { 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retain_interval", 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.044 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "31536000", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain crashes before pruning them", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.045 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_recent_interval": { 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_recent_interval", 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1209600", 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.046 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "time interval in which to warn about recent crashes", 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dashboard", 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-05-08T05:40:04.047 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_HOST": { 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_HOST", 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.048 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.049 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_ENABLED": { 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_ENABLED", 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.050 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.051 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "ENABLE_BROWSABLE_API": { 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ENABLE_BROWSABLE_API", 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.052 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.053 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.054 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.055 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.056 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_NFS": { 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.057 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RBD": { 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.058 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "FEATURE_TOGGLE_RGW": { 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.059 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.060 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_PASSWORD": { 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_PASSWORD", 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.061 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.062 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_URL": { 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_URL", 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.063 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_API_USERNAME": { 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_API_USERNAME", 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.064 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.065 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.066 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.067 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "ISSUE_TRACKER_API_KEY": { 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.068 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "MANAGED_BY_CLUSTERS": { 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "[]", 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.069 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "MULTICLUSTER_CONFIG": { 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "MULTICLUSTER_CONFIG", 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "{}", 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.070 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_HOST": { 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_HOST", 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-05-08T05:40:04.071 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.072 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.073 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.074 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.075 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.076 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.077 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.078 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_ENABLED": { 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_ENABLED", 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.079 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.080 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.081 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8", 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.082 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "REST_REQUESTS_TIMEOUT": { 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "45", 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.083 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ACCESS_KEY": { 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ACCESS_KEY", 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.084 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "admin", 2024-05-08T05:40:04.085 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SECRET_KEY": { 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SECRET_KEY", 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.086 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "RGW_API_SSL_VERIFY": { 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "RGW_API_SSL_VERIFY", 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.087 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "UNSAFE_TLS_v1_2": { 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "UNSAFE_TLS_v1_2", 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.088 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.089 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.090 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.091 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "cross_origin_url": { 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cross_origin_url", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.092 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "crt_file": { 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "crt_file", 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.093 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "debug": { 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "debug", 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Enable/disable debug options", 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.094 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "jwt_token_ttl": { 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "jwt_token_ttl", 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "28800", 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.095 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.096 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.097 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.098 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.099 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.100 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.101 INFO:teuthology.orchestra.run.smithi059.stdout: "motd": { 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "motd", 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "The message of the day", 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.102 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect_resolve_ip_addr": { 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "redirect_resolve_ip_addr", 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-05-08T05:40:04.103 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-05-08T05:40:04.104 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8080", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.105 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.106 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl_server_port": { 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl_server_port", 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.107 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8443", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "redirect", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.108 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "redirect" 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.109 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "url_prefix": { 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url_prefix", 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.110 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.111 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_monitoring": { 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_monitoring", 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "monitor device health metrics", 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.112 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.113 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.114 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.115 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.116 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.117 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "mark_out_threshold": { 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mark_out_threshold", 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "2419200", 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.118 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "pool_name": { 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pool_name", 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "device_health_metrics", 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name of pool in which to store device health metrics", 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.119 INFO:teuthology.orchestra.run.smithi059.stdout: "retention_period": { 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "retention_period", 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15552000", 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long to retain device health metrics", 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_frequency": { 2024-05-08T05:40:04.120 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_frequency", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to scrape device health metrics", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "self_heal": { 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "self_heal", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.121 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.122 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how frequently to wake up and check device health", 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.123 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "warn_threshold": { 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "warn_threshold", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7257600", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.124 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "diskprediction_local", 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.125 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.126 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.127 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.128 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.129 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "predict_interval": { 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predict_interval", 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "86400", 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.130 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "predictor_model": { 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "predictor_model", 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "prophetstor", 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.131 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "600", 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.132 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.133 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "influx", 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": false, 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "influxdb python module not found", 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "batch_size": { 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "batch_size", 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.134 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5000", 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "database": { 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "database", 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.135 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "ceph", 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "hostname": { 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "hostname", 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.136 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server hostname", 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.137 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "30", 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "5", 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.138 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.139 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.140 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.141 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.142 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "password": { 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "password", 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.143 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "password of InfluxDB server user", 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "port": { 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "port", 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "8086", 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.144 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "InfluxDB server port", 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.145 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "ssl": { 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ssl", 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "false", 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.146 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "threads": { 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threads", 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:04.147 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "32", 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "username": { 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "username", 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.148 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "username of InfluxDB server user", 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "verify_ssl": { 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "verify_ssl", 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "true", 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.149 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "insights", 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.150 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.151 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.152 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.153 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.154 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.155 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "iostat", 2024-05-08T05:40:04.156 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.157 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.158 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.159 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.160 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.161 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.162 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "k8sevents", 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_event_retention_days": { 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "ceph_event_retention_days", 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "7", 2024-05-08T05:40:04.163 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Days to hold ceph event information within local cache", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "config_check_secs": { 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "config_check_secs", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "10", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.164 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.165 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.166 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.167 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.168 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.169 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.170 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "localpool", 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "failure_domain": { 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "failure_domain", 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "host", 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.171 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "failure domain for any created local pool", 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.172 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.173 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.174 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.175 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.176 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "min_size": { 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "min_size", 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default min_size for any created local pool", 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.177 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "num_rep": { 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "num_rep", 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3", 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default replica count for any created local pool", 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.178 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_num": { 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_num", 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "128", 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "default pg_num for any created local pool", 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.179 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "prefix": { 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prefix", 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "name prefix for any created local pool", 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.180 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.181 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "subtree": { 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "subtree", 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "rack", 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "CRUSH level for which to create a local pool", 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.182 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mds_autoscaler", 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.183 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.184 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.185 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.186 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.187 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.188 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirroring", 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.189 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.190 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.191 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.192 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.193 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.194 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "nfs", 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.195 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.196 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.197 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.198 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.199 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.200 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.201 INFO:teuthology.orchestra.run.smithi059.stdout: "fail_fs": { 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "fail_fs", 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.202 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.203 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.204 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.205 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.206 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator": { 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "orchestrator", 2024-05-08T05:40:04.207 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "cephadm", 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "rook", 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "test_orchestrator" 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Orchestrator backend", 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.208 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.209 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_perf_query", 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.210 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.211 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.212 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.213 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.214 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.215 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "osd_support", 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.216 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.217 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.218 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.219 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.220 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.221 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "pg_autoscaler", 2024-05-08T05:40:04.222 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.223 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.224 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.225 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.226 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.227 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.228 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.229 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "threshold": { 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "threshold", 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "3.0", 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1.0", 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "scaling threshold", 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.230 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "progress", 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_pg_recovery_event": { 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_pg_recovery_event", 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.231 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow the module to show pg recovery progress", 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.232 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.233 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.234 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.235 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.236 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.237 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "max_completed_events": { 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_completed_events", 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "50", 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.238 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "number of past completed events to remember", 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "sleep_interval": { 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sleep_interval", 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "5", 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.239 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "how long the module is going to sleep", 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.240 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "prometheus", 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.241 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "cache": { 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "cache", 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.242 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "exclude_perf_counters": { 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "exclude_perf_counters", 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.243 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.244 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.245 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.246 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.247 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools": { 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools", 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.248 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_stats_pools_refresh_interval": { 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "300", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.249 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "scrape_interval": { 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "scrape_interval", 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15.0", 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.250 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "::", 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.251 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "9283", 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.252 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "stale_cache_strategy": { 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stale_cache_strategy", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.253 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "log", 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_behaviour": { 2024-05-08T05:40:04.254 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_behaviour", 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "default", 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "default", 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "error" 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.255 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "standby_error_status_code": { 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "standby_error_status_code", 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "500", 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "400", 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "599", 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.256 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.257 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.258 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.259 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.260 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.261 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_snap_create": { 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_snap_create", 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10", 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.262 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "mirror_snapshot_schedule": { 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "mirror_snapshot_schedule", 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.263 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.264 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "trash_purge_schedule": { 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "trash_purge_schedule", 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.265 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "restful", 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.266 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "enable_auth": { 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enable_auth", 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.267 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "key_file": { 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "key_file", 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.268 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.269 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.270 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.271 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.272 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.273 INFO:teuthology.orchestra.run.smithi059.stdout: "server_addr": { 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_addr", 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.274 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "server_port": { 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "server_port", 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.275 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.276 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rgw", 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.277 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.278 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.279 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.280 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.281 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.282 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rook", 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.283 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.284 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.285 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.286 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.287 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.288 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "storage_class": { 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "storage_class", 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.289 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "local", 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "selftest", 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.290 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.291 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.292 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.293 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.294 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.295 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "roption1": { 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption1", 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.296 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "roption2": { 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "roption2", 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "xyz", 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.297 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption1": { 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption1", 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.298 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption2": { 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption2", 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.299 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption3": { 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption3", 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "float", 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.300 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption4": { 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption4", 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.301 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption5": { 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption5", 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.302 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption6": { 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption6", 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.303 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "rwoption7": { 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rwoption7", 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "1", 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "42", 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.304 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "testkey": { 2024-05-08T05:40:04.305 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testkey", 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "testlkey": { 2024-05-08T05:40:04.306 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testlkey", 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "testnewline": { 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "testnewline", 2024-05-08T05:40:04.307 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.308 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "smb", 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.309 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.310 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.311 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.312 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.313 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.314 INFO:teuthology.orchestra.run.smithi059.stdout: "update_orchestration": { 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "update_orchestration", 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.315 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "allow_m_granularity": { 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "allow_m_granularity", 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.316 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "allow minute scheduled snapshots", 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "dump_on_update": { 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "dump_on_update", 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.317 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "dump database to debug log on update", 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.318 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.319 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.320 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.321 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.322 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.323 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "stats", 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.324 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.325 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.326 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.327 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.328 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.329 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "status", 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.330 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.331 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.332 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.333 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.334 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.335 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telegraf", 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "address": { 2024-05-08T05:40:04.336 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "address", 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:04.337 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "15", 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.338 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.339 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.340 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.341 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.342 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.343 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.344 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "telemetry", 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_basic": { 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_basic", 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share basic cluster information (size, version)", 2024-05-08T05:40:04.345 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_crash": { 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_crash", 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.346 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_device": { 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_device", 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.347 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_ident": { 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_ident", 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-05-08T05:40:04.348 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "channel_perf": { 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "channel_perf", 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Share various performance metrics of a cluster", 2024-05-08T05:40:04.349 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "contact": { 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "contact", 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.350 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "description": { 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "description", 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.351 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "device_url": { 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "device_url", 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.352 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "enabled": { 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "enabled", 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.353 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "24", 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "8", 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.354 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "last_opt_revision": { 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "last_opt_revision", 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "1", 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.355 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard": { 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard", 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.356 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "leaderboard_description": { 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "leaderboard_description", 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.357 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.358 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.359 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.360 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.361 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.362 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "organization": { 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "organization", 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.363 INFO:teuthology.orchestra.run.smithi059.stdout: "proxy": { 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "proxy", 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.364 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "url": { 2024-05-08T05:40:04.365 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "url", 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.366 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "test_orchestrator", 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.367 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.368 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.369 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.370 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.371 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.372 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.373 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.374 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.375 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.376 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.377 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "max_concurrent_clones": { 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "max_concurrent_clones", 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "4", 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.378 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Number of asynchronous cloner threads", 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "periodic_async_work": { 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "periodic_async_work", 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.379 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Periodically check for async work", 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_delay": { 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_delay", 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.380 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "snapshot_clone_no_wait": { 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snapshot_clone_no_wait", 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "True", 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.381 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.382 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix", 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "can_run": true, 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "error_string": "", 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "module_options": { 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "discovery_interval": { 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "discovery_interval", 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "uint", 2024-05-08T05:40:04.383 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "100", 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "identifier": { 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "identifier", 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.384 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "interval": { 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "interval", 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "secs", 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.385 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "60", 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "log_level": { 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_level", 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.386 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.387 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster": { 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster", 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.388 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_cluster_level": { 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_cluster_level", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "info", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [ 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "critical", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "debug", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "error", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "info", 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "warning" 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.389 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "log_to_file": { 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "log_to_file", 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "bool", 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "False", 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.390 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "sqlite3_killpoint": { 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "sqlite3_killpoint", 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "dev", 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 1, 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "0", 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.391 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_host": { 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_host", 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "", 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.392 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_port": { 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_port", 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "int", 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "10051", 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "zabbix_sender": { 2024-05-08T05:40:04.393 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "zabbix_sender", 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "str", 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "level": "advanced", 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "flags": 0, 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "min": "", 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "max": "", 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "enum_allowed": [], 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "desc": "", 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "long_desc": "", 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "tags": [], 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: "see_also": [] 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.394 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "services": { 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "dashboard": "https://172.21.15.59:8443/" 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "always_on_modules": { 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "octopus": [ 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.395 INFO:teuthology.orchestra.run.smithi059.stdout: "pacific": [ 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "quincy": [ 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-05-08T05:40:04.396 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "reef": [ 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-05-08T05:40:04.397 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: ], 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "squid": [ 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "balancer", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "crash", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "devicehealth", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "orchestrator", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "pg_autoscaler", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "progress", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "rbd_support", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "status", 2024-05-08T05:40:04.398 INFO:teuthology.orchestra.run.smithi059.stdout: "telemetry", 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: "volumes" 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: "last_failure_osd_epoch": 91, 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: "active_clients": [ 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "devicehealth", 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1735254341 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.399 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "libcephsqlite", 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 792233339 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "rbd_support", 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 3772510012 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.400 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 2445441377 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "snap_schedule", 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.401 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1479761738 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: "name": "volumes", 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: "addrvec": [ 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: { 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: "type": "v2", 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: "addr": "172.21.15.59:0", 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: "nonce": 1529896966 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:04.402 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:04.403 INFO:teuthology.orchestra.run.smithi059.stdout: ] 2024-05-08T05:40:04.403 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:40:04.403 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:03 smithi096 ceph-mon[34253]: pgmap v326: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 462 KiB/s wr, 236 op/s 2024-05-08T05:40:04.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:03 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1739321547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T05:40:04.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:03 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1495915594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T05:40:04.404 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:03 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3591581453' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T05:40:04.404 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:03 smithi143 ceph-mon[34337]: pgmap v326: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 462 KiB/s wr, 236 op/s 2024-05-08T05:40:04.404 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:03 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1739321547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T05:40:04.404 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:03 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1495915594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T05:40:04.404 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:03 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3591581453' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T05:40:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fef940b29e0 msgr2=0x7fef940b4ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fef940b29e0 0x7fef940b4ea0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fefa400abe0 tx=0x7fefa400a2d0 comp rx=0 tx=0).stop 2024-05-08T05:40:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa81045b0 msgr2=0x7fefa8199630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa81045b0 0x7fefa8199630 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fef9800d950 tx=0x7fef9800dd50 comp rx=0 tx=0).stop 2024-05-08T05:40:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 shutdown_connections 2024-05-08T05:40:04.405 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fef940b29e0 0x7fef940b4ea0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fefa810cfd0 0x7fefa819d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fefa81045b0 0x7fefa8199630 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 --2- 172.21.15.59:0/3591581453 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fefa8103c90 0x7fefa81990f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 >> 172.21.15.59:0/3591581453 conn(0x7fefa8078a10 msgr2=0x7fefa80fe9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.021+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 shutdown_connections 2024-05-08T05:40:04.406 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:03.022+0000 7fefb0771640 1 -- 172.21.15.59:0/3591581453 wait complete. 2024-05-08T05:40:04.447 DEBUG:tasks.check_counter:Getting stats from a 2024-05-08T05:40:04.447 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.a perf dump 2024-05-08T05:40:04.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:03 smithi059 ceph-mon[30304]: pgmap v326: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 462 KiB/s wr, 236 op/s 2024-05-08T05:40:04.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:03 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1739321547' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-05-08T05:40:04.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:03 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1495915594' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-05-08T05:40:04.448 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:03 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3591581453' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-05-08T05:40:04.714 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:05 smithi143 ceph-mon[34337]: pgmap v327: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 462 KiB/s wr, 236 op/s 2024-05-08T05:40:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:05 smithi096 ceph-mon[34253]: pgmap v327: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 462 KiB/s wr, 236 op/s 2024-05-08T05:40:05.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:05 smithi059 ceph-mon[30304]: pgmap v327: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 629 KiB/s rd, 462 KiB/s wr, 236 op/s 2024-05-08T05:40:06.563 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1647, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 1647, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 496690, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 522923, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 2, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.141719665, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.046316833, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.037312606, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.044815134, 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:06.564 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1646, 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.053119928, 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000032272 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1004, 2024-05-08T05:40:06.565 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 980, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 2010180, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 380988, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.114324940, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.033717787, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.055836621, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.030666053, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 979, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.036861089, 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000037651 2024-05-08T05:40:06.566 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1831424, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 150688 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 3, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 185, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 588, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 471966, 2024-05-08T05:40:06.567 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 4, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.014794517, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.010056323, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.002403415, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000023719, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 182, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.012545417, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000068930 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.568 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 471648 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 3901, 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.103400377, 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000026506 2024-05-08T05:40:06.569 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 0, 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 0, 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-08T05:40:06.570 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 0, 2024-05-08T05:40:06.571 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 1, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-08T05:40:06.572 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:06.573 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-08T05:40:06.574 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:06.575 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 0, 2024-05-08T05:40:06.576 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 10, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 0, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 2, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 4197187, 2024-05-08T05:40:06.577 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 4196879, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 10 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 3, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 3, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 0, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 1, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 1, 2024-05-08T05:40:06.578 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 0, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 0, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 0, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 0, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 0, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 0, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 50888, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 215416 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 0, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-05-08T05:40:06.579 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 0, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 0, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.580 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.581 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.582 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.583 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.584 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.584 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.584 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-08T05:40:06.584 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.584 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.584 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.585 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:06.586 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.587 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.588 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.589 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.590 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-08T05:40:06.591 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 2, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 5, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 3, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 2, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 0, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 0, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 4526, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:06.592 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:06.593 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:06.594 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 190161, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 52, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-08T05:40:06.595 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 9512, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 182, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 8512, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 13, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:06.596 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 2368, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 21, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 3, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 2368, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2368, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 1.402982449, 2024-05-08T05:40:06.597 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000592475 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2368, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2369 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 2368, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 2368, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 0, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 1538, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 0, 2024-05-08T05:40:06.598 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 828, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 1, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:06.599 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-08T05:40:06.600 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 100, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 12, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 5, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 5, 2024-05-08T05:40:06.601 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 0, 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 2, 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 0 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 0, 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-08T05:40:06.602 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 2650, 2024-05-08T05:40:06.603 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 2251657, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 2650, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2650, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2251657, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:06.604 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 2368, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 2888, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2368, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2888, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.605 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 2368, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 2368, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2368, 2024-05-08T05:40:06.606 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2368, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:06.607 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x5639253306e0": { 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-08T05:40:06.608 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:06.609 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:06.610 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:40:07.335 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:07 smithi059 ceph-mon[30304]: pgmap v328: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 512 KiB/s wr, 261 op/s 2024-05-08T05:40:07.363 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-05-08T05:40:07.363 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-05-08T05:40:07.364 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-05-08T05:40:07.364 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-05-08T05:40:07.364 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T05:40:07.364 INFO:tasks.check_counter:Daemon mds.a mds.dir_split=0 2024-05-08T05:40:07.364 DEBUG:tasks.check_counter:Getting stats from d 2024-05-08T05:40:07.364 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.d perf dump 2024-05-08T05:40:07.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:07 smithi143 ceph-mon[34337]: pgmap v328: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 512 KiB/s wr, 261 op/s 2024-05-08T05:40:07.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:07 smithi096 ceph-mon[34253]: pgmap v328: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 648 KiB/s rd, 512 KiB/s wr, 261 op/s 2024-05-08T05:40:07.620 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:09.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:09 smithi059 ceph-mon[30304]: pgmap v329: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-05-08T05:40:09.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:09 smithi059 ceph-mon[30304]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T05:40:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:09 smithi143 ceph-mon[34337]: pgmap v329: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-05-08T05:40:09.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:09 smithi143 ceph-mon[34337]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T05:40:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:09 smithi096 ceph-mon[34253]: pgmap v329: 129 pgs: 129 active+clean; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 70 KiB/s wr, 32 op/s 2024-05-08T05:40:09.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:09 smithi096 ceph-mon[34253]: osdmap e101: 12 total, 12 up, 12 in 2024-05-08T05:40:09.813 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:40:09.813 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:09.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 0, 2024-05-08T05:40:09.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2, 2024-05-08T05:40:09.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 0, 2024-05-08T05:40:09.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 0, 2024-05-08T05:40:09.813 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 0, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.001444270, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.000266443, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:09.814 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 258, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 247, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1721408, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 152080, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 2, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.053742796, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.014993761, 2024-05-08T05:40:09.815 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.047676202, 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.004797594, 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 246, 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.010950233, 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000044513 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1721408, 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 152080 2024-05-08T05:40:09.816 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 6, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 190, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 3312, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 190832, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 5, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.014518499, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.010270956, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.001759634, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 188, 2024-05-08T05:40:09.817 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.011295961, 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000060084 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 190832 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 0, 2024-05-08T05:40:09.818 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:09.819 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:09.820 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:09.821 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:09.821 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:09.821 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:09.821 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:09.821 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:09.821 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:09.821 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 148714, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 30, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-08T05:40:09.822 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 928, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 2, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-08T05:40:09.823 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 260, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1685007, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 260, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 260, 2024-05-08T05:40:09.824 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1685007, 2024-05-08T05:40:09.825 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:09.825 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:09.825 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:09.825 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:09.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:09.825 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:09.825 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:40:10.573 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-05-08T05:40:10.573 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-05-08T05:40:10.574 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-05-08T05:40:10.574 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-05-08T05:40:10.574 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-05-08T05:40:10.574 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.d 2024-05-08T05:40:10.574 DEBUG:tasks.check_counter:Getting stats from g 2024-05-08T05:40:10.574 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.g perf dump 2024-05-08T05:40:11.086 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:11.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:11 smithi096 ceph-mon[34253]: pgmap v331: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-05-08T05:40:11.558 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:11 smithi059 ceph-mon[30304]: pgmap v331: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-05-08T05:40:11.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:11 smithi143 ceph-mon[34337]: pgmap v331: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-05-08T05:40:13.322 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:40:13.322 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 824, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 826, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 276809, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 262033, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 6, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 2, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.091024060, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.037282278, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.020425010, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.024179471, 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:13.323 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 824, 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.038558020, 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000046793 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 1578, 2024-05-08T05:40:13.324 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 1760, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 978853, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 974017, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 7, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 5, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.113198560, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.036948565, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.028280765, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.038260632, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 1758, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.054720134, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000031126 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:13.325 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 474064 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 288, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 264, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1835796, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 154085, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-05-08T05:40:13.326 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.050074105, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.016170155, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.039458957, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.005003664, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 263, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.013211101, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000050232 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.327 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1834448, 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 153120 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 1, 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-MDSRank": { 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 4027, 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.099757094, 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000024772 2024-05-08T05:40:13.328 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "queue_len": 0, 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "complete_latency": { 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "mds": { 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "request": 0, 2024-05-08T05:40:13.329 INFO:teuthology.orchestra.run.smithi059.stdout: "reply": 0, 2024-05-08T05:40:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "reply_latency": { 2024-05-08T05:40:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_reply": 0, 2024-05-08T05:40:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "forward": 0, 2024-05-08T05:40:13.330 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_commit": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_split": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_merge": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes": 93, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_top": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_bottom": 93, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pin_tail": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_pinned": 93, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_expired": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "inodes_with_caps": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "caps": 0, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "subtrees": 1, 2024-05-08T05:40:13.331 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_hit": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_forward": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_discover": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "traverse_lock": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "load_cent": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "q": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "exported": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "exported_inodes": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "imported": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "imported_inodes": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:13.332 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rfiles": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rbytes": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "root_rsnaps": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_inode_file_caps": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T05:40:13.333 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_grant": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_caps_dirty": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_cap_release": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "process_request_cap_release": 0 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_cache": { 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_update_receipt": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_try_discover": 0, 2024-05-08T05:40:13.334 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_send_discover": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "dir_handle_discover": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_created": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_enqueued": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_reintegrated": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "strays_migrated": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_started": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "recovery_completed": 0, 2024-05-08T05:40:13.335 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_path": 0, 2024-05-08T05:40:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_quiesce_inode": 0, 2024-05-08T05:40:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_exportdir": 0, 2024-05-08T05:40:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_flush": 0, 2024-05-08T05:40:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragmentdir": 0, 2024-05-08T05:40:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "ireq_inodestats": 0 2024-05-08T05:40:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.336 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_log": { 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "evlrg": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "evadd": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "evex": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "evtrm": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "ev": 313, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "evexg": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "evexd": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "segadd": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "segex": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "segtrm": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "seg": 2, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "segmjr": 2, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "segexg": 0, 2024-05-08T05:40:13.337 INFO:teuthology.orchestra.run.smithi059.stdout: "segexd": 0, 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "expos": 4194304, 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "wrpos": 4769901, 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "rdpos": 4767813, 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "jlat": { 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "replayed": 313 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_mem": { 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "ino": 20, 2024-05-08T05:40:13.338 INFO:teuthology.orchestra.run.smithi059.stdout: "ino+": 145, 2024-05-08T05:40:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "ino-": 125, 2024-05-08T05:40:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "dir": 3, 2024-05-08T05:40:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "dir+": 14, 2024-05-08T05:40:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "dir-": 11, 2024-05-08T05:40:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "dn": 93, 2024-05-08T05:40:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "dn+": 217, 2024-05-08T05:40:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "dn-": 124, 2024-05-08T05:40:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "cap": 0, 2024-05-08T05:40:13.339 INFO:teuthology.orchestra.run.smithi059.stdout: "cap+": 0, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "cap-": 0, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "rss": 50928, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "heap": 215416 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_server": { 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_client_request": 0, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "dispatch_server_request": 0, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_request": 0, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_client_session": 0, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "handle_peer_request": 0, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "req_create_latency": { 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.340 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getattr_latency": { 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.341 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "req_link_latency": { 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookup_latency": { 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:13.342 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupino_latency": { 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupname_latency": { 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.343 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "req_lssnap_latency": { 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.344 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mkdir_latency": { 2024-05-08T05:40:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mknod_latency": { 2024-05-08T05:40:13.345 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "req_mksnap_latency": { 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "req_open_latency": { 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.346 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "req_readdir_latency": { 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rename_latency": { 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.347 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmdir_latency": { 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.348 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setattr_latency": { 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.349 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.350 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setlayout_latency": { 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "req_setxattr_latency": { 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.351 INFO:teuthology.orchestra.run.smithi059.stdout: "req_symlink_latency": { 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "req_unlink_latency": { 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:13.352 INFO:teuthology.orchestra.run.smithi059.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_sessions": { 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "session_count": 1, 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "session_add": 3, 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "session_remove": 2, 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_open": 1, 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "sessions_stale": 0, 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "total_load": 0, 2024-05-08T05:40:13.353 INFO:teuthology.orchestra.run.smithi059.stdout: "average_load": 0, 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "avg_session_uptime": 1934, 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:13.354 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:13.355 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:13.356 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 198927, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 75, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:13.357 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 9752, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 188, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 120145, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 929, 2024-05-08T05:40:13.358 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "objecter": { 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_active": 0, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_laggy": 0, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send": 2399, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_send_bytes": 21, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_resend": 3, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_reply": 2399, 2024-05-08T05:40:13.359 INFO:teuthology.orchestra.run.smithi059.stdout: "op_latency": { 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2399, 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2.813964741, 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.001172974 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "op_inflight": 0, 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "oplen_avg": { 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 2399, 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 2400 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.360 INFO:teuthology.orchestra.run.smithi059.stdout: "op": 2399, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "op_r": 2399, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "op_w": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "op_rmw": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "op_pg": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_stat": 1550, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_create": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_read": 847, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_write": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writefull": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_writesame": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_append": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_zero": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_truncate": 0, 2024-05-08T05:40:13.361 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_delete": 0, 2024-05-08T05:40:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_mapext": 0, 2024-05-08T05:40:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_getxattr": 1, 2024-05-08T05:40:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_setxattr": 0, 2024-05-08T05:40:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:13.362 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_call": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_watch": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_notify": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "osdop_other": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_active": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_send": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_resend": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "linger_ping": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_active": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_send": 0, 2024-05-08T05:40:13.363 INFO:teuthology.orchestra.run.smithi059.stdout: "poolop_resend": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_active": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_send": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "poolstat_resend": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_active": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_send": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "statfs_resend": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "command_active": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "command_send": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "command_resend": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "map_epoch": 101, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "map_full": 0, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "map_inc": 13, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_sessions": 6, 2024-05-08T05:40:13.364 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_open": 6, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_session_close": 0, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_laggy": 0, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_wr": 0, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_rd": 2, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_del": 0 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "oft": { 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_objs": 0, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_updates": 0, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "omap_total_removes": 0 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.365 INFO:teuthology.orchestra.run.smithi059.stdout: "osd-slow-ops": { 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "slow_ops_count": 0 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "purge_queue": { 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing": 0, 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executing_high_water": 0, 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed_ops": 0, 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_executed": 0, 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:13.366 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 2686, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 2832172, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 2686, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2686, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2832172, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:13.367 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 2399, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 575602, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2399, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 575602, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.368 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 1024, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 2399, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 2399, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 2399, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 2399, 2024-05-08T05:40:13.369 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-08T05:40:13.370 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-08T05:40:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-08T05:40:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-08T05:40:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-08T05:40:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:13.371 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-write_buf_throttle-0x5561fd6b06e0": { 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 3758096384, 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 0, 2024-05-08T05:40:13.372 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 0, 2024-05-08T05:40:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-08T05:40:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-08T05:40:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 0, 2024-05-08T05:40:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 0, 2024-05-08T05:40:13.373 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:13.374 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:13.374 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:40:13.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:13 smithi096 ceph-mon[34253]: pgmap v332: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-05-08T05:40:13.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:13 smithi059 ceph-mon[30304]: pgmap v332: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-05-08T05:40:13.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:13 smithi143 ceph-mon[34337]: pgmap v332: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-05-08T05:40:14.320 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-05-08T05:40:14.320 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-05-08T05:40:14.320 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-05-08T05:40:14.320 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-05-08T05:40:14.320 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T05:40:14.320 INFO:tasks.check_counter:Daemon mds.g mds.dir_split=0 2024-05-08T05:40:14.320 DEBUG:tasks.check_counter:Getting stats from j 2024-05-08T05:40:14.320 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.j perf dump 2024-05-08T05:40:14.583 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:15.512 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:15 smithi143 ceph-mon[34337]: pgmap v333: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-05-08T05:40:15.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:15 smithi096 ceph-mon[34253]: pgmap v333: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-05-08T05:40:15.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:15 smithi059 ceph-mon[30304]: pgmap v333: 129 pgs: 24 active+clean+snaptrim, 33 active+clean, 72 active+clean+snaptrim_wait; 118 MiB data, 2.8 GiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 60 KiB/s wr, 36 op/s 2024-05-08T05:40:17.217 INFO:teuthology.orchestra.run.smithi059.stdout:{ 2024-05-08T05:40:17.217 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 261, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 250, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 1722224, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 154000, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 4, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 0, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.051462660, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.015575348, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.040584970, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.005006178, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 249, 2024-05-08T05:40:17.218 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.013745495, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000055202 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 1722224, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 154000 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 6, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 193, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 3312, 2024-05-08T05:40:17.219 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 191360, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 5, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 3, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.016533757, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.011676143, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.002278329, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 191, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.010835022, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000056727 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:17.220 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 191360 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_messages": 0, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages": 2, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_bytes": 0, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_bytes": 0, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_created_connections": 2, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_active_connections": 1, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_total_time": 0.000256543, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_send_time": 0.000000000, 2024-05-08T05:40:17.221 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_recv_time": 0.000187428, 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T05:40:17.222 INFO:teuthology.orchestra.run.smithi059.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "cct": { 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "total_workers": 0, 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "unhealthy_workers": 0 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "mempool": { 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:17.223 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:17.224 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:17.225 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_items": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:17.226 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_bytes": 148731, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_anon_items": 31, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_bytes": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_items": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_bytes": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_items": 0, 2024-05-08T05:40:17.227 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "pgmap_items": 0, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_bytes": 928, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "mds_co_items": 2, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_1_items": 0, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "unittest_2_items": 0 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: }, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "val": 0, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "max": 104857600, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "get_started": 0, 2024-05-08T05:40:17.228 INFO:teuthology.orchestra.run.smithi059.stdout: "get": 263, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "get_sum": 1685397, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "get_or_fail_success": 263, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "take": 0, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "take_sum": 0, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "put": 263, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "put_sum": 1685397, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "wait": { 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "avgcount": 0, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "sum": 0.000000000, 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: "avgtime": 0.000000000 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout: } 2024-05-08T05:40:17.229 INFO:teuthology.orchestra.run.smithi059.stdout:} 2024-05-08T05:40:17.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:17 smithi059 ceph-mon[30304]: pgmap v334: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-05-08T05:40:17.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:17 smithi096 ceph-mon[34253]: pgmap v334: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-05-08T05:40:17.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:17 smithi143 ceph-mon[34337]: pgmap v334: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-05-08T05:40:17.909 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-05-08T05:40:17.909 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-05-08T05:40:17.909 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-05-08T05:40:17.909 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-05-08T05:40:17.910 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-05-08T05:40:17.910 WARNING:tasks.check_counter:Counter 'mds.dir_split' not found on daemon mds.j 2024-05-08T05:40:17.910 DEBUG:tasks.check_counter:Getting stats from b 2024-05-08T05:40:17.910 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.b perf dump 2024-05-08T05:40:18.162 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:40:19.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:19 smithi096 ceph-mon[34253]: pgmap v335: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-05-08T05:40:19.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:19 smithi059 ceph-mon[30304]: pgmap v335: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-05-08T05:40:19.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:19 smithi143 ceph-mon[34337]: pgmap v335: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 716 B/s wr, 11 op/s 2024-05-08T05:40:20.158 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-08T05:40:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 29995, 2024-05-08T05:40:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 30175, 2024-05-08T05:40:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 8045318, 2024-05-08T05:40:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 75336953, 2024-05-08T05:40:20.158 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 128, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 3, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 2.004668391, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.667675789, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.549062375, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.665682739, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 30175, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 5.530947200, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000183295 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3349, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000733303, 2024-05-08T05:40:20.159 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000218 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 145264, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 37760 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 78882, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 131504, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 24309955, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 121204764, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 13, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 7, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 4.839360777, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 2.028985775, 2024-05-08T05:40:20.160 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 2.228134352, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.890549529, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 131505, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 11.869028747, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000090255 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 86840, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.022493748, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000259 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 330608, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 274560 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.161 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 25089, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 26034, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 8693770, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 58942685, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 30, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 8, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 1.823870163, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.672233021, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.519229361, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.556826103, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 26028, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.894760356, 2024-05-08T05:40:20.162 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000111217 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5726, 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001043107, 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000182 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1880720, 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 487344 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.163 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 90385, 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 35.542498773, 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000393234 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 9122, 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.762314186, 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000302818 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:20.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 21313, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 21246, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 21246, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 151.820188905, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.007145824 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 67, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 1135, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 29, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-05-08T05:40:20.165 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 11203, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 5516, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 5677, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 10, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 3321, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 167, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 2, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 3, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 4, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 23656, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 23013, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 57, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:20.166 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 2, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 3, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 160, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 10, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 543, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 4, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 234, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:20.167 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 17, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 13756, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 19792, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 7861, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 20103, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 13339, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 488, 2024-05-08T05:40:20.168 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 14738 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 1, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 12, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 60, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 4964, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 4901, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 3, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:20.169 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 8, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 24736, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 3, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 29, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 46801, 2024-05-08T05:40:20.170 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 43722, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 43722, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 3079, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 2081, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 102, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 97, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 97, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 5, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 1, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 3, 2024-05-08T05:40:20.171 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 165371393, 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 172867199, 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 25457, 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 58.739499717, 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002307400 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-05-08T05:40:20.172 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 5959, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 11021, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 5062, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 288, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 326, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 38, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 11203, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 16187, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 4984, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 3, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 9542, 2024-05-08T05:40:20.173 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 9539, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 124932, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 231800 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 40725, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 479, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 21313, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 41, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 974, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5051, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 24.158403248, 2024-05-08T05:40:20.174 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.004782895 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 203, 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 5.542136437, 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.027301164 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3, 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.004425701, 2024-05-08T05:40:20.175 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001475233 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 909, 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.528405192, 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000581303 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-05-08T05:40:20.176 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000251576, 2024-05-08T05:40:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000251576 2024-05-08T05:40:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-05-08T05:40:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:20.177 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.178 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11, 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.002240134, 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000203648 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 72, 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.064902227, 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000901419 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.179 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11, 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.332421220, 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.030220110 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 405, 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.083313049, 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000205711 2024-05-08T05:40:20.180 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 566, 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.178342462, 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002081877 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 79, 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.178655490, 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002261461 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.181 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 66, 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 41.144921626, 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.623407903 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5, 2024-05-08T05:40:20.182 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.034050333, 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.006810066 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 9003, 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 69.862837423, 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.007759950 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:20.183 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 8, 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.002991141, 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000373892 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-05-08T05:40:20.184 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 9, 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.023415836, 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.002601759 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5, 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.003196847, 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000639369 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4839, 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 8.675278963, 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001792783 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.185 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 6, 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 4, 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-05-08T05:40:20.186 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 4850, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 2425, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 632, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 240, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 240, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:20.187 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:20.188 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 961092, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 4992, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-08T05:40:20.189 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 9752, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 188, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 22225177, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 213080, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:20.190 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 84623, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 184427972, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 37, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 84623, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 84623, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 3590.060402545, 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.042424168 2024-05-08T05:40:20.191 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 84623, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 102765 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 84623, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 56, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 84567, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 827, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 10361, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 56, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 25513, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 193, 2024-05-08T05:40:20.192 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 5, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 10434, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 15932, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-05-08T05:40:20.193 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 36715, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-05-08T05:40:20.194 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 101, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 22, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 2504, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 225 2024-05-08T05:40:20.195 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 24981 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-05-08T05:40:20.196 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 34, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 6, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 9820, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 4901, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 133962, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 29599751, 2024-05-08T05:40:20.197 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 133962, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 133962, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 29599751, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:20.198 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 84623, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 185171469, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 84623, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 185171469, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.199 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 84623, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 84623, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 84623, 2024-05-08T05:40:20.200 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 84623, 2024-05-08T05:40:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:20.201 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-08T05:40:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 4901, 2024-05-08T05:40:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 743497, 2024-05-08T05:40:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 4901, 2024-05-08T05:40:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:20.202 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 56, 2024-05-08T05:40:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 743497, 2024-05-08T05:40:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:20.203 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x557b01734500": { 2024-05-08T05:40:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-08T05:40:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 46801, 2024-05-08T05:40:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 168672895, 2024-05-08T05:40:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:20.204 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 46801, 2024-05-08T05:40:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 25457, 2024-05-08T05:40:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 168672895, 2024-05-08T05:40:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:20.205 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:20.206 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:20.206 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-08T05:40:21.271 INFO:tasks.check_counter:Daemon mds.b mds.exported=3 2024-05-08T05:40:21.271 INFO:tasks.check_counter:Daemon mds.b mds.imported=10 2024-05-08T05:40:21.271 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-05-08T05:40:21.271 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-05-08T05:40:21.271 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-05-08T05:40:21.271 INFO:tasks.check_counter:Daemon mds.b mds.dir_split=29 2024-05-08T05:40:21.271 DEBUG:tasks.check_counter:Getting stats from e 2024-05-08T05:40:21.272 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.e perf dump 2024-05-08T05:40:21.518 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:40:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:21 smithi096 ceph-mon[34253]: pgmap v336: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 607 B/s wr, 13 op/s 2024-05-08T05:40:21.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:21 smithi096 ceph-mon[34253]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T05:40:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:21 smithi059 ceph-mon[30304]: pgmap v336: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 607 B/s wr, 13 op/s 2024-05-08T05:40:21.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:21 smithi059 ceph-mon[30304]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T05:40:21.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:21 smithi143 ceph-mon[34337]: pgmap v336: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 607 B/s wr, 13 op/s 2024-05-08T05:40:21.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:21 smithi143 ceph-mon[34337]: osdmap e102: 12 total, 12 up, 12 in 2024-05-08T05:40:23.520 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:23 smithi096 ceph-mon[34253]: pgmap v338: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-08T05:40:23.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:23 smithi059 ceph-mon[30304]: pgmap v338: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-08T05:40:23.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:23 smithi143 ceph-mon[34337]: pgmap v338: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-08T05:40:24.118 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-08T05:40:24.118 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 156, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 158, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 48537, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 669964, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 11, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.016169651, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.004436522, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.005522334, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.003796119, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 156, 2024-05-08T05:40:24.119 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.019035136, 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000122020 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000150, 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000075 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T05:40:24.120 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 401, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 507, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1870287, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 563279, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 14, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 7, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.066574845, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.021159251, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.049656154, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.007385073, 2024-05-08T05:40:24.121 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 504, 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.044313344, 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000087923 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 11, 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000001551, 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000141 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1853744, 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 485424 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.122 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2434, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2368, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 755410, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 1138991, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 9, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 5, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.171571086, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.081621777, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.098222911, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.004659485, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2368, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.131309474, 2024-05-08T05:40:24.123 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000055451 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 3926, 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000886098, 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000225 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1088, 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 157616 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-08T05:40:24.124 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 189, 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.128077238, 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000677657 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:24.125 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 138, 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.008128783, 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000058904 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 328, 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 315, 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 315, 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.352443505, 2024-05-08T05:40:24.126 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001118868 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 13, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 11, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 44, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 0, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 44, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 44, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 126, 2024-05-08T05:40:24.127 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 2, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 734, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 387, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 13, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 1, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 4, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 1, 2024-05-08T05:40:24.128 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 113, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 1, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 15, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 4570, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 133836562, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:24.129 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 73, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 149, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 2, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 147, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 74, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 5, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 180 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.130 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 2, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 1, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 60, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 59, 2024-05-08T05:40:24.131 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 8, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 404, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 1, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:24.132 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 313, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 313, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 2, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-05-08T05:40:24.133 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4769901, 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 68, 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.391679935, 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.005759999 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-05-08T05:40:24.134 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 31, 2024-05-08T05:40:24.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 155, 2024-05-08T05:40:24.135 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 124, 2024-05-08T05:40:24.135 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 14, 2024-05-08T05:40:24.135 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 25, 2024-05-08T05:40:24.135 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 11, 2024-05-08T05:40:24.135 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 44, 2024-05-08T05:40:24.135 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 258, 2024-05-08T05:40:24.135 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 214, 2024-05-08T05:40:24.135 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 119, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 119, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 57376, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 223608 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 501, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 1, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 328, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 16, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 144, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 44, 2024-05-08T05:40:24.136 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.067125279, 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001525574 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.137 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 176, 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.052827333, 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000300155 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.138 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.139 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-05-08T05:40:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.140 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001731650, 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001731650 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.141 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 12, 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.002217234, 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000184769 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6, 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.005078802, 2024-05-08T05:40:24.142 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000846467 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 13, 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.022776196, 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001752015 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.143 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.076273784, 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.076273784 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.144 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.007217720, 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003608860 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.145 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.146 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:24.146 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.146 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.146 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.146 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.146 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-05-08T05:40:24.146 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000613143, 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000613143 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-05-08T05:40:24.147 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 59, 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.116582364, 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001975972 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-05-08T05:40:24.148 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 1, 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 3, 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 2, 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 1, 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 116, 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 116, 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 2158, 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-08T05:40:24.149 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:24.150 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:24.151 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 280810, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 112, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:24.152 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 9752, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 188, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 164788, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 1626, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:24.153 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 259, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 602672, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 33, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 259, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 259, 2024-05-08T05:40:24.154 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 1.406967322, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.005432306 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 259, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 295 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 259, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 4, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 255, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-05-08T05:40:24.155 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 10, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 4, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 72, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 17, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 129, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-05-08T05:40:24.156 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 20, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-05-08T05:40:24.157 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 2, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-05-08T05:40:24.158 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 102, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 14, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 12, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 12, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 32, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 9 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.159 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 412 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 5, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 117, 2024-05-08T05:40:24.160 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 59, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 2987, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 2393388, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 2987, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2987, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2393388, 2024-05-08T05:40:24.161 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 259, 2024-05-08T05:40:24.162 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 611831, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 259, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 611831, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:24.163 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 259, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 259, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 259, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 259, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:24.164 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 59, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 9159, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 59, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 4, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 9159, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.165 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x558577a6a500": { 2024-05-08T05:40:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-08T05:40:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 313, 2024-05-08T05:40:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 575597, 2024-05-08T05:40:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:24.166 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 313, 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 68, 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 575597, 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:24.167 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:24.168 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-08T05:40:24.927 INFO:tasks.check_counter:Daemon mds.e mds.exported=1 2024-05-08T05:40:24.927 INFO:tasks.check_counter:Daemon mds.e mds.imported=1 2024-05-08T05:40:24.927 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=6 2024-05-08T05:40:24.927 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-05-08T05:40:24.927 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T05:40:24.927 INFO:tasks.check_counter:Daemon mds.e mds.dir_split=0 2024-05-08T05:40:24.927 DEBUG:tasks.check_counter:Getting stats from h 2024-05-08T05:40:24.928 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.h perf dump 2024-05-08T05:40:25.165 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:40:25.381 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:25 smithi059 ceph-mon[30304]: pgmap v339: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-08T05:40:25.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:25 smithi096 ceph-mon[34253]: pgmap v339: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-08T05:40:25.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:25 smithi143 ceph-mon[34337]: pgmap v339: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 102 B/s wr, 10 op/s 2024-05-08T05:40:26.906 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 356, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 456, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1872561, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 502636, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 8, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.063687714, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.022735409, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.042831162, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.006727879, 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:26.907 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 455, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.043204346, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000094954 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1864752, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 484592 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 1755, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 1422, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 522919, 2024-05-08T05:40:26.908 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 784485, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 11, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 7, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.137191975, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.065096089, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.077929173, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.006169118, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1419, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.065624262, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000046246 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2865, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000634333, 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000221 2024-05-08T05:40:26.909 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 6496, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 172800 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 88, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 130, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 24351, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 115052, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 16, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 6, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.014238897, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.003846499, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.009164117, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000612610, 2024-05-08T05:40:26.910 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 129, 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.016165770, 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000125316 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 103, 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000037493, 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000364 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 6576, 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 5328 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-05-08T05:40:26.911 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 98, 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.009892029, 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000100939 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 18, 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000702975, 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000039054 2024-05-08T05:40:26.912 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 31, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 18, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 18, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.015125795, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000840321 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 13, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 11, 2024-05-08T05:40:26.913 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 15, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 2, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 13, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 13, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 15, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 1, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 1, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 3, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 72, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 44, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 11, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 3, 2024-05-08T05:40:26.914 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 1, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 8, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 1, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 4, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 4570, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 133836562, 2024-05-08T05:40:26.915 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 6, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 14, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 8, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:26.916 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 14, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 1, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 3, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 10 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 4, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 4, 2024-05-08T05:40:26.917 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 4, 2024-05-08T05:40:26.918 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-05-08T05:40:26.918 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-05-08T05:40:26.918 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:26.918 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:26.918 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:26.918 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-05-08T05:40:26.919 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-05-08T05:40:26.919 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 8, 2024-05-08T05:40:26.919 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 79, 2024-05-08T05:40:26.919 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:26.920 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 1, 2024-05-08T05:40:26.920 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-05-08T05:40:26.920 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-05-08T05:40:26.920 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:26.920 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-05-08T05:40:26.920 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.921 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-05-08T05:40:26.921 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-05-08T05:40:26.921 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 32, 2024-05-08T05:40:26.921 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-05-08T05:40:26.921 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-05-08T05:40:26.921 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 32, 2024-05-08T05:40:26.922 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-05-08T05:40:26.922 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-05-08T05:40:26.922 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 2, 2024-05-08T05:40:26.922 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-05-08T05:40:26.922 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-05-08T05:40:26.922 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-05-08T05:40:26.923 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-05-08T05:40:26.923 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-05-08T05:40:26.923 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-05-08T05:40:26.926 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-05-08T05:40:26.926 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4248436, 2024-05-08T05:40:26.927 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 0, 2024-05-08T05:40:26.927 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-05-08T05:40:26.927 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 23, 2024-05-08T05:40:26.927 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.045841570, 2024-05-08T05:40:26.927 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001993111 2024-05-08T05:40:26.927 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.928 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 0 2024-05-08T05:40:26.928 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.928 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-05-08T05:40:26.928 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 19, 2024-05-08T05:40:26.928 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 32, 2024-05-08T05:40:26.928 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 13, 2024-05-08T05:40:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 18, 2024-05-08T05:40:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 24, 2024-05-08T05:40:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 6, 2024-05-08T05:40:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 15, 2024-05-08T05:40:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 35, 2024-05-08T05:40:26.929 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 20, 2024-05-08T05:40:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 1, 2024-05-08T05:40:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 14, 2024-05-08T05:40:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 13, 2024-05-08T05:40:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 54356, 2024-05-08T05:40:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 223608 2024-05-08T05:40:26.930 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.931 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-05-08T05:40:26.931 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 51, 2024-05-08T05:40:26.931 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 11, 2024-05-08T05:40:26.931 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 31, 2024-05-08T05:40:26.931 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 28, 2024-05-08T05:40:26.931 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 88, 2024-05-08T05:40:26.931 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-05-08T05:40:26.932 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 1, 2024-05-08T05:40:26.932 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.003502410, 2024-05-08T05:40:26.932 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003502410 2024-05-08T05:40:26.932 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.932 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-05-08T05:40:26.932 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 5, 2024-05-08T05:40:26.933 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.002425062, 2024-05-08T05:40:26.933 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000485012 2024-05-08T05:40:26.933 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.933 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:26.933 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.933 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.934 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.934 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-05-08T05:40:26.934 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.934 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.934 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.934 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.934 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-05-08T05:40:26.934 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 6, 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.001564968, 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000260828 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.935 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-05-08T05:40:26.936 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.937 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.937 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.937 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.937 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-05-08T05:40:26.937 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.937 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.937 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.938 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2, 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000737066, 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000368533 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.939 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.940 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.941 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.942 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4, 2024-05-08T05:40:26.943 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.006896289, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.001724072 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 5, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 3, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-05-08T05:40:26.944 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 8, 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 4, 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 622, 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:26.945 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:26.946 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:26.947 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 288613, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 102, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 9512, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 182, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:26.948 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 87909, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 365, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 93, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 67789, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 32, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 93, 2024-05-08T05:40:26.949 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 93, 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.351991811, 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.003784858 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 93, 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 126 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 93, 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 1, 2024-05-08T05:40:26.950 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 92, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 10, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 1, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 24, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 15, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 20, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:26.951 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 20, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-05-08T05:40:26.952 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 102, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 14, 2024-05-08T05:40:26.953 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 11, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 11, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 29, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 0, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 7 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 1, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-05-08T05:40:26.954 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 97 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 8, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 4, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 8, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 4, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 2195, 2024-05-08T05:40:26.955 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 2201658, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 2195, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2195, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2201658, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:26.956 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 93, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 68417, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 93, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 68417, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:26.957 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 93, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 93, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 93, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 93, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.958 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 4, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 628, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 4, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 1, 2024-05-08T05:40:26.959 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 628, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x560aa21e0500": { 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 32, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 54132, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 32, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:26.960 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:26.961 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 23, 2024-05-08T05:40:26.961 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 54132, 2024-05-08T05:40:26.961 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:26.961 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:26.961 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:26.961 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:26.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:26.961 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:26.961 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-08T05:40:27.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:27 smithi096 ceph-mon[34253]: pgmap v340: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T05:40:27.587 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:27 smithi059 ceph-mon[30304]: pgmap v340: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T05:40:27.712 INFO:tasks.check_counter:Daemon mds.h mds.exported=1 2024-05-08T05:40:27.712 INFO:tasks.check_counter:Daemon mds.h mds.imported=1 2024-05-08T05:40:27.712 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-05-08T05:40:27.713 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-05-08T05:40:27.713 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T05:40:27.713 INFO:tasks.check_counter:Daemon mds.h mds.dir_split=0 2024-05-08T05:40:27.713 DEBUG:tasks.check_counter:Getting stats from k 2024-05-08T05:40:27.713 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.k perf dump 2024-05-08T05:40:27.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:27 smithi143 ceph-mon[34337]: pgmap v340: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T05:40:27.948 INFO:teuthology.orchestra.run.smithi096.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.b/config 2024-05-08T05:40:29.449 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:29 smithi096 ceph-mon[34253]: pgmap v341: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T05:40:29.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:29 smithi059 ceph-mon[30304]: pgmap v341: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T05:40:29.730 INFO:teuthology.orchestra.run.smithi096.stdout:{ 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 2443, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 2445, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 744773, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 776881, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.171117159, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.056264436, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.040604416, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.060698023, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2443, 2024-05-08T05:40:29.731 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.065859083, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000026958 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 303, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 277, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 1841564, 2024-05-08T05:40:29.732 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 161002, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 3, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 2, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.053845396, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.015876931, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.043110600, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.006261230, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 277, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.014283137, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000051563 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.733 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 1841008, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 160304 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_messages": 3, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages": 195, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_bytes": 588, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_bytes": 480190, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_created_connections": 6, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_active_connections": 4, 2024-05-08T05:40:29.734 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_total_time": 0.011240793, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_send_time": 0.007218640, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_recv_time": 0.001896830, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_running_fast_dispatch_time": 0.000019719, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 192, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.012883226, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000067100 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.735 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "msgr_send_encrypted_bytes": 479872 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "cct": { 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "total_workers": 1, 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "unhealthy_workers": 0 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-MDSRank": { 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 4115, 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.095621636, 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000023237 2024-05-08T05:40:29.736 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "queue_len": 0, 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "complete_latency": { 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "mds": { 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "request": 0, 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "reply": 0, 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "reply_latency": { 2024-05-08T05:40:29.737 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_reply": 0, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "forward": 0, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_commit": 0, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_split": 0, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_merge": 0, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes": 5, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_top": 0, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_bottom": 5, 2024-05-08T05:40:29.738 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pin_tail": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_pinned": 5, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_expired": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "inodes_with_caps": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "caps": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "subtrees": 1, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_hit": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_forward": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_discover": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "traverse_lock": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "load_cent": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "q": 0, 2024-05-08T05:40:29.739 INFO:teuthology.orchestra.run.smithi096.stdout: "exported": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "exported_inodes": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "imported": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "imported_inodes": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rfiles": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rbytes": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "root_rsnaps": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:29.740 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_inode_file_caps": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_grant": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_caps_dirty": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_cap_release": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "process_request_cap_release": 0 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_cache": { 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_update_receipt": 0, 2024-05-08T05:40:29.741 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_try_discover": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_send_discover": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "dir_handle_discover": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_created": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_enqueued": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_reintegrated": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "strays_migrated": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_started": 0, 2024-05-08T05:40:29.742 INFO:teuthology.orchestra.run.smithi096.stdout: "recovery_completed": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_path": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_quiesce_inode": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_exportdir": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_flush": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragmentdir": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "ireq_inodestats": 0 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_log": { 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "evlrg": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "evadd": 0, 2024-05-08T05:40:29.743 INFO:teuthology.orchestra.run.smithi096.stdout: "evex": 0, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "evtrm": 0, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "ev": 32, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "evexg": 0, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "evexd": 0, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "segadd": 0, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "segex": 0, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "segtrm": 0, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "seg": 2, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "segmjr": 2, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "segexg": 0, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "segexd": 0, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "expos": 4194304, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "wrpos": 4248436, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "rdpos": 4245831, 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "jlat": { 2024-05-08T05:40:29.744 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "replayed": 32 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_mem": { 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "ino": 4, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "ino+": 23, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "ino-": 19, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "dir": 2, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "dir+": 14, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "dir-": 12, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "dn": 5, 2024-05-08T05:40:29.745 INFO:teuthology.orchestra.run.smithi096.stdout: "dn+": 25, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "dn-": 20, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "cap": 0, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "cap+": 0, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "cap-": 0, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "rss": 53176, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "heap": 215416 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_server": { 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_client_request": 0, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "dispatch_server_request": 0, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_request": 0, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_client_session": 0, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "handle_peer_request": 0, 2024-05-08T05:40:29.746 INFO:teuthology.orchestra.run.smithi096.stdout: "req_create_latency": { 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getattr_latency": { 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.747 INFO:teuthology.orchestra.run.smithi096.stdout: "req_link_latency": { 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookup_latency": { 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.748 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupino_latency": { 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupname_latency": { 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.749 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "req_lssnap_latency": { 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mkdir_latency": { 2024-05-08T05:40:29.750 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.751 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:29 smithi143 ceph-mon[34337]: pgmap v341: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T05:40:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.751 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mknod_latency": { 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "req_mksnap_latency": { 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "req_open_latency": { 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.752 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "req_readdir_latency": { 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rename_latency": { 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.753 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmdir_latency": { 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.754 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setattr_latency": { 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.755 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setlayout_latency": { 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "req_setxattr_latency": { 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.756 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "req_symlink_latency": { 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "req_unlink_latency": { 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:29.757 INFO:teuthology.orchestra.run.smithi096.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_sessions": { 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "session_count": 2, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "session_add": 5, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "session_remove": 3, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_open": 2, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "sessions_stale": 0, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "total_load": 0, 2024-05-08T05:40:29.758 INFO:teuthology.orchestra.run.smithi096.stdout: "average_load": 0, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "avg_session_uptime": 5192, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "mempool": { 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:29.759 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:29.760 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_items": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_bytes": 182081, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_anon_items": 48, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:29.761 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_bytes": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_items": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_bytes": 9512, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_items": 182, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "pgmap_items": 0, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_bytes": 17248, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "mds_co_items": 52, 2024-05-08T05:40:29.762 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_1_items": 0, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "unittest_2_items": 0 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "objecter": { 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "op_active": 0, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "op_laggy": 0, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send": 2446, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "op_send_bytes": 21, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "op_resend": 3, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "op_reply": 2446, 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "op_latency": { 2024-05-08T05:40:29.763 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2446, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2.415943755, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000987712 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "op_inflight": 0, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "oplen_avg": { 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 2446, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 2447 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "op": 2446, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "op_r": 2446, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "op_w": 0, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "op_rmw": 0, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "op_pg": 0, 2024-05-08T05:40:29.764 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_stat": 1584, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_create": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_read": 860, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_write": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writefull": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_writesame": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_append": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_zero": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_truncate": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_delete": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_mapext": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_getxattr": 1, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_setxattr": 0, 2024-05-08T05:40:29.765 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_call": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_watch": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_notify": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "osdop_other": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_active": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_send": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_resend": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "linger_ping": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_active": 0, 2024-05-08T05:40:29.766 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_send": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "poolop_resend": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_active": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_send": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "poolstat_resend": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_active": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_send": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "statfs_resend": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "command_active": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "command_send": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "command_resend": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "map_epoch": 102, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "map_full": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "map_inc": 14, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_sessions": 4, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_open": 4, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_session_close": 0, 2024-05-08T05:40:29.767 INFO:teuthology.orchestra.run.smithi096.stdout: "osd_laggy": 0, 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_wr": 0, 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_rd": 2, 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_del": 0 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "oft": { 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_objs": 0, 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_updates": 0, 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "omap_total_removes": 0 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "osd-slow-ops": { 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "slow_ops_count": 0 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "purge_queue": { 2024-05-08T05:40:29.768 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executing_high_water": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed_ops": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_executed": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 2745, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 2321711, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 2745, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:29.769 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2745, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2321711, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 104857600, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:29.770 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 2446, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 54137, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2446, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 54137, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 1024, 2024-05-08T05:40:29.771 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 2446, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 2446, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 2446, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 2446, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:29.772 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:29.773 INFO:teuthology.orchestra.run.smithi096.stdout: }, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "throttle-write_buf_throttle-0x55a880c966e0": { 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "val": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "max": 3758096384, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "get_started": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "get": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "get_sum": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "take": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "take_sum": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "put": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "put_sum": 0, 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "wait": { 2024-05-08T05:40:29.774 INFO:teuthology.orchestra.run.smithi096.stdout: "avgcount": 0, 2024-05-08T05:40:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "sum": 0.000000000, 2024-05-08T05:40:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: "avgtime": 0.000000000 2024-05-08T05:40:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:29.775 INFO:teuthology.orchestra.run.smithi096.stdout: } 2024-05-08T05:40:29.775 INFO:teuthology.orchestra.run.smithi096.stdout:} 2024-05-08T05:40:30.576 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-05-08T05:40:30.576 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-05-08T05:40:30.576 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=0 2024-05-08T05:40:30.576 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-05-08T05:40:30.576 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T05:40:30.576 INFO:tasks.check_counter:Daemon mds.k mds.dir_split=0 2024-05-08T05:40:30.576 DEBUG:tasks.check_counter:Getting stats from c 2024-05-08T05:40:30.576 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.c perf dump 2024-05-08T05:40:30.830 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:40:31.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:31 smithi143 ceph-mon[34337]: pgmap v342: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T05:40:31.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:31 smithi096 ceph-mon[34253]: pgmap v342: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T05:40:31.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:31 smithi059 ceph-mon[30304]: pgmap v342: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-05-08T05:40:32.651 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-08T05:40:32.651 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:32.651 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1818, 2024-05-08T05:40:32.651 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1396, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 2326301, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 648834, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 8, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.153511086, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.055674346, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.105420565, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.012071751, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1399, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.059329806, 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000042408 2024-05-08T05:40:32.652 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2421, 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000595041, 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000245 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1857824, 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 168608 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 55, 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 261, 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 12221, 2024-05-08T05:40:32.653 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 570156, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 18, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 8, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.020947330, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.009599530, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.008304970, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000659784, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 254, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.028223359, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000111115 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 52, 2024-05-08T05:40:32.654 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000023382, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000449 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 2176, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 482912 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 25, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 27, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 8441, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 9443, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 5, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.004058549, 2024-05-08T05:40:32.655 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.000496102, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.002733277, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.000740860, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 26, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.013170181, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000506545 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 9, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000001631, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000181 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3920, 2024-05-08T05:40:32.656 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 1664 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 50, 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.002451889, 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000049037 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-08T05:40:32.657 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2, 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000009390, 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000004695 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 13, 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 0, 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-05-08T05:40:32.658 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 13, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 11, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 15, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 0, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 15, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 0, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 13, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 6, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 0, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 0, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 3, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 31, 2024-05-08T05:40:32.659 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 17, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 13, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 1, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 4570, 2024-05-08T05:40:32.660 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 133836562, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 6, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:32.661 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 0, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 0, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 0, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 0 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 3, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 2, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 3, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:32.662 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_path": 8, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_inode": 50, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-05-08T05:40:32.663 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 10, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 10, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 2, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 2, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 2, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-05-08T05:40:32.664 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 4194304, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 4197187, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 0, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 9, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.014343318, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.001593702 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 0 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 19, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 22, 2024-05-08T05:40:32.665 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 3, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 18, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 22, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 4, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 15, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 21, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 6, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 0, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 0, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 0, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 52528, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 223608 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 13, 2024-05-08T05:40:32.666 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 0, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 13, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 27, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 50, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:32.667 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.668 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.669 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.670 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.671 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-05-08T05:40:32.671 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.671 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.671 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.671 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.671 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-05-08T05:40:32.671 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.671 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.672 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.673 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.674 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.675 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.676 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:32.677 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 2, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 5, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 3, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 2, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 0, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 0, 2024-05-08T05:40:32.678 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 491, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:32.679 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:32.680 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 239557, 2024-05-08T05:40:32.681 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 82, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 9512, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 182, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:32.682 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 86061, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 327, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 48, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 13455, 2024-05-08T05:40:32.683 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 33, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 48, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 48, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.219994250, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.004583213 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 48, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 70 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 48, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 0, 2024-05-08T05:40:32.684 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 48, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 0, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 10, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 0, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 9, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 10, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 6, 2024-05-08T05:40:32.685 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 20, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:32.686 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-05-08T05:40:32.687 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 102, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 14, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 9, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 9, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 14, 2024-05-08T05:40:32.688 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 0, 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 1 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 1, 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 58 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T05:40:32.689 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 0, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 0, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 0, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 1894, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 2156343, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 1894, 2024-05-08T05:40:32.690 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 1894, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2156343, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:32.691 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 48, 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 13455, 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 48, 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 13455, 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:32.692 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 48, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 48, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 48, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 48, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.693 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:32.694 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x56301eaac500": { 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 10, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 2883, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:32.695 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 10, 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 9, 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2883, 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:32.696 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-08T05:40:33.472 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:33 smithi143 ceph-mon[34337]: pgmap v343: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-08T05:40:33.500 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-05-08T05:40:33.500 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-05-08T05:40:33.500 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-05-08T05:40:33.500 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-05-08T05:40:33.500 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T05:40:33.500 INFO:tasks.check_counter:Daemon mds.c mds.dir_split=0 2024-05-08T05:40:33.500 DEBUG:tasks.check_counter:Getting stats from f 2024-05-08T05:40:33.501 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.f perf dump 2024-05-08T05:40:33.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:33 smithi096 ceph-mon[34253]: pgmap v343: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-08T05:40:33.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:33 smithi059 ceph-mon[30304]: pgmap v343: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.4 KiB/s rd, 8 op/s 2024-05-08T05:40:33.738 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:40:35.426 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:35 smithi143 ceph-mon[34337]: pgmap v344: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:35.450 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-08T05:40:35.450 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:35.450 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 533, 2024-05-08T05:40:35.450 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 529, 2024-05-08T05:40:35.450 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 33746731, 2024-05-08T05:40:35.450 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 184740, 2024-05-08T05:40:35.450 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-05-08T05:40:35.450 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.071473024, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.016115390, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.036421965, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.014963848, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 528, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.023229532, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000043995 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.451 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 215984, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 42272 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 299, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 346, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 26147604, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 266598, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 6, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 5, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.041888711, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.010553577, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.020994038, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.007867129, 2024-05-08T05:40:35.452 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 345, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.016649547, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000048259 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 3152, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 173040 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 133, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 135, 2024-05-08T05:40:35.453 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 12619585, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 42301, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 6, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 5, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.017657724, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.003376347, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.009277797, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.003918644, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 133, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.007200114, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000054136 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:35.454 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1444, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.082581327, 2024-05-08T05:40:35.455 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000057189 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 0, 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 0, 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.456 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 0, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 0, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 13832, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 6459, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 7373, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 0, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 4605, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 0, 2024-05-08T05:40:35.457 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 2, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 0, 2024-05-08T05:40:35.458 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 4, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 234, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 6, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T05:40:35.459 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 0 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-05-08T05:40:35.460 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_path": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_inode": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:35.461 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 17034, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-05-08T05:40:35.462 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 5, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 15, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 165371393, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 172867199, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 172867063, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 17034 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-05-08T05:40:35.463 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 4785, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 9149, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 4364, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 276, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 276, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 0, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 13832, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 13832, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 0, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 0, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 0, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 0, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 76404, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 215392 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-05-08T05:40:35.464 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 0, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 0, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 0, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 0, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 0, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:35.465 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.466 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.467 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-05-08T05:40:35.468 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.469 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.469 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.469 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.469 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-05-08T05:40:35.469 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.469 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-05-08T05:40:35.470 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:35.471 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.472 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.473 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-05-08T05:40:35.474 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:35.475 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 2, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 2, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 0, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 0, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 0, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 0, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 2171, 2024-05-08T05:40:35.476 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:35.477 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:35.478 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 409666, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 4934, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:35.479 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 9512, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 182, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 21281804, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 177384, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.480 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 875, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 26, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 4, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 875, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 875, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.774979799, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000885691 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 875, 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 876 2024-05-08T05:40:35.481 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 875, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 875, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 489, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 383, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 0, 2024-05-08T05:40:35.482 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 2, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-05-08T05:40:35.483 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-05-08T05:40:35.484 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 102, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 8, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 10, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 10, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 0, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 2, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 0 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 0, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-05-08T05:40:35.485 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 0 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 0, 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 0, 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 0, 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:35.486 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 961, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 72420127, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 961, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 961, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 72420127, 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:35.487 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 875, 2024-05-08T05:40:35.488 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 72047271, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 875, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 72047271, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:35.489 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 875, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 875, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 875, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 875, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:35.490 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:35.491 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x55ddde7126e0": { 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:35.492 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:35.493 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:35.493 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-08T05:40:35.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:35 smithi059 ceph-mon[30304]: pgmap v344: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:35.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:35 smithi096 ceph-mon[34253]: pgmap v344: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:36.332 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-05-08T05:40:36.332 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-05-08T05:40:36.332 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=6 2024-05-08T05:40:36.332 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-05-08T05:40:36.332 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T05:40:36.332 INFO:tasks.check_counter:Daemon mds.f mds.dir_split=0 2024-05-08T05:40:36.333 DEBUG:tasks.check_counter:Getting stats from i 2024-05-08T05:40:36.333 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.i perf dump 2024-05-08T05:40:36.569 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:40:37.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:37 smithi059 ceph-mon[30304]: pgmap v345: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T05:40:37.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:37 smithi143 ceph-mon[34337]: pgmap v345: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T05:40:37.782 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:37 smithi096 ceph-mon[34253]: pgmap v345: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T05:40:38.361 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-08T05:40:38.361 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 2138, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1777, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 769341, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 2010006, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 9, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 4, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.152098084, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.059164721, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.096296740, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.010560598, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1775, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.137041588, 2024-05-08T05:40:38.362 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000077206 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 3372, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000662058, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000196 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 0, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 0 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 177, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 366, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 47678, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 645708, 2024-05-08T05:40:38.363 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 11, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 7, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.028022784, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.011345690, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.007916463, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.005250998, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 364, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.036253864, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000099598 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 9, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000001490, 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000165 2024-05-08T05:40:38.364 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 4240, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 484240 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1156, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1585, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 2141285, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 1003918, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 14, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 5, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.113348605, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.041161874, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.083357146, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.008443694, 2024-05-08T05:40:38.365 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1584, 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.160471043, 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000101307 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1075, 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000292051, 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000271 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1862080, 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 163488 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.366 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 145, 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.246385159, 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.001699207 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-08T05:40:38.367 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 279, 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.028061946, 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000100580 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 309, 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 300, 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 300, 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.720248321, 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.002400827 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-05-08T05:40:38.368 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 9, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 20, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 1, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 15, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 0, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 1, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 14, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 14, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 430, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 2, 2024-05-08T05:40:38.369 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 2, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 3, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 613, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 456, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 8, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 2, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 8, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 422, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 1, 2024-05-08T05:40:38.370 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 141, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 4570, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 133836562, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 6, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:38.371 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 151, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 150, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 434, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 281, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 3, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 299 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-05-08T05:40:38.372 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 3, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 141, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 141, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:38.373 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_path": 8, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_inode": 916, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 8, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 1, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-05-08T05:40:38.374 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 635, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 635, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 2, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 2, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 2, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-05-08T05:40:38.375 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 4194304, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 5357093, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 0, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 29, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.377518025, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.013017862 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 0 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 19, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 447, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 428, 2024-05-08T05:40:38.376 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 18, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 33, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 15, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 15, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 587, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 572, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 2, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 285, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 283, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 61064, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 223608 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 456, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 1, 2024-05-08T05:40:38.377 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 309, 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 17, 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 223, 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1, 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.009021216, 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.009021216 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1, 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000705946, 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000705946 2024-05-08T05:40:38.378 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 140, 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.090511146, 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000646508 2024-05-08T05:40:38.379 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1, 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000197137, 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000197137 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.380 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.381 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.382 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 16, 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.016211641, 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.001013227 2024-05-08T05:40:38.383 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.384 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.385 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.386 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.387 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 141, 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.603601235, 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.004280859 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.388 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 1, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 3, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 2, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 1, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 287, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 287, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 666, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:38.389 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:38.390 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:38.391 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 297045, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 103, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 9512, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 182, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:38.392 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 79333, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 336, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 388, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 1539233, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 32, 2024-05-08T05:40:38.393 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 388, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 388, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 4.325887195, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.011149193 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 388, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 437 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 388, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 1, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 387, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-05-08T05:40:38.394 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 0, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 10, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 1, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 30, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 14, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 1, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 294, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 0, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 20, 2024-05-08T05:40:38.395 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-05-08T05:40:38.396 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 102, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 14, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 12, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 12, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-05-08T05:40:38.397 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 58, 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 0, 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 9 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 1, 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 924 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.398 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 10, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 5, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 282, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 141, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 3467, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 2638394, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 3467, 2024-05-08T05:40:38.399 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 3467, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2638394, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:38.400 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 388, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 1561370, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 388, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 1561370, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:38.401 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 388, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 388, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 388, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 388, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.402 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 141, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 22137, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 141, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 1, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 22137, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.403 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x564cb7efa500": { 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 635, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 1162789, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 635, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 29, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 1162789, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:38.404 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:38.405 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:38.405 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:38.405 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-08T05:40:39.183 INFO:tasks.check_counter:Daemon mds.i mds.exported=8 2024-05-08T05:40:39.183 INFO:tasks.check_counter:Daemon mds.i mds.imported=1 2024-05-08T05:40:39.183 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-05-08T05:40:39.184 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-05-08T05:40:39.184 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T05:40:39.184 INFO:tasks.check_counter:Daemon mds.i mds.dir_split=1 2024-05-08T05:40:39.184 DEBUG:tasks.check_counter:Getting stats from l 2024-05-08T05:40:39.184 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph daemon mds.l perf dump 2024-05-08T05:40:39.421 INFO:teuthology.orchestra.run.smithi143.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.c/config 2024-05-08T05:40:39.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:39 smithi143 ceph-mon[34337]: pgmap v346: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:39.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:39 smithi059 ceph-mon[30304]: pgmap v346: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:39.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:39 smithi096 ceph-mon[34253]: pgmap v346: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:41.317 INFO:teuthology.orchestra.run.smithi143.stdout:{ 2024-05-08T05:40:41.317 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-0": { 2024-05-08T05:40:41.317 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 302, 2024-05-08T05:40:41.317 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 278, 2024-05-08T05:40:41.317 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 1839938, 2024-05-08T05:40:41.317 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 162740, 2024-05-08T05:40:41.317 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 5, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 1, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.054287501, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.016094839, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.042584531, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.006463485, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 277, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.013070359, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000047185 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.318 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 1839104, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 162416 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-1": { 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 760, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 956, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 186810, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 726473, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 7, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 5, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.082186178, 2024-05-08T05:40:41.319 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.027351340, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.017992757, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.028719798, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 953, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.041642555, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000043696 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 320, 2024-05-08T05:40:41.320 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 485424 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "AsyncMessenger::Worker-2": { 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_messages": 1730, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages": 1730, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_bytes": 1680788, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_bytes": 549374, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_created_connections": 4, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_active_connections": 3, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_total_time": 0.162869531, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_send_time": 0.051527346, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_recv_time": 0.043744568, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_running_fast_dispatch_time": 0.053237080, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_messages_queue_lat": { 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 1729, 2024-05-08T05:40:41.321 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.066985988, 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000038742 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_handle_ack_lat": { 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "msgr_send_encrypted_bytes": 448 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "cct": { 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "total_workers": 1, 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: "unhealthy_workers": 0 2024-05-08T05:40:41.322 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-MDSRank": { 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 4118, 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.118857275, 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000028862 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "finisher-PurgeQueue": { 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "queue_len": 0, 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "complete_latency": { 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.323 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "mds": { 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "request": 0, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "reply": 0, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "reply_latency": { 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_reply": 0, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "forward": 0, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_complete": 0, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_fetch_keys": 0, 2024-05-08T05:40:41.324 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_commit": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_split": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_merge": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes": 142, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_top": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_bottom": 142, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pin_tail": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_pinned": 142, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_expired": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "inodes_with_caps": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "caps": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "subtrees": 1, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_hit": 0, 2024-05-08T05:40:41.325 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_forward": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_discover": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_dir_fetch": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_remote_ino": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "traverse_lock": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "load_cent": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "q": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "exported": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "exported_inodes": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "imported": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "imported_inodes": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_dir_fetch": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_backtrace_fetch": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "openino_peer_discover": 0, 2024-05-08T05:40:41.326 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rfiles": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rbytes": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "root_rsnaps": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_fetch": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_set_tag": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_backtrace_repaired": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_inotable_repaired": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_inodes": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dir_base_inodes": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_dirfrag_rstats": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "scrub_file_inodes": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_inode_file_caps": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_revoke": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_grant": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_trunc": 0, 2024-05-08T05:40:41.327 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "ceph_cap_op_flush_ack": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_caps_dirty": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_cap_release": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "process_request_cap_release": 0 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_cache": { 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_update_receipt": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_try_discover": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_send_discover": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "dir_handle_discover": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays": 0, 2024-05-08T05:40:41.328 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_delayed": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "num_strays_enqueuing": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_created": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_enqueued": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_reintegrated": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "strays_migrated": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_processing": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_enqueued": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "num_recovering_prioritized": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_started": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "recovery_completed": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_path": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_quiesce_inode": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_enqueue_scrub": 0, 2024-05-08T05:40:41.329 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_exportdir": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_flush": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragmentdir": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_fragstats": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "ireq_inodestats": 0 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_log": { 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "evlrg": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "evadd": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "evex": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "evtrm": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "ev": 635, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "evexg": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "evexd": 0, 2024-05-08T05:40:41.330 INFO:teuthology.orchestra.run.smithi143.stdout: "segadd": 0, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "segex": 0, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "segtrm": 0, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "seg": 2, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "segmjr": 2, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "segexg": 0, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "segexd": 0, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "expos": 4194304, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "wrpos": 5357093, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "rdpos": 5354488, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "jlat": { 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.331 INFO:teuthology.orchestra.run.smithi143.stdout: "replayed": 635 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_mem": { 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "ino": 4, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "ino+": 437, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "ino-": 433, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "dir": 2, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "dir+": 22, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "dir-": 20, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "dn": 142, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "dn+": 576, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "dn-": 434, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "cap": 0, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "cap+": 0, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "cap-": 0, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "rss": 53144, 2024-05-08T05:40:41.332 INFO:teuthology.orchestra.run.smithi143.stdout: "heap": 215416 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_server": { 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_client_request": 0, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "dispatch_server_request": 0, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_request": 0, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_client_session": 0, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "handle_peer_request": 0, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "req_create_latency": { 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getattr_latency": { 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.333 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getfilelock_latency": { 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "req_link_latency": { 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookup_latency": { 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.334 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookuphash_latency": { 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupino_latency": { 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupname_latency": { 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.335 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupparent_latency": { 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lookupsnap_latency": { 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "req_lssnap_latency": { 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.336 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mkdir_latency": { 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mknod_latency": { 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.337 INFO:teuthology.orchestra.run.smithi143.stdout: "req_mksnap_latency": { 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "req_open_latency": { 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "req_readdir_latency": { 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.338 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rename_latency": { 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "req_renamesnap_latency": { 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "req_snapdiff_latency": { 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.339 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmdir_latency": { 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmsnap_latency": { 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "req_rmxattr_latency": { 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setattr_latency": { 2024-05-08T05:40:41.340 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setdirlayout_latency": { 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setfilelock_latency": { 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setlayout_latency": { 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.341 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "req_setxattr_latency": { 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "req_symlink_latency": { 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "req_unlink_latency": { 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.342 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_revoke_eviction": 0, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "cap_acquisition_throttle": 0, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "req_getvxattr_latency": { 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_sessions": { 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "session_count": 1, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "session_add": 3, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "session_remove": 2, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_open": 2, 2024-05-08T05:40:41.343 INFO:teuthology.orchestra.run.smithi143.stdout: "sessions_stale": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "total_load": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "average_load": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "avg_session_uptime": 3691, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "mdthresh_evicted": 0 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "mempool": { 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_bytes": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bloom_filter_items": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_bytes": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_alloc_items": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_bytes": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_data_items": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_bytes": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_onode_items": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_bytes": 0, 2024-05-08T05:40:41.344 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_meta_items": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_bytes": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_other_items": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_bytes": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_cache_buffer_items": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_bytes": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_extent_items": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_bytes": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_blob_items": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_bytes": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_shared_blob_items": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_bytes": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_inline_bl_items": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_bytes": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_fsck_items": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_bytes": 0, 2024-05-08T05:40:41.345 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_txc_items": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_bytes": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_deferred_items": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_bytes": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluestore_writing_items": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_bytes": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_items": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_bytes": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_reader_items": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_bytes": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "bluefs_file_writer_items": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_bytes": 190195, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_anon_items": 54, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_bytes": 0, 2024-05-08T05:40:41.346 INFO:teuthology.orchestra.run.smithi143.stdout: "buffer_meta_items": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_bytes": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_items": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_bytes": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_mapbl_items": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_bytes": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_pglog_items": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_bytes": 9512, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_items": 182, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_bytes": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "osdmap_mapping_items": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_bytes": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "pgmap_items": 0, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_bytes": 113696, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "mds_co_items": 600, 2024-05-08T05:40:41.347 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_bytes": 0, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_1_items": 0, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_bytes": 0, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "unittest_2_items": 0 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "objecter": { 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "op_active": 0, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "op_laggy": 0, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send": 2485, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "op_send_bytes": 21, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "op_resend": 3, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "op_reply": 2485, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "op_latency": { 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2485, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 1.771953750, 2024-05-08T05:40:41.348 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000713059 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "op_inflight": 0, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "oplen_avg": { 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 2485, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 2486 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "op": 2485, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "op_r": 2485, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "op_w": 0, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "op_rmw": 0, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "op_pg": 0, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_stat": 1608, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_create": 0, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_read": 875, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_write": 0, 2024-05-08T05:40:41.349 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writefull": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_writesame": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_append": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_zero": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_truncate": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_delete": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_mapext": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_sparse_read": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_clonerange": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_getxattr": 1, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_setxattr": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_cmpxattr": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_rmxattr": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_resetxattrs": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_call": 0, 2024-05-08T05:40:41.350 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_watch": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_notify": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_src_cmpxattr": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_pgls_filter": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "osdop_other": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_active": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_send": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_resend": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "linger_ping": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_active": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_send": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "poolop_resend": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_active": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_send": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "poolstat_resend": 0, 2024-05-08T05:40:41.351 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_active": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_send": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "statfs_resend": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "command_active": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "command_send": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "command_resend": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "map_epoch": 102, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "map_full": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "map_inc": 14, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_sessions": 5, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_open": 5, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_session_close": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "osd_laggy": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_wr": 0, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_rd": 2, 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_del": 0 2024-05-08T05:40:41.352 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "oft": { 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_objs": 0, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_kv_pairs": 0, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_updates": 0, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "omap_total_removes": 0 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "osd-slow-ops": { 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "slow_ops_count": 0 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "purge_queue": { 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops": 0, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_ops_high_water": 0, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing": 0, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executing_high_water": 0, 2024-05-08T05:40:41.353 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed_ops": 0, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_executed": 0, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "pq_item_in_journal": 0 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 2788, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 3438244, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 2788, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:41.354 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2788, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 3438244, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_bytes": { 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 104857600, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:41.355 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2485, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 1162794, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2485, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 1162794, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-objecter_ops": { 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:41.356 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 1024, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 2485, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 2485, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 2485, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 2485, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.357 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle": { 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.358 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: }, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "throttle-write_buf_throttle-0x557b4bcc66e0": { 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "val": 0, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "max": 3758096384, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "get_started": 0, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "get": 0, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "get_sum": 0, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_fail": 0, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "get_or_fail_success": 0, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "take": 0, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "take_sum": 0, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "put": 0, 2024-05-08T05:40:41.359 INFO:teuthology.orchestra.run.smithi143.stdout: "put_sum": 0, 2024-05-08T05:40:41.360 INFO:teuthology.orchestra.run.smithi143.stdout: "wait": { 2024-05-08T05:40:41.360 INFO:teuthology.orchestra.run.smithi143.stdout: "avgcount": 0, 2024-05-08T05:40:41.360 INFO:teuthology.orchestra.run.smithi143.stdout: "sum": 0.000000000, 2024-05-08T05:40:41.360 INFO:teuthology.orchestra.run.smithi143.stdout: "avgtime": 0.000000000 2024-05-08T05:40:41.360 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:41.360 INFO:teuthology.orchestra.run.smithi143.stdout: } 2024-05-08T05:40:41.360 INFO:teuthology.orchestra.run.smithi143.stdout:} 2024-05-08T05:40:41.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:41 smithi143 ceph-mon[34337]: pgmap v347: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T05:40:41.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:41 smithi059 ceph-mon[30304]: pgmap v347: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T05:40:41.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:41 smithi096 ceph-mon[34253]: pgmap v347: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T05:40:42.121 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-05-08T05:40:42.122 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-05-08T05:40:42.122 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=0 2024-05-08T05:40:42.122 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-05-08T05:40:42.122 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-05-08T05:40:42.122 INFO:tasks.check_counter:Daemon mds.l mds.dir_split=0 2024-05-08T05:40:42.122 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-05-08T05:40:42.133 INFO:tasks.kclient:Unmounting kernel clients... 2024-05-08T05:40:42.134 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:40:42.134 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:40:42.157 DEBUG:teuthology.orchestra.run.smithi059:> set -ex 2024-05-08T05:40:42.157 DEBUG:teuthology.orchestra.run.smithi059:> dd if=/proc/self/mounts of=/dev/stdout 2024-05-08T05:40:42.217 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd blocklist ls 2024-05-08T05:40:42.524 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:43.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:43 smithi059 ceph-mon[30304]: pgmap v348: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:43.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:43 smithi143 ceph-mon[34337]: pgmap v348: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:43.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:43 smithi096 ceph-mon[34253]: pgmap v348: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:44.391 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.388+0000 7fb45800f700 1 -- 172.21.15.59:0/3800072814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb450109600 msgr2=0x7fb4501099e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.388+0000 7fb45800f700 1 --2- 172.21.15.59:0/3800072814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb450109600 0x7fb4501099e0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009990 tx=0x7fb44c009ca0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.388+0000 7fb45800f700 1 -- 172.21.15.59:0/3800072814 shutdown_connections 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.388+0000 7fb45800f700 1 --2- 172.21.15.59:0/3800072814 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb450103eb0 0x7fb450110390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.388+0000 7fb45800f700 1 --2- 172.21.15.59:0/3800072814 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb450109600 0x7fb4501099e0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.388+0000 7fb45800f700 1 --2- 172.21.15.59:0/3800072814 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb4501034f0 0x7fb450103970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.388+0000 7fb45800f700 1 -- 172.21.15.59:0/3800072814 >> 172.21.15.59:0/3800072814 conn(0x7fb45007a650 msgr2=0x7fb45007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.388+0000 7fb45800f700 1 -- 172.21.15.59:0/3800072814 shutdown_connections 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.388+0000 7fb45800f700 1 -- 172.21.15.59:0/3800072814 wait complete. 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb45800f700 1 Processor -- start 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb45800f700 1 -- start start 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb45800f700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4501034f0 0x7fb4500757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb45800f700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb450103eb0 0x7fb450075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb45800f700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb450109600 0x7fb450076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:44.392 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb45800f700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fb450113650 con 0x7fb4501034f0 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb45800f700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fb4501134d0 con 0x7fb450103eb0 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb45800f700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fb4501137d0 con 0x7fb450109600 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb455dab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4501034f0 0x7fb4500757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb455dab700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4501034f0 0x7fb4500757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45810/0 (socket says 172.21.15.59:45810) 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb455dab700 1 -- 172.21.15.59:0/3597889883 learned_addr learned my addr 172.21.15.59:0/3597889883 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb4555aa700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb450103eb0 0x7fb450075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb4565ac700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb450109600 0x7fb450076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb455dab700 1 -- 172.21.15.59:0/3597889883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb450109600 msgr2=0x7fb450076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb455dab700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb450109600 0x7fb450076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb455dab700 1 -- 172.21.15.59:0/3597889883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb450103eb0 msgr2=0x7fb450075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb455dab700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb450103eb0 0x7fb450075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb455dab700 1 -- 172.21.15.59:0/3597889883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4501a7fb0 con 0x7fb4501034f0 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.389+0000 7fb4565ac700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb450109600 0x7fb450076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-05-08T05:40:44.393 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.390+0000 7fb455dab700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4501034f0 0x7fb4500757a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb44c000c00 tx=0x7fb44c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:44.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.390+0000 7fb442ffd700 1 -- 172.21.15.59:0/3597889883 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c027070 con 0x7fb4501034f0 2024-05-08T05:40:44.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.390+0000 7fb442ffd700 1 -- 172.21.15.59:0/3597889883 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c0041d0 con 0x7fb4501034f0 2024-05-08T05:40:44.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.390+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4501a8240 con 0x7fb4501034f0 2024-05-08T05:40:44.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.390+0000 7fb442ffd700 1 -- 172.21.15.59:0/3597889883 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00f460 con 0x7fb4501034f0 2024-05-08T05:40:44.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.390+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4501a86f0 con 0x7fb4501034f0 2024-05-08T05:40:44.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.391+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45006a190 con 0x7fb4501034f0 2024-05-08T05:40:44.394 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.393+0000 7fb442ffd700 1 -- 172.21.15.59:0/3597889883 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fb44c02a070 con 0x7fb4501034f0 2024-05-08T05:40:44.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.394+0000 7fb442ffd700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb43c0b2a70 0x7fb43c0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:44.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.395+0000 7fb4555aa700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb43c0b2a70 0x7fb43c0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:44.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.395+0000 7fb442ffd700 1 -- 172.21.15.59:0/3597889883 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fb44c0e3d60 con 0x7fb4501034f0 2024-05-08T05:40:44.395 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.395+0000 7fb4555aa700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb43c0b2a70 0x7fb43c0b4f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb444005ea0 tx=0x7fb444005e30 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:44.397 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.397+0000 7fb442ffd700 1 -- 172.21.15.59:0/3597889883 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fb44c031070 con 0x7fb4501034f0 2024-05-08T05:40:44.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.630+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fb4501a8950 con 0x7fb4501034f0 2024-05-08T05:40:44.631 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.631+0000 7fb442ffd700 1 -- 172.21.15.59:0/3597889883 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v102) ==== 81+0+937 (secure 0 0 0) 0x7fb44c013070 con 0x7fb4501034f0 2024-05-08T05:40:44.631 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2839794658 2024-05-09T05:29:33.818933+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/546809822 2024-05-09T05:29:33.818933+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3042855743 2024-05-09T05:29:33.818933+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1220365613 2024-05-09T05:29:33.818933+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/546809822 2024-05-09T05:29:33.818933+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/1451301612 2024-05-09T05:18:03.888613+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3771264184 2024-05-09T05:18:03.888613+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/95297313 2024-05-09T05:18:03.888613+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3190007358 2024-05-09T05:29:33.818933+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2518610768 2024-05-09T05:17:35.412203+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/2478322707 2024-05-09T05:17:35.412203+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/95297313 2024-05-09T05:18:03.888613+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6800/817743540 2024-05-09T05:17:35.412203+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:6801/817743540 2024-05-09T05:17:35.412203+0000 2024-05-08T05:40:44.632 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/3865726924 2024-05-09T05:17:35.412203+0000 2024-05-08T05:40:44.633 INFO:teuthology.orchestra.run.smithi059.stdout:172.21.15.59:0/362628113 2024-05-09T05:18:03.888613+0000 2024-05-08T05:40:44.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.633+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb43c0b2a70 msgr2=0x7fb43c0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:44.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.633+0000 7fb45800f700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb43c0b2a70 0x7fb43c0b4f30 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fb444005ea0 tx=0x7fb444005e30 comp rx=0 tx=0).stop 2024-05-08T05:40:44.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4501034f0 msgr2=0x7fb4500757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:44.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4501034f0 0x7fb4500757a0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fb44c000c00 tx=0x7fb44c0047b0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 shutdown_connections 2024-05-08T05:40:44.634 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fb43c0b2a70 0x7fb43c0b4f30 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fb450109600 0x7fb450076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fb4501034f0 0x7fb4500757a0 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 --2- 172.21.15.59:0/3597889883 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fb450103eb0 0x7fb450075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:44.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 >> 172.21.15.59:0/3597889883 conn(0x7fb45007a650 msgr2=0x7fb450101f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:44.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 shutdown_connections 2024-05-08T05:40:44.635 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:44.634+0000 7fb45800f700 1 -- 172.21.15.59:0/3597889883 wait complete. 2024-05-08T05:40:44.635 INFO:teuthology.orchestra.run.smithi059.stderr:listed 16 entries 2024-05-08T05:40:45.342 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-05-08T05:40:45.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:40:45.342 DEBUG:teuthology.orchestra.run.smithi059:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-05-08T05:40:45.441 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:40:45.441 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:40:45.441 DEBUG:teuthology.orchestra.run.smithi059:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-05-08T05:40:45.477 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:40:45.477 DEBUG:teuthology.orchestra.run.smithi059:> ip netns list 2024-05-08T05:40:45.533 INFO:teuthology.orchestra.run.smithi059.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-05-08T05:40:45.533 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:40:45.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-05-08T05:40:45.564 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:45 smithi143 ceph-mon[34337]: pgmap v349: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:45.564 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:45 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3597889883' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:40:45.582 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:45 smithi096 ceph-mon[34253]: pgmap v349: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:45.582 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:45 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3597889883' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:40:45.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:45 smithi059 ceph-mon[30304]: pgmap v349: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:45.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:45 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3597889883' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-05-08T05:40:45.600 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-05-08T05:40:45.600 DEBUG:teuthology.orchestra.run.smithi059:> sudo ip link delete ceph-brx 2024-05-08T05:40:45.689 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-05-08T05:40:45.699 INFO:tasks.cephadm:Teardown begin 2024-05-08T05:40:45.699 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-05-08T05:40:45.700 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:40:45.954 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:47.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:47 smithi059 ceph-mon[30304]: pgmap v350: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T05:40:47.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.610+0000 7ff218124700 1 -- 172.21.15.59:0/1045343653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff210108410 msgr2=0x7ff2101087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:47.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.610+0000 7ff218124700 1 --2- 172.21.15.59:0/1045343653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff210108410 0x7ff2101087f0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00b0a0 tx=0x7ff20c00b3b0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.610+0000 7ff218124700 1 -- 172.21.15.59:0/1045343653 shutdown_connections 2024-05-08T05:40:47.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.610+0000 7ff218124700 1 --2- 172.21.15.59:0/1045343653 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff210102d60 0x7ff21010f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.610+0000 7ff218124700 1 --2- 172.21.15.59:0/1045343653 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff210108410 0x7ff2101087f0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.610+0000 7ff218124700 1 --2- 172.21.15.59:0/1045343653 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff2101023a0 0x7ff210102820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.610+0000 7ff218124700 1 -- 172.21.15.59:0/1045343653 >> 172.21.15.59:0/1045343653 conn(0x7ff210079350 msgr2=0x7ff210079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:47.611 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 -- 172.21.15.59:0/1045343653 shutdown_connections 2024-05-08T05:40:47.612 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 -- 172.21.15.59:0/1045343653 wait complete. 2024-05-08T05:40:47.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 Processor -- start 2024-05-08T05:40:47.613 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 -- start start 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2101023a0 0x7ff210104080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff210102d60 0x7ff2101045c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff210108410 0x7ff210104b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff210112560 con 0x7ff2101023a0 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101123e0 con 0x7ff210102d60 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.611+0000 7ff218124700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff2101126e0 con 0x7ff210108410 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff215ec0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2101023a0 0x7ff210104080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff215ec0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2101023a0 0x7ff210104080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:45830/0 (socket says 172.21.15.59:45830) 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff215ec0700 1 -- 172.21.15.59:0/468983316 learned_addr learned my addr 172.21.15.59:0/468983316 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff2156bf700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff210102d60 0x7ff2101045c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff2166c1700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff210108410 0x7ff210104b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff215ec0700 1 -- 172.21.15.59:0/468983316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff210108410 msgr2=0x7ff210104b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff215ec0700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff210108410 0x7ff210104b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.614 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff215ec0700 1 -- 172.21.15.59:0/468983316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff210102d60 msgr2=0x7ff2101045c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff215ec0700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff210102d60 0x7ff2101045c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff215ec0700 1 -- 172.21.15.59:0/468983316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2101a80f0 con 0x7ff2101023a0 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff215ec0700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2101023a0 0x7ff210104080 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00b070 tx=0x7ff20c004d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff206ffd700 1 -- 172.21.15.59:0/468983316 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c021070 con 0x7ff2101023a0 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff206ffd700 1 -- 172.21.15.59:0/468983316 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff20c007e30 con 0x7ff2101023a0 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2101a8380 con 0x7ff2101023a0 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff206ffd700 1 -- 172.21.15.59:0/468983316 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff20c0046c0 con 0x7ff2101023a0 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.612+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2101a8880 con 0x7ff2101023a0 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.613+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff21006a190 con 0x7ff2101023a0 2024-05-08T05:40:47.615 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.615+0000 7ff206ffd700 1 -- 172.21.15.59:0/468983316 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff20c007980 con 0x7ff2101023a0 2024-05-08T05:40:47.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.616+0000 7ff206ffd700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1fc0b2a70 0x7ff1fc0b4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:47.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.616+0000 7ff206ffd700 1 -- 172.21.15.59:0/468983316 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7ff20c00f020 con 0x7ff2101023a0 2024-05-08T05:40:47.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.616+0000 7ff2156bf700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1fc0b2a70 0x7ff1fc0b4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:47.616 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.616+0000 7ff2156bf700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1fc0b2a70 0x7ff1fc0b4f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff200009680 tx=0x7ff200009260 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:47.619 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.619+0000 7ff206ffd700 1 -- 172.21.15.59:0/468983316 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff20c02a020 con 0x7ff2101023a0 2024-05-08T05:40:47.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:47 smithi143 ceph-mon[34337]: pgmap v350: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T05:40:47.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:47 smithi096 ceph-mon[34253]: pgmap v350: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-05-08T05:40:47.888 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.888+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2101a8ab0 con 0x7ff2101023a0 2024-05-08T05:40:47.889 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.889+0000 7ff206ffd700 1 -- 172.21.15.59:0/468983316 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11932 (secure 0 0 0) 0x7ff20c0ab840 con 0x7ff2101023a0 2024-05-08T05:40:47.889 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:40:47.890 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:39:55.325460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25876":{"gid":25876,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2351774265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2351774265},{"type":"v1","addr":"172.21.15.143:6835","nonce":2351774265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:40:47.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1fc0b2a70 msgr2=0x7ff1fc0b4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:47.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1fc0b2a70 0x7ff1fc0b4f30 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff200009680 tx=0x7ff200009260 comp rx=0 tx=0).stop 2024-05-08T05:40:47.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2101023a0 msgr2=0x7ff210104080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:47.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2101023a0 0x7ff210104080 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7ff20c00b070 tx=0x7ff20c004d00 comp rx=0 tx=0).stop 2024-05-08T05:40:47.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 shutdown_connections 2024-05-08T05:40:47.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff1fc0b2a70 0x7ff1fc0b4f30 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.892 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff210108410 0x7ff210104b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff2101023a0 0x7ff210104080 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 --2- 172.21.15.59:0/468983316 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff210102d60 0x7ff2101045c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:47.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.891+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 >> 172.21.15.59:0/468983316 conn(0x7ff210079350 msgr2=0x7ff2101017b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:47.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.892+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 shutdown_connections 2024-05-08T05:40:47.893 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:47.892+0000 7ff218124700 1 -- 172.21.15.59:0/468983316 wait complete. 2024-05-08T05:40:47.893 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 41 2024-05-08T05:40:48.526 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:48 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/468983316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:40:48.528 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs dump --format=json 2024-05-08T05:40:48.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:48 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/468983316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:40:48.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:48 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/468983316' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:40:48.783 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:49.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:49 smithi059 ceph-mon[30304]: pgmap v351: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:49.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:49 smithi143 ceph-mon[34337]: pgmap v351: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:49.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:49 smithi096 ceph-mon[34253]: pgmap v351: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-05-08T05:40:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.560+0000 7f98d27e4700 1 -- 172.21.15.59:0/2062654801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc104100 msgr2=0x7f98cc108120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.560+0000 7f98d27e4700 1 --2- 172.21.15.59:0/2062654801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc104100 0x7f98cc108120 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f98b40099c0 tx=0x7f98b4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.560+0000 7f98d27e4700 1 -- 172.21.15.59:0/2062654801 shutdown_connections 2024-05-08T05:40:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.560+0000 7f98d27e4700 1 --2- 172.21.15.59:0/2062654801 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc108850 0x7f98cc1103d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.560+0000 7f98d27e4700 1 --2- 172.21.15.59:0/2062654801 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc103750 0x7f98cc103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.560+0000 7f98d27e4700 1 --2- 172.21.15.59:0/2062654801 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc104100 0x7f98cc108120 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.561 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.560+0000 7f98d27e4700 1 -- 172.21.15.59:0/2062654801 >> 172.21.15.59:0/2062654801 conn(0x7f98cc07a600 msgr2=0x7f98cc07aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:50.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.560+0000 7f98d27e4700 1 -- 172.21.15.59:0/2062654801 shutdown_connections 2024-05-08T05:40:50.562 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.560+0000 7f98d27e4700 1 -- 172.21.15.59:0/2062654801 wait complete. 2024-05-08T05:40:50.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d27e4700 1 Processor -- start 2024-05-08T05:40:50.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d27e4700 1 -- start start 2024-05-08T05:40:50.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d27e4700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc103750 0x7f98cc0757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:50.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d27e4700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc104100 0x7f98cc075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:50.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d27e4700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc108850 0x7f98cc076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:50.563 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d27e4700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc113710 con 0x7f98cc108850 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d27e4700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc113590 con 0x7f98cc104100 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d27e4700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f98cc113890 con 0x7f98cc103750 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d0d81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc108850 0x7f98cc076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98cbfff700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc103750 0x7f98cc0757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d0d81700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc108850 0x7f98cc076220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:48990/0 (socket says 172.21.15.59:48990) 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98d0d81700 1 -- 172.21.15.59:0/472277146 learned_addr learned my addr 172.21.15.59:0/472277146 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.561+0000 7f98c3fff700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc104100 0x7f98cc075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98d0d81700 1 -- 172.21.15.59:0/472277146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc103750 msgr2=0x7f98cc0757a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98d0d81700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc103750 0x7f98cc0757a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98d0d81700 1 -- 172.21.15.59:0/472277146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc104100 msgr2=0x7f98cc075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98d0d81700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc104100 0x7f98cc075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98d0d81700 1 -- 172.21.15.59:0/472277146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98cc1a80c0 con 0x7f98cc108850 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98c3fff700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc104100 0x7f98cc075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-05-08T05:40:50.564 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98d0d81700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc108850 0x7f98cc076220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f98c400b970 tx=0x7f98c400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:50.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98c9ffb700 1 -- 172.21.15.59:0/472277146 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4019070 con 0x7f98cc108850 2024-05-08T05:40:50.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98cc1a83b0 con 0x7f98cc108850 2024-05-08T05:40:50.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98c9ffb700 1 -- 172.21.15.59:0/472277146 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98c400cde0 con 0x7f98cc108850 2024-05-08T05:40:50.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98cc1a88c0 con 0x7f98cc108850 2024-05-08T05:40:50.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.562+0000 7f98c9ffb700 1 -- 172.21.15.59:0/472277146 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98c4007370 con 0x7f98cc108850 2024-05-08T05:40:50.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.563+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98cc104be0 con 0x7f98cc108850 2024-05-08T05:40:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.564+0000 7f98c9ffb700 1 -- 172.21.15.59:0/472277146 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f98c401c070 con 0x7f98cc108850 2024-05-08T05:40:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.565+0000 7f98c9ffb700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f98b80b29f0 0x7f98b80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.565+0000 7f98cbfff700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f98b80b29f0 0x7f98b80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.565+0000 7f98c9ffb700 1 -- 172.21.15.59:0/472277146 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f98c40e25a0 con 0x7f98cc108850 2024-05-08T05:40:50.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.566+0000 7f98cbfff700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f98b80b29f0 0x7f98b80b4eb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009a10 tx=0x7f98bc009190 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:50.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.569+0000 7f98c9ffb700 1 -- 172.21.15.59:0/472277146 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f98c4023070 con 0x7f98cc108850 2024-05-08T05:40:50.569 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:50 smithi059 ceph-mon[30304]: pgmap v352: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 14 op/s 2024-05-08T05:40:50.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:50 smithi143 ceph-mon[34337]: pgmap v352: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 14 op/s 2024-05-08T05:40:50.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:50 smithi096 ceph-mon[34253]: pgmap v352: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 14 op/s 2024-05-08T05:40:50.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.839+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f98cc02e510 con 0x7f98cc108850 2024-05-08T05:40:50.840 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.840+0000 7f98c9ffb700 1 -- 172.21.15.59:0/472277146 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) ==== 94+0+11932 (secure 0 0 0) 0x7f98c40a9c60 con 0x7f98cc108850 2024-05-08T05:40:50.841 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:40:50.841 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":41,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14517,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6835/3565849854","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6834","nonce":3565849854},{"type":"v1","addr":"172.21.15.59:6835","nonce":3565849854}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14529,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.59:6837/4212370259","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6836","nonce":4212370259},{"type":"v1","addr":"172.21.15.59:6837","nonce":4212370259}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":41,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-05-08T05:25:27.601131+0000","modified":"2024-05-08T05:39:55.325460+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24473,"mds_1":24443,"mds_2":24433,"mds_3":24421,"mds_4":24419},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14565":{"gid":14565,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6839/2756426260","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6838","nonce":2756426260},{"type":"v1","addr":"172.21.15.59:6839","nonce":2756426260}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14571":{"gid":14571,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.59:6841/2496793439","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6840","nonce":2496793439},{"type":"v1","addr":"172.21.15.59:6841","nonce":2496793439}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24385":{"gid":24385,"name":"l","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6833/1879950931","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6832","nonce":1879950931},{"type":"v1","addr":"172.21.15.143:6833","nonce":1879950931}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24401":{"gid":24401,"name":"k","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.96:6833/2593895617","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6832","nonce":2593895617},{"type":"v1","addr":"172.21.15.96:6833","nonce":2593895617}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24419":{"gid":24419,"name":"e","rank":4,"incarnation":26,"state":"up:active","state_seq":26,"addr":"172.21.15.96:6835/2294126295","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6834","nonce":2294126295},{"type":"v1","addr":"172.21.15.96:6835","nonce":2294126295}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24421":{"gid":24421,"name":"c","rank":3,"incarnation":24,"state":"up:active","state_seq":24,"addr":"172.21.15.143:6837/555225813","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6836","nonce":555225813},{"type":"v1","addr":"172.21.15.143:6837","nonce":555225813}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24433":{"gid":24433,"name":"i","rank":2,"incarnation":22,"state":"up:active","state_seq":22,"addr":"172.21.15.143:6839/2166229809","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6838","nonce":2166229809},{"type":"v1","addr":"172.21.15.143:6839","nonce":2166229809}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24443":{"gid":24443,"name":"h","rank":1,"incarnation":20,"state":"up:active","state_seq":23,"addr":"172.21.15.96:6837/780978870","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6836","nonce":780978870},{"type":"v1","addr":"172.21.15.96:6837","nonce":780978870}]},"join_fscid":-1,"export_targets":[0],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24473":{"gid":24473,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":5,"addr":"172.21.15.96:6839/126053836","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6838","nonce":126053836},{"type":"v1","addr":"172.21.15.96:6839","nonce":126053836}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_25876":{"gid":25876,"name":"f","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.143:6835/2351774265","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6834","nonce":2351774265},{"type":"v1","addr":"172.21.15.143:6835","nonce":2351774265}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24473,"qdb_cluster":[24419,24433,24421,24443,24473]},"id":1}]} 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f98b80b29f0 msgr2=0x7f98b80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f98b80b29f0 0x7f98b80b4eb0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f98bc009a10 tx=0x7f98bc009190 comp rx=0 tx=0).stop 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc108850 msgr2=0x7f98cc076220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc108850 0x7f98cc076220 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f98c400b970 tx=0x7f98c400bc80 comp rx=0 tx=0).stop 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 shutdown_connections 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f98b80b29f0 0x7f98b80b4eb0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f98cc108850 0x7f98cc076220 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f98cc103750 0x7f98cc0757a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 --2- 172.21.15.59:0/472277146 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f98cc104100 0x7f98cc075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:50.843 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 >> 172.21.15.59:0/472277146 conn(0x7f98cc07a600 msgr2=0x7f98cc102d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 shutdown_connections 2024-05-08T05:40:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:50.842+0000 7f98d27e4700 1 -- 172.21.15.59:0/472277146 wait complete. 2024-05-08T05:40:50.844 INFO:teuthology.orchestra.run.smithi059.stderr:dumped fsmap epoch 41 2024-05-08T05:40:51.516 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:51 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/472277146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:40:51.546 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd dump --format=json 2024-05-08T05:40:51.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:51 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/472277146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:40:51.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:51 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/472277146' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-05-08T05:40:51.804 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:52 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:40:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:52 smithi059 ceph-mon[30304]: pgmap v353: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-08T05:40:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:52 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:40:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:52 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:40:52.586 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:52 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:40:52.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:52 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:40:52.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:52 smithi143 ceph-mon[34337]: pgmap v353: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-08T05:40:52.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:52 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:40:52.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:52 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:40:52.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:52 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:40:52.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:52 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-05-08T05:40:52.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:52 smithi096 ceph-mon[34253]: pgmap v353: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-08T05:40:52.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:52 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-05-08T05:40:52.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:52 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-05-08T05:40:52.780 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:52 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' 2024-05-08T05:40:53.565 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.562+0000 7f0631e18700 1 -- 172.21.15.59:0/694774960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 msgr2=0x7f062c106720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.562+0000 7f0631e18700 1 --2- 172.21.15.59:0/694774960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 0x7f062c106720 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f061c0099c0 tx=0x7f061c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.562+0000 7f0631e18700 1 -- 172.21.15.59:0/694774960 shutdown_connections 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.562+0000 7f0631e18700 1 --2- 172.21.15.59:0/694774960 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f062c108b70 0x7f062c10f330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.562+0000 7f0631e18700 1 --2- 172.21.15.59:0/694774960 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 0x7f062c106720 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.562+0000 7f0631e18700 1 --2- 172.21.15.59:0/694774960 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c1081b0 0x7f062c108630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.562+0000 7f0631e18700 1 -- 172.21.15.59:0/694774960 >> 172.21.15.59:0/694774960 conn(0x7f062c0776c0 msgr2=0x7f062c077ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.562+0000 7f0631e18700 1 -- 172.21.15.59:0/694774960 shutdown_connections 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.562+0000 7f0631e18700 1 -- 172.21.15.59:0/694774960 wait complete. 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f0631e18700 1 Processor -- start 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f0631e18700 1 -- start start 2024-05-08T05:40:53.566 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f0631e18700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 0x7f062c103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f0631e18700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f062c1081b0 0x7f062c103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f0631e18700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c108b70 0x7f062c103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f0631e18700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f062c112610 con 0x7f062c106340 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f0631e18700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f062c112490 con 0x7f062c1081b0 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f0631e18700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f062c112790 con 0x7f062c108b70 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f062b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 0x7f062c103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f062b7fe700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 0x7f062c103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49010/0 (socket says 172.21.15.59:49010) 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.563+0000 7f062b7fe700 1 -- 172.21.15.59:0/3754308230 learned_addr learned my addr 172.21.15.59:0/3754308230 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f062bfff700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c108b70 0x7f062c103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f062affd700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f062c1081b0 0x7f062c103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f062b7fe700 1 -- 172.21.15.59:0/3754308230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c108b70 msgr2=0x7f062c103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f062b7fe700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c108b70 0x7f062c103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f062b7fe700 1 -- 172.21.15.59:0/3754308230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f062c1081b0 msgr2=0x7f062c103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f062b7fe700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f062c1081b0 0x7f062c103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.567 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f062b7fe700 1 -- 172.21.15.59:0/3754308230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f062c1a8110 con 0x7f062c106340 2024-05-08T05:40:53.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f062b7fe700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 0x7f062c103040 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b580 tx=0x7f061c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:53.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f0628ff9700 1 -- 172.21.15.59:0/3754308230 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c027070 con 0x7f062c106340 2024-05-08T05:40:53.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f062c1a83a0 con 0x7f062c106340 2024-05-08T05:40:53.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f0628ff9700 1 -- 172.21.15.59:0/3754308230 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f061c003f20 con 0x7f062c106340 2024-05-08T05:40:53.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f0628ff9700 1 -- 172.21.15.59:0/3754308230 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f061c022d70 con 0x7f062c106340 2024-05-08T05:40:53.568 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.564+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f062c1a8850 con 0x7f062c106340 2024-05-08T05:40:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.565+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f062c06a190 con 0x7f062c106340 2024-05-08T05:40:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.567+0000 7f0628ff9700 1 -- 172.21.15.59:0/3754308230 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f061c022500 con 0x7f062c106340 2024-05-08T05:40:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.567+0000 7f0628ff9700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f06140b29f0 0x7f06140b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.567+0000 7f062affd700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f06140b29f0 0x7f06140b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.567+0000 7f0628ff9700 1 -- 172.21.15.59:0/3754308230 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f061c0e3a10 con 0x7f062c106340 2024-05-08T05:40:53.569 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.568+0000 7f062affd700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f06140b29f0 0x7f06140b4eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0620005950 tx=0x7f06200058e0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:53.572 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.571+0000 7f0628ff9700 1 -- 172.21.15.59:0/3754308230 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f061c013070 con 0x7f062c106340 2024-05-08T05:40:53.795 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.794+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f062c02e510 con 0x7f062c106340 2024-05-08T05:40:53.797 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.796+0000 7f0628ff9700 1 -- 172.21.15.59:0/3754308230 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v102) ==== 74+0+25257 (secure 0 0 0) 0x7f061c0ab0b0 con 0x7f062c106340 2024-05-08T05:40:53.798 INFO:teuthology.orchestra.run.smithi059.stdout: 2024-05-08T05:40:53.798 INFO:teuthology.orchestra.run.smithi059.stdout:{"epoch":102,"fsid":"0bf01a10-0cfa-11ef-bc97-c7b262605968","created":"2024-05-08T05:17:00.521511+0000","modified":"2024-05-08T05:40:20.723802+0000","last_up_change":"2024-05-08T05:24:20.031838+0000","last_in_change":"2024-05-08T05:23:59.794035+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-05-08T05:20:31.976415+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-05-08T05:25:21.249621+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-05-08T05:25:24.480989+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-05-08T05:25:35.040484+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"101","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":101,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"2bd987ee-e978-4d9b-89b6-6acca697cac4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6802","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6803","nonce":4120729833}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6804","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6805","nonce":4120729833}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6808","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6809","nonce":4120729833}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6806","nonce":4120729833},{"type":"v1","addr":"172.21.15.59:6807","nonce":4120729833}]},"public_addr":"172.21.15.59:6803/4120729833","cluster_addr":"172.21.15.59:6805/4120729833","heartbeat_back_addr":"172.21.15.59:6809/4120729833","heartbeat_front_addr":"172.21.15.59:6807/4120729833","state":["exists","up"]},{"osd":1,"uuid":"855aa3c7-8818-419d-8a7d-078e3bd10535","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6800","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6801","nonce":2776039768}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6802","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6803","nonce":2776039768}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6806","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6807","nonce":2776039768}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6804","nonce":2776039768},{"type":"v1","addr":"172.21.15.96:6805","nonce":2776039768}]},"public_addr":"172.21.15.96:6801/2776039768","cluster_addr":"172.21.15.96:6803/2776039768","heartbeat_back_addr":"172.21.15.96:6807/2776039768","heartbeat_front_addr":"172.21.15.96:6805/2776039768","state":["exists","up"]},{"osd":2,"uuid":"a05037ce-9958-4ac3-9377-9a6bef15510d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6800","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6801","nonce":815254096}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6802","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6803","nonce":815254096}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6806","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6807","nonce":815254096}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6804","nonce":815254096},{"type":"v1","addr":"172.21.15.143:6805","nonce":815254096}]},"public_addr":"172.21.15.143:6801/815254096","cluster_addr":"172.21.15.143:6803/815254096","heartbeat_back_addr":"172.21.15.143:6807/815254096","heartbeat_front_addr":"172.21.15.143:6805/815254096","state":["exists","up"]},{"osd":3,"uuid":"40198b3b-3215-4d59-9a89-4c18fa5271b1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6810","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6811","nonce":3135413737}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6812","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6813","nonce":3135413737}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6816","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6817","nonce":3135413737}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6814","nonce":3135413737},{"type":"v1","addr":"172.21.15.59:6815","nonce":3135413737}]},"public_addr":"172.21.15.59:6811/3135413737","cluster_addr":"172.21.15.59:6813/3135413737","heartbeat_back_addr":"172.21.15.59:6817/3135413737","heartbeat_front_addr":"172.21.15.59:6815/3135413737","state":["exists","up"]},{"osd":4,"uuid":"328d5640-5ab1-4baf-b141-039dce5211c9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":31,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6808","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6809","nonce":2351393060}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6810","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6811","nonce":2351393060}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6814","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6815","nonce":2351393060}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6812","nonce":2351393060},{"type":"v1","addr":"172.21.15.96:6813","nonce":2351393060}]},"public_addr":"172.21.15.96:6809/2351393060","cluster_addr":"172.21.15.96:6811/2351393060","heartbeat_back_addr":"172.21.15.96:6815/2351393060","heartbeat_front_addr":"172.21.15.96:6813/2351393060","state":["exists","up"]},{"osd":5,"uuid":"592fc3c9-1310-441d-b834-bec5021dba8e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":36,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6808","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6809","nonce":1078449086}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6810","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6811","nonce":1078449086}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6814","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6815","nonce":1078449086}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6812","nonce":1078449086},{"type":"v1","addr":"172.21.15.143:6813","nonce":1078449086}]},"public_addr":"172.21.15.143:6809/1078449086","cluster_addr":"172.21.15.143:6811/1078449086","heartbeat_back_addr":"172.21.15.143:6815/1078449086","heartbeat_front_addr":"172.21.15.143:6813/1078449086","state":["exists","up"]},{"osd":6,"uuid":"bce63f2d-f8c7-4304-bec8-8f1d803e2791","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":42,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6818","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6819","nonce":126911389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6820","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6821","nonce":126911389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6824","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6825","nonce":126911389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6822","nonce":126911389},{"type":"v1","addr":"172.21.15.59:6823","nonce":126911389}]},"public_addr":"172.21.15.59:6819/126911389","cluster_addr":"172.21.15.59:6821/126911389","heartbeat_back_addr":"172.21.15.59:6825/126911389","heartbeat_front_addr":"172.21.15.59:6823/126911389","state":["exists","up"]},{"osd":7,"uuid":"f10748de-9037-46bd-9db3-1d88f6f93af8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":47,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6816","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6817","nonce":2845324468}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6818","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6819","nonce":2845324468}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6822","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6823","nonce":2845324468}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6820","nonce":2845324468},{"type":"v1","addr":"172.21.15.96:6821","nonce":2845324468}]},"public_addr":"172.21.15.96:6817/2845324468","cluster_addr":"172.21.15.96:6819/2845324468","heartbeat_back_addr":"172.21.15.96:6823/2845324468","heartbeat_front_addr":"172.21.15.96:6821/2845324468","state":["exists","up"]},{"osd":8,"uuid":"6e84c2e9-f309-4779-8e07-0b13fc05d3c1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6816","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6817","nonce":2760087187}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6818","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6819","nonce":2760087187}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6822","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6823","nonce":2760087187}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6820","nonce":2760087187},{"type":"v1","addr":"172.21.15.143:6821","nonce":2760087187}]},"public_addr":"172.21.15.143:6817/2760087187","cluster_addr":"172.21.15.143:6819/2760087187","heartbeat_back_addr":"172.21.15.143:6823/2760087187","heartbeat_front_addr":"172.21.15.143:6821/2760087187","state":["exists","up"]},{"osd":9,"uuid":"d6035037-1d0b-49dd-ae6c-2708beba9c2b","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":60,"up_thru":82,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6826","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6827","nonce":440517197}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6828","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6829","nonce":440517197}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6832","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6833","nonce":440517197}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.59:6830","nonce":440517197},{"type":"v1","addr":"172.21.15.59:6831","nonce":440517197}]},"public_addr":"172.21.15.59:6827/440517197","cluster_addr":"172.21.15.59:6829/440517197","heartbeat_back_addr":"172.21.15.59:6833/440517197","heartbeat_front_addr":"172.21.15.59:6831/440517197","state":["exists","up"]},{"osd":10,"uuid":"18a53f85-31e5-4969-b393-9dfc48d9aaa3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":65,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6824","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6825","nonce":3187941389}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6826","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6827","nonce":3187941389}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6830","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6831","nonce":3187941389}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.96:6828","nonce":3187941389},{"type":"v1","addr":"172.21.15.96:6829","nonce":3187941389}]},"public_addr":"172.21.15.96:6825/3187941389","cluster_addr":"172.21.15.96:6827/3187941389","heartbeat_back_addr":"172.21.15.96:6831/3187941389","heartbeat_front_addr":"172.21.15.96:6829/3187941389","state":["exists","up"]},{"osd":11,"uuid":"c1c3b9ed-8849-4a99-a885-fed187c8aaf7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":71,"up_thru":89,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6824","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6825","nonce":1379436601}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6826","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6827","nonce":1379436601}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6830","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6831","nonce":1379436601}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.143:6828","nonce":1379436601},{"type":"v1","addr":"172.21.15.143:6829","nonce":1379436601}]},"public_addr":"172.21.15.143:6825/1379436601","cluster_addr":"172.21.15.143:6827/1379436601","heartbeat_back_addr":"172.21.15.143:6831/1379436601","heartbeat_front_addr":"172.21.15.143:6829/1379436601","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:19:40.102549+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:03.950968+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:27.776798+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:20:52.950504+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:17.949018+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:21:43.106883+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:08.678192+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:34.116921+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:22:59.280215+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:25.691024+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:23:51.996197+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-05-08T05:24:18.343863+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.5","mappings":[{"from":8,"to":11}]},{"pgid":"2.e","mappings":[{"from":2,"to":1}]},{"pgid":"2.10","mappings":[{"from":2,"to":4}]},{"pgid":"2.1d","mappings":[{"from":8,"to":11}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.1d","mappings":[{"from":4,"to":8}]},{"pgid":"3.21","mappings":[{"from":5,"to":9}]},{"pgid":"3.2c","mappings":[{"from":4,"to":9}]},{"pgid":"4.2","mappings":[{"from":9,"to":1}]},{"pgid":"4.9","mappings":[{"from":5,"to":8}]},{"pgid":"4.12","mappings":[{"from":9,"to":10}]},{"pgid":"4.18","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.59:0/2839794658":"2024-05-09T05:29:33.818933+0000","172.21.15.59:6801/546809822":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/3042855743":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/1220365613":"2024-05-09T05:29:33.818933+0000","172.21.15.59:6800/546809822":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/1451301612":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3771264184":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6801/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:0/3190007358":"2024-05-09T05:29:33.818933+0000","172.21.15.59:0/2518610768":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/2478322707":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6800/95297313":"2024-05-09T05:18:03.888613+0000","172.21.15.59:6800/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:6801/817743540":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/3865726924":"2024-05-09T05:17:35.412203+0000","172.21.15.59:0/362628113":"2024-05-09T05:18:03.888613+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]},{"pool":4,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-05-08T05:40:53.799 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f06140b29f0 msgr2=0x7f06140b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f06140b29f0 0x7f06140b4eb0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f0620005950 tx=0x7f06200058e0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 msgr2=0x7f062c103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 0x7f062c103040 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f061c00b580 tx=0x7f061c0047b0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 shutdown_connections 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f06140b29f0 0x7f06140b4eb0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f062c108b70 0x7f062c103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f062c106340 0x7f062c103040 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 --2- 172.21.15.59:0/3754308230 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f062c1081b0 0x7f062c103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 >> 172.21.15.59:0/3754308230 conn(0x7f062c0776c0 msgr2=0x7f062c102040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 shutdown_connections 2024-05-08T05:40:53.800 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:53.798+0000 7f0631e18700 1 -- 172.21.15.59:0/3754308230 wait complete. 2024-05-08T05:40:54.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:53 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3754308230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:40:54.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:53 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3754308230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:40:54.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:53 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3754308230' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-05-08T05:40:54.571 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs fail cephfs 2024-05-08T05:40:54.832 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:55.197 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:54 smithi059 ceph-mon[30304]: pgmap v354: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-08T05:40:55.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:54 smithi143 ceph-mon[34337]: pgmap v354: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-08T05:40:55.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:54 smithi096 ceph-mon[34253]: pgmap v354: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 1023 B/s wr, 10 op/s 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.638+0000 7f37581c2700 1 -- 172.21.15.59:0/2084395185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750103780 msgr2=0x7f3750103b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.638+0000 7f37581c2700 1 --2- 172.21.15.59:0/2084395185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750103780 0x7f3750103b60 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f374c0099c0 tx=0x7f374c009cd0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.638+0000 7f37581c2700 1 -- 172.21.15.59:0/2084395185 shutdown_connections 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.638+0000 7f37581c2700 1 --2- 172.21.15.59:0/2084395185 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3750108870 0x7f37501103f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.638+0000 7f37581c2700 1 --2- 172.21.15.59:0/2084395185 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750103780 0x7f3750103b60 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.638+0000 7f37581c2700 1 --2- 172.21.15.59:0/2084395185 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3750104130 0x7f3750108140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.638+0000 7f37581c2700 1 -- 172.21.15.59:0/2084395185 >> 172.21.15.59:0/2084395185 conn(0x7f375007a610 msgr2=0x7f375007aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.638+0000 7f37581c2700 1 -- 172.21.15.59:0/2084395185 shutdown_connections 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.638+0000 7f37581c2700 1 -- 172.21.15.59:0/2084395185 wait complete. 2024-05-08T05:40:56.640 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f37581c2700 1 Processor -- start 2024-05-08T05:40:56.641 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f37581c2700 1 -- start start 2024-05-08T05:40:56.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f37581c2700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3750103780 0x7f375019f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:56.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f37581c2700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750104130 0x7f375019fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:56.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f37581c2700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3750108870 0x7f37501998a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:56.642 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f37581c2700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7f3750113660 con 0x7f3750104130 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f37581c2700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7f37501134e0 con 0x7f3750103780 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f37581c2700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7f37501137e0 con 0x7f3750108870 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f375575d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750104130 0x7f375019fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f3755f5e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3750103780 0x7f375019f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f375575d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750104130 0x7f375019fc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:49026/0 (socket says 172.21.15.59:49026) 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f375575d700 1 -- 172.21.15.59:0/3891194348 learned_addr learned my addr 172.21.15.59:0/3891194348 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.639+0000 7f375675f700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3750108870 0x7f37501998a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f375575d700 1 -- 172.21.15.59:0/3891194348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3750108870 msgr2=0x7f37501998a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f375575d700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3750108870 0x7f37501998a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f375575d700 1 -- 172.21.15.59:0/3891194348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3750103780 msgr2=0x7f375019f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f375575d700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3750103780 0x7f375019f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f375575d700 1 -- 172.21.15.59:0/3891194348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f375019a000 con 0x7f3750104130 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f3755f5e700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3750103780 0x7f375019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-05-08T05:40:56.643 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f375575d700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750104130 0x7f375019fc20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f374000c9b0 tx=0x7f374000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:56.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f3746ffd700 1 -- 172.21.15.59:0/3891194348 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740007b00 con 0x7f3750104130 2024-05-08T05:40:56.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f375019a2f0 con 0x7f3750104130 2024-05-08T05:40:56.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f3746ffd700 1 -- 172.21.15.59:0/3891194348 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f374000ce50 con 0x7f3750104130 2024-05-08T05:40:56.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f3746ffd700 1 -- 172.21.15.59:0/3891194348 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3740018660 con 0x7f3750104130 2024-05-08T05:40:56.644 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.640+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37501a65b0 con 0x7f3750104130 2024-05-08T05:40:56.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.641+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f375010bab0 con 0x7f3750104130 2024-05-08T05:40:56.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.642+0000 7f3746ffd700 1 -- 172.21.15.59:0/3891194348 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7f374000f450 con 0x7f3750104130 2024-05-08T05:40:56.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.643+0000 7f3746ffd700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f373c0b29f0 0x7f373c0b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:56.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.643+0000 7f3755f5e700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f373c0b29f0 0x7f373c0b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:56.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.643+0000 7f3746ffd700 1 -- 172.21.15.59:0/3891194348 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f37400e2920 con 0x7f3750104130 2024-05-08T05:40:56.645 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.644+0000 7f3755f5e700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f373c0b29f0 0x7f373c0b4eb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f375019a9e0 tx=0x7f374c000bc0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:56.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.648+0000 7f3746ffd700 1 -- 172.21.15.59:0/3891194348 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7f3740014030 con 0x7f3750104130 2024-05-08T05:40:56.908 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.907+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f375019add0 con 0x7f3750104130 2024-05-08T05:40:56.911 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.909+0000 7f3746ffd700 1 -- 172.21.15.59:0/3891194348 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v42) ==== 161+0+0 (secure 0 0 0) 0x7f37400a9f30 con 0x7f3750104130 2024-05-08T05:40:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.913+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f373c0b29f0 msgr2=0x7f373c0b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.913+0000 7f37581c2700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f373c0b29f0 0x7f373c0b4eb0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f375019a9e0 tx=0x7f374c000bc0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.913+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750104130 msgr2=0x7f375019fc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.913+0000 7f37581c2700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750104130 0x7f375019fc20 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7f374000c9b0 tx=0x7f374000ccc0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.914+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 shutdown_connections 2024-05-08T05:40:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.914+0000 7f37581c2700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7f373c0b29f0 0x7f373c0b4eb0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.914+0000 7f37581c2700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7f3750108870 0x7f37501998a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.914+0000 7f37581c2700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7f3750103780 0x7f375019f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.916 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.914+0000 7f37581c2700 1 --2- 172.21.15.59:0/3891194348 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7f3750104130 0x7f375019fc20 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:56.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.914+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 >> 172.21.15.59:0/3891194348 conn(0x7f375007a610 msgr2=0x7f3750102ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:56.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.914+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 shutdown_connections 2024-05-08T05:40:56.917 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:56.914+0000 7f37581c2700 1 -- 172.21.15.59:0/3891194348 wait complete. 2024-05-08T05:40:56.917 INFO:teuthology.orchestra.run.smithi059.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-05-08T05:40:57.103 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f[64685]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.103 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f[64685]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.103 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:40:56 smithi143 ceph-mds[64689]: -- 172.21.15.143:0/509762870 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562b09e37a40 con 0x562b09e0ac00 2024-05-08T05:40:57.103 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f[64685]: starting mds.f at 2024-05-08T05:40:57.103 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c[64997]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.103 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c[64997]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.103 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:40:56 smithi143 ceph-mds[65001]: -- 172.21.15.143:0/2835169934 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561f36e65a40 con 0x561f36e38c00 2024-05-08T05:40:57.103 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c[64997]: starting mds.c at 2024-05-08T05:40:57.104 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l[64369]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.104 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l[64369]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.104 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:40:56 smithi143 ceph-mds[64373]: -- 172.21.15.143:0/3688411081 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x56260a6bfa40 con 0x56260a692800 2024-05-08T05:40:57.104 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l[64369]: starting mds.l at 2024-05-08T05:40:57.104 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i[65304]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.104 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i[65304]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.104 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:40:56 smithi143 ceph-mds[65308]: -- 172.21.15.143:0/3030761962 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af01a7ba40 con 0x55af01a4e800 2024-05-08T05:40:57.104 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:40:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i[65304]: starting mds.i at 2024-05-08T05:40:57.257 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:40:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g[72165]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.257 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:40:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g[72165]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.257 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:40:56 smithi059 ceph-mds[72169]: -- 172.21.15.59:0/895754550 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559879db7a40 con 0x559879d8a800 2024-05-08T05:40:57.257 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:40:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g[72165]: starting mds.g at 2024-05-08T05:40:57.257 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a[30300]: 2024-05-08T05:40:56.908+0000 7fd46b171700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:40:57.257 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:57 smithi059 ceph-mon[30304]: pgmap v355: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 14 op/s 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:57 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3891194348' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:57 smithi059 ceph-mon[30304]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:57 smithi059 ceph-mon[30304]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:57 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/3891194348' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:57 smithi059 ceph-mon[30304]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:57 smithi059 ceph-mon[30304]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:40:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-a[72589]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:40:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-a[72589]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:40:56 smithi059 ceph-mds[72593]: -- 172.21.15.59:0/91472509 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d975b79a40 con 0x55d975b4cc00 2024-05-08T05:40:57.258 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:40:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-a[72589]: starting mds.a at 2024-05-08T05:40:57.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:57 smithi096 ceph-mon[34253]: pgmap v355: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 14 op/s 2024-05-08T05:40:57.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:57 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3891194348' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T05:40:57.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:57 smithi096 ceph-mon[34253]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T05:40:57.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:57 smithi096 ceph-mon[34253]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:40:57.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:57 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/3891194348' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T05:40:57.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:57 smithi096 ceph-mon[34253]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T05:40:57.278 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:57 smithi096 ceph-mon[34253]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T05:40:57.278 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e[63836]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.278 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e[63836]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.279 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:40:56 smithi096 ceph-mds[63840]: -- 172.21.15.96:0/580601274 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e13ac25a40 con 0x55e13abf8c00 2024-05-08T05:40:57.279 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e[63836]: starting mds.e at 2024-05-08T05:40:57.279 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h[64147]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.279 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h[64147]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.279 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:40:56 smithi096 ceph-mds[64151]: -- 172.21.15.96:0/330198798 <== mon.2 v2:172.21.15.96:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x556f58487a40 con 0x556f5845b000 2024-05-08T05:40:57.279 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h[64147]: starting mds.h at 2024-05-08T05:40:57.279 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k[63518]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.279 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k[63518]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.279 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:40:56 smithi096 ceph-mds[63522]: -- 172.21.15.96:0/1741095399 <== mon.0 v2:172.21.15.59:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x557885c97a40 con 0x557885c6a800 2024-05-08T05:40:57.280 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k[63518]: starting mds.k at 2024-05-08T05:40:57.280 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b[64457]: ignoring --setuser ceph since I am not root 2024-05-08T05:40:57.280 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b[64457]: ignoring --setgroup ceph since I am not root 2024-05-08T05:40:57.280 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:40:56 smithi096 ceph-mds[64461]: -- 172.21.15.96:0/186321554 <== mon.1 v2:172.21.15.143:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e70b7ffa40 con 0x55e70b7d2800 2024-05-08T05:40:57.280 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:40:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b[64457]: starting mds.b at 2024-05-08T05:40:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:57 smithi143 ceph-mon[34337]: pgmap v355: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1023 B/s wr, 14 op/s 2024-05-08T05:40:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:57 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3891194348' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-05-08T05:40:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:57 smithi143 ceph-mon[34337]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-05-08T05:40:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:57 smithi143 ceph-mon[34337]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-05-08T05:40:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:57 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/3891194348' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-05-08T05:40:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:57 smithi143 ceph-mon[34337]: osdmap e103: 12 total, 12 up, 12 in 2024-05-08T05:40:57.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:57 smithi143 ceph-mon[34337]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-05-08T05:40:57.673 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-05-08T05:40:57.931 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:40:59.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6832/1326223123,v1:172.21.15.96:6833/1326223123] up:boot 2024-05-08T05:40:59.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6832/2305026531,v1:172.21.15.143:6833/2305026531] up:boot 2024-05-08T05:40:59.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.59:6838/121243852,v1:172.21.15.59:6839/121243852] up:boot 2024-05-08T05:40:59.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.59:6840/2171905360,v1:172.21.15.59:6841/2171905360] up:boot 2024-05-08T05:40:59.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6834/3230296011,v1:172.21.15.143:6835/3230296011] up:boot 2024-05-08T05:40:59.071 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6835/3284335372,v1:172.21.15.96:6837/3284335372] up:boot 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6836/3721418287,v1:172.21.15.143:6837/3721418287] up:boot 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6834/2130434307,v1:172.21.15.96:6836/2130434307] up:boot 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.143:6838/2145912216,v1:172.21.15.143:6839/2145912216] up:boot 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: mds.? [v2:172.21.15.96:6838/3716723012,v1:172.21.15.96:6839/3716723012] up:boot 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:40:59.072 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:40:59.073 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:40:58 smithi059 ceph-mon[30304]: pgmap v357: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-08T05:40:59.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6832/1326223123,v1:172.21.15.96:6833/1326223123] up:boot 2024-05-08T05:40:59.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6832/2305026531,v1:172.21.15.143:6833/2305026531] up:boot 2024-05-08T05:40:59.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.59:6838/121243852,v1:172.21.15.59:6839/121243852] up:boot 2024-05-08T05:40:59.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.59:6840/2171905360,v1:172.21.15.59:6841/2171905360] up:boot 2024-05-08T05:40:59.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6834/3230296011,v1:172.21.15.143:6835/3230296011] up:boot 2024-05-08T05:40:59.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6835/3284335372,v1:172.21.15.96:6837/3284335372] up:boot 2024-05-08T05:40:59.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6836/3721418287,v1:172.21.15.143:6837/3721418287] up:boot 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6834/2130434307,v1:172.21.15.96:6836/2130434307] up:boot 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.143:6838/2145912216,v1:172.21.15.143:6839/2145912216] up:boot 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: mds.? [v2:172.21.15.96:6838/3716723012,v1:172.21.15.96:6839/3716723012] up:boot 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:40:59.244 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:40:58 smithi143 ceph-mon[34337]: pgmap v357: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6832/1326223123,v1:172.21.15.96:6833/1326223123] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6832/2305026531,v1:172.21.15.143:6833/2305026531] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.59:6838/121243852,v1:172.21.15.59:6839/121243852] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.59:6840/2171905360,v1:172.21.15.59:6841/2171905360] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6834/3230296011,v1:172.21.15.143:6835/3230296011] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6835/3284335372,v1:172.21.15.96:6837/3284335372] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6836/3721418287,v1:172.21.15.143:6837/3721418287] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6834/2130434307,v1:172.21.15.96:6836/2130434307] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.143:6838/2145912216,v1:172.21.15.143:6839/2145912216] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: mds.? [v2:172.21.15.96:6838/3716723012,v1:172.21.15.96:6839/3716723012] up:boot 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: fsmap cephfs:0/5 12 up:standby, 5 failed 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-05-08T05:40:59.280 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-05-08T05:40:59.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-05-08T05:40:59.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-05-08T05:40:59.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-05-08T05:40:59.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-05-08T05:40:59.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: from='mgr.24962 172.21.15.59:0/613024770' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-05-08T05:40:59.281 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:40:58 smithi096 ceph-mon[34253]: pgmap v357: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-05-08T05:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.692+0000 7fd9839e0700 1 -- 172.21.15.59:0/682640306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c06bb40 msgr2=0x7fd97c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.692+0000 7fd9839e0700 1 --2- 172.21.15.59:0/682640306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c06bb40 0x7fd97c110330 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7fd974009990 tx=0x7fd974009ca0 comp rx=0 tx=0).stop 2024-05-08T05:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.692+0000 7fd9839e0700 1 -- 172.21.15.59:0/682640306 shutdown_connections 2024-05-08T05:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.692+0000 7fd9839e0700 1 --2- 172.21.15.59:0/682640306 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c06bb40 0x7fd97c110330 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.692+0000 7fd9839e0700 1 --2- 172.21.15.59:0/682640306 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd97c109510 0x7fd97c1098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.692+0000 7fd9839e0700 1 --2- 172.21.15.59:0/682640306 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd97c06b180 0x7fd97c06b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.692+0000 7fd9839e0700 1 -- 172.21.15.59:0/682640306 >> 172.21.15.59:0/682640306 conn(0x7fd97c079350 msgr2=0x7fd97c079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.692+0000 7fd9839e0700 1 -- 172.21.15.59:0/682640306 shutdown_connections 2024-05-08T05:40:59.694 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.692+0000 7fd9839e0700 1 -- 172.21.15.59:0/682640306 wait complete. 2024-05-08T05:40:59.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd9839e0700 1 Processor -- start 2024-05-08T05:40:59.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd9839e0700 1 -- start start 2024-05-08T05:40:59.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd9839e0700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd97c06b180 0x7fd97c102470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:59.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd9839e0700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd97c06bb40 0x7fd97c1029b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:59.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd9839e0700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c109510 0x7fd97c102ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:59.696 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd9839e0700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c113640 con 0x7fd97c109510 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd9839e0700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1134c0 con 0x7fd97c06b180 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd9839e0700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fd97c1137c0 con 0x7fd97c06bb40 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd981f7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c109510 0x7fd97c102ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd981f7d700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c109510 0x7fd97c102ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40712/0 (socket says 172.21.15.59:40712) 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd981f7d700 1 -- 172.21.15.59:0/1669055261 learned_addr learned my addr 172.21.15.59:0/1669055261 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd98177c700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd97c06b180 0x7fd97c102470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.693+0000 7fd980f7b700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd97c06bb40 0x7fd97c1029b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd981f7d700 1 -- 172.21.15.59:0/1669055261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd97c06bb40 msgr2=0x7fd97c1029b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd981f7d700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd97c06bb40 0x7fd97c1029b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd981f7d700 1 -- 172.21.15.59:0/1669055261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd97c06b180 msgr2=0x7fd97c102470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd981f7d700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd97c06b180 0x7fd97c102470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd981f7d700 1 -- 172.21.15.59:0/1669055261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd97c1a8010 con 0x7fd97c109510 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd981f7d700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c109510 0x7fd97c102ef0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd9740057c0 tx=0x7fd9740057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:59.697 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd9727fc700 1 -- 172.21.15.59:0/1669055261 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974027070 con 0x7fd97c109510 2024-05-08T05:40:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd97c1a82a0 con 0x7fd97c109510 2024-05-08T05:40:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd9727fc700 1 -- 172.21.15.59:0/1669055261 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd974003680 con 0x7fd97c109510 2024-05-08T05:40:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd9727fc700 1 -- 172.21.15.59:0/1669055261 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd974022b40 con 0x7fd97c109510 2024-05-08T05:40:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.694+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd97c1a8750 con 0x7fd97c109510 2024-05-08T05:40:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.695+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd97c06a190 con 0x7fd97c109510 2024-05-08T05:40:59.698 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.697+0000 7fd9727fc700 1 -- 172.21.15.59:0/1669055261 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fd97402a070 con 0x7fd97c109510 2024-05-08T05:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.697+0000 7fd9727fc700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fd9680b29f0 0x7fd9680b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.697+0000 7fd98177c700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fd9680b29f0 0x7fd9680b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.697+0000 7fd9727fc700 1 -- 172.21.15.59:0/1669055261 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9838+0+0 (secure 0 0 0) 0x7fd9740e4040 con 0x7fd97c109510 2024-05-08T05:40:59.699 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.698+0000 7fd98177c700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fd9680b29f0 0x7fd9680b4eb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd978007fd0 tx=0x7fd97800c040 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:40:59.702 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.701+0000 7fd9727fc700 1 -- 172.21.15.59:0/1669055261 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fd974031020 con 0x7fd97c109510 2024-05-08T05:40:59.963 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:40:59.962+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7fd97c02e510 con 0x7fd97c109510 2024-05-08T05:41:00.015 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.014+0000 7fd9727fc700 1 -- 172.21.15.59:0/1669055261 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v44) ==== 104+0+0 (secure 0 0 0) 0x7fd974013070 con 0x7fd97c109510 2024-05-08T05:41:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fd9680b29f0 msgr2=0x7fd9680b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fd9680b29f0 0x7fd9680b4eb0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fd978007fd0 tx=0x7fd97800c040 comp rx=0 tx=0).stop 2024-05-08T05:41:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c109510 msgr2=0x7fd97c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c109510 0x7fd97c102ef0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd9740057c0 tx=0x7fd9740057f0 comp rx=0 tx=0).stop 2024-05-08T05:41:00.018 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 shutdown_connections 2024-05-08T05:41:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fd9680b29f0 0x7fd9680b4eb0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fd97c109510 0x7fd97c102ef0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fd97c06b180 0x7fd97c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 --2- 172.21.15.59:0/1669055261 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fd97c06bb40 0x7fd97c1029b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.016+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 >> 172.21.15.59:0/1669055261 conn(0x7fd97c079350 msgr2=0x7fd97c1010b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:00.019 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.017+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 shutdown_connections 2024-05-08T05:41:00.020 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:00.017+0000 7fd9839e0700 1 -- 172.21.15.59:0/1669055261 wait complete. 2024-05-08T05:41:00.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:00 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1669055261' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:41:00.336 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:00 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1669055261' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:41:00.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:00 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1669055261' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-05-08T05:41:00.747 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-05-08T05:41:01.013 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:41:01.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:01 smithi059 ceph-mon[30304]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T05:41:01.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:01 smithi059 ceph-mon[30304]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T05:41:01.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:01 smithi059 ceph-mon[30304]: Cluster is now healthy 2024-05-08T05:41:01.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:01 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1669055261' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:41:01.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:01 smithi059 ceph-mon[30304]: fsmap 12 up:standby 2024-05-08T05:41:01.273 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:01 smithi059 ceph-mon[30304]: pgmap v358: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-08T05:41:01.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:01 smithi096 ceph-mon[34253]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T05:41:01.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:01 smithi096 ceph-mon[34253]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T05:41:01.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:01 smithi096 ceph-mon[34253]: Cluster is now healthy 2024-05-08T05:41:01.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:01 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1669055261' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:41:01.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:01 smithi096 ceph-mon[34253]: fsmap 12 up:standby 2024-05-08T05:41:01.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:01 smithi096 ceph-mon[34253]: pgmap v358: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-08T05:41:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:01 smithi143 ceph-mon[34337]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-05-08T05:41:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:01 smithi143 ceph-mon[34337]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-05-08T05:41:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:01 smithi143 ceph-mon[34337]: Cluster is now healthy 2024-05-08T05:41:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:01 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1669055261' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-05-08T05:41:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:01 smithi143 ceph-mon[34337]: fsmap 12 up:standby 2024-05-08T05:41:01.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:01 smithi143 ceph-mon[34337]: pgmap v358: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-08T05:41:02.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.760+0000 7fa6c534e700 1 -- 172.21.15.59:0/3264817259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103f00 msgr2=0x7fa6c01103b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:02.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.760+0000 7fa6c534e700 1 --2- 172.21.15.59:0/3264817259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103f00 0x7fa6c01103b0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7fa6b80099c0 tx=0x7fa6b8009cd0 comp rx=0 tx=0).stop 2024-05-08T05:41:02.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.760+0000 7fa6c534e700 1 -- 172.21.15.59:0/3264817259 shutdown_connections 2024-05-08T05:41:02.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.760+0000 7fa6c534e700 1 --2- 172.21.15.59:0/3264817259 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103f00 0x7fa6c01103b0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:02.763 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.760+0000 7fa6c534e700 1 --2- 172.21.15.59:0/3264817259 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6c01095e0 0x7fa6c01099c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.760+0000 7fa6c534e700 1 --2- 172.21.15.59:0/3264817259 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6c0103540 0x7fa6c01039c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.760+0000 7fa6c534e700 1 -- 172.21.15.59:0/3264817259 >> 172.21.15.59:0/3264817259 conn(0x7fa6c007a650 msgr2=0x7fa6c007aa60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.760+0000 7fa6c534e700 1 -- 172.21.15.59:0/3264817259 shutdown_connections 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.760+0000 7fa6c534e700 1 -- 172.21.15.59:0/3264817259 wait complete. 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.761+0000 7fa6c534e700 1 Processor -- start 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.761+0000 7fa6c534e700 1 -- start start 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.761+0000 7fa6c534e700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103540 0x7fa6c00757a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.761+0000 7fa6c534e700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6c0103f00 0x7fa6c0075ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.761+0000 7fa6c534e700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6c01095e0 0x7fa6c0076220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.761+0000 7fa6c534e700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c0113740 con 0x7fa6c0103540 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.761+0000 7fa6c534e700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01135c0 con 0x7fa6c01095e0 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.761+0000 7fa6c534e700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fa6c01138c0 con 0x7fa6c0103f00 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6beffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103540 0x7fa6c00757a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:02.764 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6beffd700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103540 0x7fa6c00757a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40728/0 (socket says 172.21.15.59:40728) 2024-05-08T05:41:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6beffd700 1 -- 172.21.15.59:0/2436747247 learned_addr learned my addr 172.21.15.59:0/2436747247 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:41:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6bf7fe700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6c01095e0 0x7fa6c0076220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6be7fc700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6c0103f00 0x7fa6c0075ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6beffd700 1 -- 172.21.15.59:0/2436747247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6c0103f00 msgr2=0x7fa6c0075ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6beffd700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6c0103f00 0x7fa6c0075ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6beffd700 1 -- 172.21.15.59:0/2436747247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6c01095e0 msgr2=0x7fa6c0076220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6beffd700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6c01095e0 0x7fa6c0076220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:02.765 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6beffd700 1 -- 172.21.15.59:0/2436747247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6c01a7fc0 con 0x7fa6c0103540 2024-05-08T05:41:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6beffd700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103540 0x7fa6c00757a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00060b0 tx=0x7fa6b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.762+0000 7fa6a7fff700 1 -- 172.21.15.59:0/2436747247 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b001a070 con 0x7fa6c0103540 2024-05-08T05:41:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.763+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6c01a82b0 con 0x7fa6c0103540 2024-05-08T05:41:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.763+0000 7fa6a7fff700 1 -- 172.21.15.59:0/2436747247 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6b000dd20 con 0x7fa6c0103540 2024-05-08T05:41:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.763+0000 7fa6a7fff700 1 -- 172.21.15.59:0/2436747247 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6b0014600 con 0x7fa6c0103540 2024-05-08T05:41:02.766 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.763+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6c01a8840 con 0x7fa6c0103540 2024-05-08T05:41:02.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.764+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa6ac005330 con 0x7fa6c0103540 2024-05-08T05:41:02.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.765+0000 7fa6a7fff700 1 -- 172.21.15.59:0/2436747247 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fa6b001d070 con 0x7fa6c0103540 2024-05-08T05:41:02.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.766+0000 7fa6a7fff700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa6a80b29f0 0x7fa6a80b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:02.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.766+0000 7fa6be7fc700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa6a80b29f0 0x7fa6a80b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:02.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.766+0000 7fa6a7fff700 1 -- 172.21.15.59:0/2436747247 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9838+0+0 (secure 0 0 0) 0x7fa6b00e2200 con 0x7fa6c0103540 2024-05-08T05:41:02.767 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.766+0000 7fa6be7fc700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa6a80b29f0 0x7fa6a80b4eb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6c007abc0 tx=0x7fa6b40073d0 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:02.772 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.770+0000 7fa6a7fff700 1 -- 172.21.15.59:0/2436747247 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fa6b0010070 con 0x7fa6c0103540 2024-05-08T05:41:02.985 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:02.983+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa6ac005620 con 0x7fa6c0103540 2024-05-08T05:41:03.109 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.108+0000 7fa6a7fff700 1 -- 172.21.15.59:0/2436747247 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v104) ==== 181+0+0 (secure 0 0 0) 0x7fa6b00a9660 con 0x7fa6c0103540 2024-05-08T05:41:03.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa6a80b29f0 msgr2=0x7fa6a80b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:03.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa6a80b29f0 0x7fa6a80b4eb0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fa6c007abc0 tx=0x7fa6b40073d0 comp rx=0 tx=0).stop 2024-05-08T05:41:03.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103540 msgr2=0x7fa6c00757a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:03.113 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103540 0x7fa6c00757a0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7fa6b00060b0 tx=0x7fa6b000d3b0 comp rx=0 tx=0).stop 2024-05-08T05:41:03.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 shutdown_connections 2024-05-08T05:41:03.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fa6a80b29f0 0x7fa6a80b4eb0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:03.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fa6c01095e0 0x7fa6c0076220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:03.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fa6c0103540 0x7fa6c00757a0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:03.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 --2- 172.21.15.59:0/2436747247 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fa6c0103f00 0x7fa6c0075ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:03.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.112+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 >> 172.21.15.59:0/2436747247 conn(0x7fa6c007a650 msgr2=0x7fa6c0102050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:03.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.113+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 shutdown_connections 2024-05-08T05:41:03.114 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:03.113+0000 7fa6c534e700 1 -- 172.21.15.59:0/2436747247 wait complete. 2024-05-08T05:41:03.115 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_metadata' removed 2024-05-08T05:41:03.429 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:03 smithi059 ceph-mon[30304]: pgmap v359: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-08T05:41:03.430 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:03 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2436747247' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T05:41:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:03 smithi143 ceph-mon[34337]: pgmap v359: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-08T05:41:03.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:03 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2436747247' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T05:41:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:03 smithi096 ceph-mon[34253]: pgmap v359: 129 pgs: 129 active+clean; 117 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail; 3.4 KiB/s rd, 6 op/s 2024-05-08T05:41:03.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:03 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2436747247' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T05:41:03.743 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-05-08T05:41:03.997 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:41:04.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:04 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2436747247' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T05:41:04.410 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:04 smithi059 ceph-mon[30304]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T05:41:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:04 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2436747247' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T05:41:04.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:04 smithi143 ceph-mon[34337]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T05:41:04.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:04 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2436747247' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T05:41:04.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:04 smithi096 ceph-mon[34253]: osdmap e104: 12 total, 12 up, 12 in 2024-05-08T05:41:05.418 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:05 smithi059 ceph-mon[30304]: pgmap v361: 97 pgs: 97 active+clean; 105 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:05.493 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:05 smithi143 ceph-mon[34337]: pgmap v361: 97 pgs: 97 active+clean; 105 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:05.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:05 smithi096 ceph-mon[34253]: pgmap v361: 97 pgs: 97 active+clean; 105 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.644+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2191015828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48108850 msgr2=0x7fdf481103d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.644+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2191015828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48108850 0x7fdf481103d0 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000b0d0 tx=0x7fdf4000b3e0 comp rx=0 tx=0).stop 2024-05-08T05:41:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.644+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2191015828 shutdown_connections 2024-05-08T05:41:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.644+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2191015828 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48108850 0x7fdf481103d0 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.644+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2191015828 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf48103750 0x7fdf48103b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.644+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2191015828 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48104100 0x7fdf48108120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.644+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2191015828 >> 172.21.15.59:0/2191015828 conn(0x7fdf4807a600 msgr2=0x7fdf4807aa10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.644+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2191015828 shutdown_connections 2024-05-08T05:41:05.648 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.644+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2191015828 wait complete. 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf4e8a3700 1 Processor -- start 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf4e8a3700 1 -- start start 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf4e8a3700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48103750 0x7fdf4819f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf4e8a3700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48104100 0x7fdf4819fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf4e8a3700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf48108850 0x7fdf48199890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf4e8a3700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7fdf48113670 con 0x7fdf48104100 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf4e8a3700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7fdf481134f0 con 0x7fdf48103750 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf4e8a3700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7fdf481137f0 con 0x7fdf48108850 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf3ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48104100 0x7fdf4819fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf3ffff700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48104100 0x7fdf4819fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:40738/0 (socket says 172.21.15.59:40738) 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.645+0000 7fdf3ffff700 1 -- 172.21.15.59:0/2826684784 learned_addr learned my addr 172.21.15.59:0/2826684784 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf47fff700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48103750 0x7fdf4819f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:05.649 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf4ce40700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf48108850 0x7fdf48199890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf3ffff700 1 -- 172.21.15.59:0/2826684784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf48108850 msgr2=0x7fdf48199890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf3ffff700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf48108850 0x7fdf48199890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf3ffff700 1 -- 172.21.15.59:0/2826684784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48103750 msgr2=0x7fdf4819f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf3ffff700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48103750 0x7fdf4819f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf3ffff700 1 -- 172.21.15.59:0/2826684784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf48199ff0 con 0x7fdf48104100 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf3ffff700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48104100 0x7fdf4819fc10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000c920 tx=0x7fdf3000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf45ffb700 1 -- 172.21.15.59:0/2826684784 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30004a30 con 0x7fdf48104100 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf45ffb700 1 -- 172.21.15.59:0/2826684784 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf300054d0 con 0x7fdf48104100 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf45ffb700 1 -- 172.21.15.59:0/2826684784 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf30007c70 con 0x7fdf48104100 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4819a2e0 con 0x7fdf48104100 2024-05-08T05:41:05.650 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.646+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf481a65a0 con 0x7fdf48104100 2024-05-08T05:41:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.647+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf4806a190 con 0x7fdf48104100 2024-05-08T05:41:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.649+0000 7fdf45ffb700 1 -- 172.21.15.59:0/2826684784 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7fdf30004c60 con 0x7fdf48104100 2024-05-08T05:41:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.649+0000 7fdf45ffb700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdf340b29f0 0x7fdf340b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.650+0000 7fdf47fff700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdf340b29f0 0x7fdf340b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:05.651 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.650+0000 7fdf45ffb700 1 -- 172.21.15.59:0/2826684784 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9284+0+0 (secure 0 0 0) 0x7fdf300e27e0 con 0x7fdf48104100 2024-05-08T05:41:05.652 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.650+0000 7fdf47fff700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdf340b29f0 0x7fdf340b4eb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf4819a9d0 tx=0x7fdf3800a380 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:05.655 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.654+0000 7fdf45ffb700 1 -- 172.21.15.59:0/2826684784 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7fdf30014030 con 0x7fdf48104100 2024-05-08T05:41:05.873 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:05.871+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fdf4802e510 con 0x7fdf48104100 2024-05-08T05:41:06.742 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.741+0000 7fdf45ffb700 1 -- 172.21.15.59:0/2826684784 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v105) ==== 169+0+0 (secure 0 0 0) 0x7fdf300a9ec0 con 0x7fdf48104100 2024-05-08T05:41:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdf340b29f0 msgr2=0x7fdf340b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdf340b29f0 0x7fdf340b4eb0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fdf4819a9d0 tx=0x7fdf3800a380 comp rx=0 tx=0).stop 2024-05-08T05:41:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48104100 msgr2=0x7fdf4819fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48104100 0x7fdf4819fc10 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7fdf3000c920 tx=0x7fdf3000cc30 comp rx=0 tx=0).stop 2024-05-08T05:41:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 shutdown_connections 2024-05-08T05:41:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7fdf340b29f0 0x7fdf340b4eb0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7fdf48108850 0x7fdf48199890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:06.747 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7fdf48103750 0x7fdf4819f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 --2- 172.21.15.59:0/2826684784 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7fdf48104100 0x7fdf4819fc10 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.745+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 >> 172.21.15.59:0/2826684784 conn(0x7fdf4807a600 msgr2=0x7fdf481014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.746+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 shutdown_connections 2024-05-08T05:41:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:06.746+0000 7fdf4e8a3700 1 -- 172.21.15.59:0/2826684784 wait complete. 2024-05-08T05:41:06.748 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data' removed 2024-05-08T05:41:06.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:06 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2826684784' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T05:41:06.994 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:06 smithi143 ceph-mon[34337]: pgmap v362: 97 pgs: 97 active+clean; 105 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:07.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:06 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2826684784' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T05:41:07.030 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:06 smithi096 ceph-mon[34253]: pgmap v362: 97 pgs: 97 active+clean; 105 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:07.073 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:06 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2826684784' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T05:41:07.073 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:06 smithi059 ceph-mon[30304]: pgmap v362: 97 pgs: 97 active+clean; 105 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:07.533 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a shell --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 -- ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-05-08T05:41:07.792 INFO:teuthology.orchestra.run.smithi059.stderr:Inferring config /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/mon.a/config 2024-05-08T05:41:07.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:07 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/2826684784' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T05:41:07.836 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:07 smithi059 ceph-mon[30304]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T05:41:08.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:07 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/2826684784' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T05:41:08.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:07 smithi096 ceph-mon[34253]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T05:41:08.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:07 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/2826684784' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T05:41:08.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:07 smithi143 ceph-mon[34337]: osdmap e105: 12 total, 12 up, 12 in 2024-05-08T05:41:08.907 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:08 smithi059 ceph-mon[30304]: pgmap v364: 33 pgs: 33 active+clean; 105 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:09.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:08 smithi096 ceph-mon[34253]: pgmap v364: 33 pgs: 33 active+clean; 105 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:09.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:08 smithi143 ceph-mon[34337]: pgmap v364: 33 pgs: 33 active+clean; 105 MiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/605511414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c81095c0 msgr2=0x7ff7c81099a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/605511414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c81095c0 0x7ff7c81099a0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7ff7c40099c0 tx=0x7ff7c4009cd0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/605511414 shutdown_connections 2024-05-08T05:41:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/605511414 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c806c5b0 0x7ff7c8110350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/605511414 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c81095c0 0x7ff7c81099a0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.453 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/605511414 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c806bbf0 0x7ff7c806c070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/605511414 >> 172.21.15.59:0/605511414 conn(0x7ff7c8079260 msgr2=0x7ff7c8079670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/605511414 shutdown_connections 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/605511414 wait complete. 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.450+0000 7ff7cfeb5700 1 Processor -- start 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cfeb5700 1 -- start start 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cfeb5700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c806bbf0 0x7ff7c8102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cfeb5700 1 --2- >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c806c5b0 0x7ff7c8102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cfeb5700 1 --2- >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c81095c0 0x7ff7c81031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cfeb5700 1 -- --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8113680 con 0x7ff7c806bbf0 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cfeb5700 1 -- --> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8113500 con 0x7ff7c81095c0 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cfeb5700 1 -- --> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] -- mon_getmap magic: 0 -- 0x7ff7c8113800 con 0x7ff7c806c5b0 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cdc51700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c806bbf0 0x7ff7c8102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cdc51700 1 --2- >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c806bbf0 0x7ff7c8102750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.59:3300/0 says I am v2:172.21.15.59:53506/0 (socket says 172.21.15.59:53506) 2024-05-08T05:41:09.454 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cdc51700 1 -- 172.21.15.59:0/1099913070 learned_addr learned my addr 172.21.15.59:0/1099913070 (peer_addr_for_me v2:172.21.15.59:0/0) 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cd450700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c806c5b0 0x7ff7c8102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7ce452700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c81095c0 0x7ff7c81031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cdc51700 1 -- 172.21.15.59:0/1099913070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c806c5b0 msgr2=0x7ff7c8102c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cdc51700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c806c5b0 0x7ff7c8102c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cdc51700 1 -- 172.21.15.59:0/1099913070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c81095c0 msgr2=0x7ff7c81031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cdc51700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c81095c0 0x7ff7c81031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cdc51700 1 -- 172.21.15.59:0/1099913070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7c81a80e0 con 0x7ff7c806bbf0 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.451+0000 7ff7cdc51700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c806bbf0 0x7ff7c8102750 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b580 tx=0x7ff7c40037c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.452+0000 7ff7beffd700 1 -- 172.21.15.59:0/1099913070 <== mon.0 v2:172.21.15.59:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c4027070 con 0x7ff7c806bbf0 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.452+0000 7ff7beffd700 1 -- 172.21.15.59:0/1099913070 <== mon.0 v2:172.21.15.59:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff7c4004110 con 0x7ff7c806bbf0 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.452+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7c81a8370 con 0x7ff7c806bbf0 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.452+0000 7ff7beffd700 1 -- 172.21.15.59:0/1099913070 <== mon.0 v2:172.21.15.59:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7c400f460 con 0x7ff7c806bbf0 2024-05-08T05:41:09.455 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.452+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7c81a87c0 con 0x7ff7c806bbf0 2024-05-08T05:41:09.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.453+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7c806a190 con 0x7ff7c806bbf0 2024-05-08T05:41:09.456 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.455+0000 7ff7beffd700 1 -- 172.21.15.59:0/1099913070 <== mon.0 v2:172.21.15.59:3300/0 4 ==== mgrmap(e 26) ==== 149783+0+0 (secure 0 0 0) 0x7ff7c402a070 con 0x7ff7c806bbf0 2024-05-08T05:41:09.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.456+0000 7ff7beffd700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff7b40b29f0 0x7ff7b40b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-05-08T05:41:09.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.456+0000 7ff7cd450700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff7b40b29f0 0x7ff7b40b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-05-08T05:41:09.457 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.456+0000 7ff7beffd700 1 -- 172.21.15.59:0/1099913070 <== mon.0 v2:172.21.15.59:3300/0 5 ==== osd_map(105..105 src has 1..105) ==== 8754+0+0 (secure 0 0 0) 0x7ff7c40e3b60 con 0x7ff7c806bbf0 2024-05-08T05:41:09.458 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.457+0000 7ff7cd450700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff7b40b29f0 0x7ff7b40b4eb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005fd0 tx=0x7ff7b800a560 comp rx=0 tx=0).ready entity=mgr.24962 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-05-08T05:41:09.461 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.460+0000 7ff7beffd700 1 -- 172.21.15.59:0/1099913070 <== mon.0 v2:172.21.15.59:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+191963 (secure 0 0 0) 0x7ff7c4031070 con 0x7ff7c806bbf0 2024-05-08T05:41:09.675 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.674+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 --> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7ff7c802e510 con 0x7ff7c806bbf0 2024-05-08T05:41:09.759 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.757+0000 7ff7beffd700 1 -- 172.21.15.59:0/1099913070 <== mon.0 v2:172.21.15.59:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v106) ==== 178+0+0 (secure 0 0 0) 0x7ff7c4013070 con 0x7ff7c806bbf0 2024-05-08T05:41:09.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.759+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff7b40b29f0 msgr2=0x7ff7b40b4eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff7b40b29f0 0x7ff7b40b4eb0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7ff7b8005fd0 tx=0x7ff7b800a560 comp rx=0 tx=0).stop 2024-05-08T05:41:09.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c806bbf0 msgr2=0x7ff7c8102750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-05-08T05:41:09.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c806bbf0 0x7ff7c8102750 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7ff7c400b580 tx=0x7ff7c40037c0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 shutdown_connections 2024-05-08T05:41:09.761 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:6800/1895772199,v1:172.21.15.59:6801/1895772199] conn(0x7ff7b40b29f0 0x7ff7b40b4eb0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.96:3300/0,v1:172.21.15.96:6789/0] conn(0x7ff7c81095c0 0x7ff7c81031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.59:3300/0,v1:172.21.15.59:6789/0] conn(0x7ff7c806bbf0 0x7ff7c8102750 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 --2- 172.21.15.59:0/1099913070 >> [v2:172.21.15.143:3300/0,v1:172.21.15.143:6789/0] conn(0x7ff7c806c5b0 0x7ff7c8102c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 >> 172.21.15.59:0/1099913070 conn(0x7ff7c8079260 msgr2=0x7ff7c8101300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 shutdown_connections 2024-05-08T05:41:09.762 INFO:teuthology.orchestra.run.smithi059.stderr:2024-05-08T05:41:09.760+0000 7ff7cfeb5700 1 -- 172.21.15.59:0/1099913070 wait complete. 2024-05-08T05:41:09.763 INFO:teuthology.orchestra.run.smithi059.stderr:pool 'cephfs_data_ec' removed 2024-05-08T05:41:10.029 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:09 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1099913070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T05:41:10.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:09 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1099913070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T05:41:10.243 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:09 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1099913070' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-05-08T05:41:10.467 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:41:10.498 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:41:10.530 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:41:10.568 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-05-08T05:41:10.568 DEBUG:teuthology.orchestra.run.smithi059:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T05:41:10.589 DEBUG:teuthology.orchestra.run.smithi096:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T05:41:10.606 DEBUG:teuthology.orchestra.run.smithi143:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-05-08T05:41:10.625 INFO:tasks.cephadm:Stopping all daemons... 2024-05-08T05:41:10.625 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-05-08T05:41:10.625 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.a 2024-05-08T05:41:10.767 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:10 smithi059 systemd[1]: Stopping Ceph mon.a for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:10.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:10 smithi096 ceph-mon[34253]: from='client.? 172.21.15.59:0/1099913070' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T05:41:10.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:10 smithi096 ceph-mon[34253]: osdmap e106: 12 total, 12 up, 12 in 2024-05-08T05:41:10.779 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:10 smithi096 ceph-mon[34253]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:10 smithi143 ceph-mon[34337]: from='client.? 172.21.15.59:0/1099913070' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T05:41:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:10 smithi143 ceph-mon[34337]: osdmap e106: 12 total, 12 up, 12 in 2024-05-08T05:41:10.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:10 smithi143 ceph-mon[34337]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:11.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:10 smithi059 ceph-mon[30304]: from='client.? 172.21.15.59:0/1099913070' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]': finished 2024-05-08T05:41:11.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:10 smithi059 ceph-mon[30304]: osdmap e106: 12 total, 12 up, 12 in 2024-05-08T05:41:11.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:10 smithi059 ceph-mon[30304]: pgmap v366: 1 pgs: 1 active+clean; 577 KiB data, 3.0 GiB used, 1.0 TiB / 1.0 TiB avail 2024-05-08T05:41:11.086 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a[30300]: 2024-05-08T05:41:10.977+0000 7fd47097c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:11.087 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a[30300]: 2024-05-08T05:41:10.977+0000 7fd47097c700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-05-08T05:41:11.421 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:11 smithi059 podman[126142]: 2024-05-08 05:41:11.152474523 +0000 UTC m=+0.294274915 container died cc0f25ef8e855159af26d490b10ca789e8fb78095b2286346a16f8fa17343dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:41:11.788 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:11 smithi059 podman[126142]: 2024-05-08 05:41:11.521117653 +0000 UTC m=+0.662918046 container cleanup cc0f25ef8e855159af26d490b10ca789e8fb78095b2286346a16f8fa17343dec (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:41:11.789 INFO:journalctl@ceph.mon.a.smithi059.stdout:May 08 05:41:11 smithi059 bash[126142]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-a 2024-05-08T05:41:11.901 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.a.service' 2024-05-08T05:41:11.947 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:11.947 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-05-08T05:41:11.947 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-05-08T05:41:11.947 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.b 2024-05-08T05:41:12.274 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:11 smithi096 systemd[1]: Stopping Ceph mon.b for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:12.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-b[34249]: 2024-05-08T05:41:12.273+0000 7f5805ca5700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:12.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-b[34249]: 2024-05-08T05:41:12.273+0000 7f5805ca5700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-05-08T05:41:12.529 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:12 smithi096 podman[70608]: 2024-05-08 05:41:12.508914988 +0000 UTC m=+0.368209829 container died f209783c0e387e9e2c60d5d895dcbde19b4b8c325cf455a3797d6e0651a7a420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-b, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-08T05:41:13.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:12 smithi096 podman[70608]: 2024-05-08 05:41:12.958116764 +0000 UTC m=+0.817411592 container cleanup f209783c0e387e9e2c60d5d895dcbde19b4b8c325cf455a3797d6e0651a7a420 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-b, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:41:13.279 INFO:journalctl@ceph.mon.b.smithi096.stdout:May 08 05:41:12 smithi096 bash[70608]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-b 2024-05-08T05:41:13.425 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.b.service' 2024-05-08T05:41:13.471 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:13.472 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-05-08T05:41:13.472 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-05-08T05:41:13.472 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.c 2024-05-08T05:41:13.993 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:13 smithi143 systemd[1]: Stopping Ceph mon.c for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:14.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:14 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-c[34333]: 2024-05-08T05:41:14.004+0000 7fa85bdc3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:14.431 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:14 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-c[34333]: 2024-05-08T05:41:14.004+0000 7fa85bdc3700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-05-08T05:41:14.432 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:14 smithi143 podman[71853]: 2024-05-08 05:41:14.137606693 +0000 UTC m=+0.468241809 container died 9a75cd2c5c25c6e3ae0e1e3b494882ef8acc2391376fc508b0f9b87caa4d6c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-08T05:41:14.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:14 smithi143 podman[71853]: 2024-05-08 05:41:14.521919435 +0000 UTC m=+0.852554551 container cleanup 9a75cd2c5c25c6e3ae0e1e3b494882ef8acc2391376fc508b0f9b87caa4d6c87 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-c, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-05-08T05:41:14.743 INFO:journalctl@ceph.mon.c.smithi143.stdout:May 08 05:41:14 smithi143 bash[71853]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mon-c 2024-05-08T05:41:15.321 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mon.c.service' 2024-05-08T05:41:15.365 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:15.366 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-05-08T05:41:15.366 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-05-08T05:41:15.366 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.x 2024-05-08T05:41:15.712 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:41:15 smithi059 systemd[1]: Stopping Ceph mgr.x for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:15.993 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:41:15 smithi059 podman[126262]: 2024-05-08 05:41:15.716440266 +0000 UTC m=+0.148882731 container died f894f8c5035ab47b868d6442cea142b266a8665234bcb0d87f234798ba8fdd03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:41:16.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:41:16 smithi059 podman[126262]: 2024-05-08 05:41:16.101440919 +0000 UTC m=+0.533883379 container cleanup f894f8c5035ab47b868d6442cea142b266a8665234bcb0d87f234798ba8fdd03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:bee299f4ac344e4c45e550d7a80abcadd7bc3d5a, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507) 2024-05-08T05:41:16.336 INFO:journalctl@ceph.mgr.x.smithi059.stdout:May 08 05:41:16 smithi059 bash[126262]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-x 2024-05-08T05:41:16.526 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.x.service' 2024-05-08T05:41:16.564 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:16.564 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-05-08T05:41:16.564 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-05-08T05:41:16.564 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.y 2024-05-08T05:41:16.917 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:41:16 smithi096 systemd[1]: Stopping Ceph mgr.y for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:17.206 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:41:16 smithi096 podman[70741]: 2024-05-08 05:41:16.917385254 +0000 UTC m=+0.133464333 container died 02b28c2448fd5d95bd91ab9a3c3d66f6694e147d7ca03673bc7842615414cd9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:41:17.529 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:41:17 smithi096 podman[70741]: 2024-05-08 05:41:17.297597969 +0000 UTC m=+0.513677052 container cleanup 02b28c2448fd5d95bd91ab9a3c3d66f6694e147d7ca03673bc7842615414cd9c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507) 2024-05-08T05:41:17.529 INFO:journalctl@ceph.mgr.y.smithi096.stdout:May 08 05:41:17 smithi096 bash[70741]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-y 2024-05-08T05:41:17.715 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.y.service' 2024-05-08T05:41:17.750 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:17.750 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-05-08T05:41:17.751 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-05-08T05:41:17.751 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.z 2024-05-08T05:41:18.098 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:41:17 smithi143 systemd[1]: Stopping Ceph mgr.z for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:18.428 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:41:18 smithi143 podman[71987]: 2024-05-08 05:41:18.098038847 +0000 UTC m=+0.147969567 container died 8ac990e3137bbebc3c120bb12d0e3de062022b2310a16267e92e9b1e827822ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5) 2024-05-08T05:41:18.743 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:41:18 smithi143 podman[71987]: 2024-05-08 05:41:18.528604518 +0000 UTC m=+0.578535243 container cleanup 8ac990e3137bbebc3c120bb12d0e3de062022b2310a16267e92e9b1e827822ab (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:41:18.743 INFO:journalctl@ceph.mgr.z.smithi143.stdout:May 08 05:41:18 smithi143 bash[71987]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mgr-z 2024-05-08T05:41:18.979 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mgr.z.service' 2024-05-08T05:41:19.015 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:19.015 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-05-08T05:41:19.015 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-05-08T05:41:19.015 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.0 2024-05-08T05:41:19.315 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:19 smithi059 systemd[1]: Stopping Ceph osd.0 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:19.586 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:19 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0[45258]: 2024-05-08T05:41:19.314+0000 7f5773284700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:19.586 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:19 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0[45258]: 2024-05-08T05:41:19.314+0000 7f5773284700 -1 osd.0 106 *** Got signal Terminated *** 2024-05-08T05:41:19.586 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:19 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0[45258]: 2024-05-08T05:41:19.314+0000 7f5773284700 -1 osd.0 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:41:25.057 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:24 smithi059 podman[126384]: 2024-05-08 05:41:24.729086557 +0000 UTC m=+5.516123797 container died bf6fa0040e0f46e2a4b1d4093a207f0c0a97bf941167b4df4133b0d3ade18b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0) 2024-05-08T05:41:25.336 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:25 smithi059 podman[126384]: 2024-05-08 05:41:25.139941743 +0000 UTC m=+5.926978982 container cleanup bf6fa0040e0f46e2a4b1d4093a207f0c0a97bf941167b4df4133b0d3ade18b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True) 2024-05-08T05:41:25.336 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:25 smithi059 bash[126384]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0 2024-05-08T05:41:25.825 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:25 smithi059 podman[126401]: 2024-05-08 05:41:25.358756206 +0000 UTC m=+0.627167198 container remove bf6fa0040e0f46e2a4b1d4093a207f0c0a97bf941167b4df4133b0d3ade18b0c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:41:26.086 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:25 smithi059 podman[126494]: 2024-05-08 05:41:25.726506269 +0000 UTC m=+0.018890423 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:41:26.086 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:25 smithi059 podman[126494]: 2024-05-08 05:41:25.85110003 +0000 UTC m=+0.143484170 container create 113e96d7cbc8e8bee938f5bf7a85f363b53a3fc29b57294f3f1b90b90c8bf67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-05-08T05:41:26.399 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:26 smithi059 podman[126494]: 2024-05-08 05:41:26.126261268 +0000 UTC m=+0.418645429 container init 113e96d7cbc8e8bee938f5bf7a85f363b53a3fc29b57294f3f1b90b90c8bf67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0-deactivate, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-08T05:41:26.399 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:26 smithi059 podman[126494]: 2024-05-08 05:41:26.129942528 +0000 UTC m=+0.422326665 container start 113e96d7cbc8e8bee938f5bf7a85f363b53a3fc29b57294f3f1b90b90c8bf67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0-deactivate, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:41:26.399 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:26 smithi059 podman[126494]: 2024-05-08 05:41:26.229239336 +0000 UTC m=+0.521623480 container attach 113e96d7cbc8e8bee938f5bf7a85f363b53a3fc29b57294f3f1b90b90c8bf67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0-deactivate, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:41:26.832 INFO:journalctl@ceph.osd.0.smithi059.stdout:May 08 05:41:26 smithi059 podman[126494]: 2024-05-08 05:41:26.456105182 +0000 UTC m=+0.748489331 container died 113e96d7cbc8e8bee938f5bf7a85f363b53a3fc29b57294f3f1b90b90c8bf67c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-0-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-08T05:41:27.154 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.0.service' 2024-05-08T05:41:27.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:27.194 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-05-08T05:41:27.195 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-05-08T05:41:27.195 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.1 2024-05-08T05:41:27.529 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:27 smithi096 systemd[1]: Stopping Ceph osd.1 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:28.029 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1[38851]: 2024-05-08T05:41:27.709+0000 7f8931c12700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:28.029 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1[38851]: 2024-05-08T05:41:27.709+0000 7f8931c12700 -1 osd.1 106 *** Got signal Terminated *** 2024-05-08T05:41:28.029 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1[38851]: 2024-05-08T05:41:27.709+0000 7f8931c12700 -1 osd.1 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:41:33.436 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:33 smithi096 podman[70867]: 2024-05-08 05:41:33.151985187 +0000 UTC m=+5.738498739 container died f8890f900aebd0deba01ed2a89ea3b2b4706ec3657689d8659c0d1c3aa34aca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5) 2024-05-08T05:41:33.779 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:33 smithi096 podman[70867]: 2024-05-08 05:41:33.527325684 +0000 UTC m=+6.113839229 container cleanup f8890f900aebd0deba01ed2a89ea3b2b4706ec3657689d8659c0d1c3aa34aca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True) 2024-05-08T05:41:33.779 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:33 smithi096 bash[70867]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1 2024-05-08T05:41:34.148 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:33 smithi096 podman[70884]: 2024-05-08 05:41:33.812366265 +0000 UTC m=+0.657485034 container remove f8890f900aebd0deba01ed2a89ea3b2b4706ec3657689d8659c0d1c3aa34aca8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0) 2024-05-08T05:41:34.441 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:34 smithi096 podman[70973]: 2024-05-08 05:41:34.051524077 +0000 UTC m=+0.018024856 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:41:34.441 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:34 smithi096 podman[70973]: 2024-05-08 05:41:34.180187997 +0000 UTC m=+0.146688758 container create fb026b310db389b2ccdfa604c7fef2e7c0c50523645357fc58f50c0d4788252c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:41:34.441 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:34 smithi096 podman[70973]: 2024-05-08 05:41:34.441537635 +0000 UTC m=+0.408038399 container init fb026b310db389b2ccdfa604c7fef2e7c0c50523645357fc58f50c0d4788252c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-08T05:41:34.733 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:34 smithi096 podman[70973]: 2024-05-08 05:41:34.444757705 +0000 UTC m=+0.411258468 container start fb026b310db389b2ccdfa604c7fef2e7c0c50523645357fc58f50c0d4788252c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:41:34.733 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:34 smithi096 podman[70973]: 2024-05-08 05:41:34.549938046 +0000 UTC m=+0.516438811 container attach fb026b310db389b2ccdfa604c7fef2e7c0c50523645357fc58f50c0d4788252c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1-deactivate, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-08T05:41:35.028 INFO:journalctl@ceph.osd.1.smithi096.stdout:May 08 05:41:34 smithi096 podman[70973]: 2024-05-08 05:41:34.733777656 +0000 UTC m=+0.700278428 container died fb026b310db389b2ccdfa604c7fef2e7c0c50523645357fc58f50c0d4788252c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-1-deactivate, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T05:41:35.388 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.1.service' 2024-05-08T05:41:35.426 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:35.426 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-05-08T05:41:35.426 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-05-08T05:41:35.426 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.2 2024-05-08T05:41:35.733 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:35 smithi143 systemd[1]: Stopping Ceph osd.2 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:35.993 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2[39488]: 2024-05-08T05:41:35.731+0000 7f7d0fbc1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:35.993 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2[39488]: 2024-05-08T05:41:35.731+0000 7f7d0fbc1700 -1 osd.2 106 *** Got signal Terminated *** 2024-05-08T05:41:35.993 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2[39488]: 2024-05-08T05:41:35.731+0000 7f7d0fbc1700 -1 osd.2 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:41:41.492 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:41 smithi143 podman[72110]: 2024-05-08 05:41:41.176297792 +0000 UTC m=+5.562356830 container died 67ed697002e707161e6ca20b68c2a2ca46ce8331d9e0b97af19ba4d2e98adaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:41:41.744 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:41 smithi143 podman[72110]: 2024-05-08 05:41:41.591387352 +0000 UTC m=+5.977446440 container cleanup 67ed697002e707161e6ca20b68c2a2ca46ce8331d9e0b97af19ba4d2e98adaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-05-08T05:41:41.744 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:41 smithi143 bash[72110]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2 2024-05-08T05:41:42.215 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:41 smithi143 podman[72127]: 2024-05-08 05:41:41.817981657 +0000 UTC m=+0.641419474 container remove 67ed697002e707161e6ca20b68c2a2ca46ce8331d9e0b97af19ba4d2e98adaae (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD) 2024-05-08T05:41:42.466 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:42 smithi143 podman[72212]: 2024-05-08 05:41:42.119278836 +0000 UTC m=+0.018290122 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:41:42.467 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:42 smithi143 podman[72212]: 2024-05-08 05:41:42.227113522 +0000 UTC m=+0.126124803 container create d4cd3d69f8e3f8b81553d6295db5ad0d96941b7abfaa8d85091f8abaeb6404a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:41:42.467 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:42 smithi143 podman[72212]: 2024-05-08 05:41:42.463304415 +0000 UTC m=+0.362315691 container init d4cd3d69f8e3f8b81553d6295db5ad0d96941b7abfaa8d85091f8abaeb6404a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux ) 2024-05-08T05:41:42.743 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:42 smithi143 podman[72212]: 2024-05-08 05:41:42.466467435 +0000 UTC m=+0.365478722 container start d4cd3d69f8e3f8b81553d6295db5ad0d96941b7abfaa8d85091f8abaeb6404a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-08T05:41:42.743 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:42 smithi143 podman[72212]: 2024-05-08 05:41:42.579959673 +0000 UTC m=+0.478970956 container attach d4cd3d69f8e3f8b81553d6295db5ad0d96941b7abfaa8d85091f8abaeb6404a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2-deactivate, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:41:43.050 INFO:journalctl@ceph.osd.2.smithi143.stdout:May 08 05:41:42 smithi143 podman[72212]: 2024-05-08 05:41:42.75534967 +0000 UTC m=+0.654360949 container died d4cd3d69f8e3f8b81553d6295db5ad0d96941b7abfaa8d85091f8abaeb6404a8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-2-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:41:43.530 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.2.service' 2024-05-08T05:41:43.572 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:43.572 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-05-08T05:41:43.572 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-05-08T05:41:43.572 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.3 2024-05-08T05:41:43.929 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:43 smithi059 systemd[1]: Stopping Ceph osd.3 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:44.336 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:43 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3[52074]: 2024-05-08T05:41:43.928+0000 7f781726c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:44.336 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:43 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3[52074]: 2024-05-08T05:41:43.928+0000 7f781726c700 -1 osd.3 106 *** Got signal Terminated *** 2024-05-08T05:41:44.336 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:43 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3[52074]: 2024-05-08T05:41:43.928+0000 7f781726c700 -1 osd.3 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:41:45.836 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:45 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3[52074]: 2024-05-08T05:41:45.524+0000 7f780f858700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.734792+0000 front 2024-05-08T05:41:22.734842+0000 (oldest deadline 2024-05-08T05:41:45.035001+0000) 2024-05-08T05:41:46.836 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:46 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3[52074]: 2024-05-08T05:41:46.534+0000 7f780f858700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.734792+0000 front 2024-05-08T05:41:22.734842+0000 (oldest deadline 2024-05-08T05:41:45.035001+0000) 2024-05-08T05:41:46.836 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:46 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:46.420+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:47.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:46 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:46.715+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:47.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:46 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:46.942+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:47.836 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:47 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3[52074]: 2024-05-08T05:41:47.534+0000 7f780f858700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.734792+0000 front 2024-05-08T05:41:22.734842+0000 (oldest deadline 2024-05-08T05:41:45.035001+0000) 2024-05-08T05:41:47.836 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:47 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:47.380+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:48.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:47 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:47.670+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:48.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:47 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:47.900+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:48.836 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:48 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3[52074]: 2024-05-08T05:41:48.553+0000 7f780f858700 -1 osd.3 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.734792+0000 front 2024-05-08T05:41:22.734842+0000 (oldest deadline 2024-05-08T05:41:45.035001+0000) 2024-05-08T05:41:48.836 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:48 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:48.336+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:49.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:48 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:48.650+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:49.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:48.910+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:49.313 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:49 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:49.010+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:49.586 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:49 smithi059 podman[126612]: 2024-05-08 05:41:49.312483496 +0000 UTC m=+5.528985637 container died 4bec9bbb0851089d61c5c1850280a8119d59f07f972fee70d60b0c3286df4667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.33.5) 2024-05-08T05:41:49.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:49 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:49.320+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:50.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:49 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:49.619+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:50.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:49 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:49.881+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:50.329 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:50 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:50.003+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:50.329 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:50 smithi059 podman[126612]: 2024-05-08 05:41:50.069822034 +0000 UTC m=+6.286324173 container cleanup 4bec9bbb0851089d61c5c1850280a8119d59f07f972fee70d60b0c3286df4667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-08T05:41:50.330 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:50 smithi059 bash[126612]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3 2024-05-08T05:41:50.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:50 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:50.341+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:50.586 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:50 smithi059 podman[126627]: 2024-05-08 05:41:50.329578556 +0000 UTC m=+1.015498449 container remove 4bec9bbb0851089d61c5c1850280a8119d59f07f972fee70d60b0c3286df4667 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507) 2024-05-08T05:41:50.743 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:50.457+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:50.967 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:50 smithi059 podman[126722]: 2024-05-08 05:41:50.606566936 +0000 UTC m=+0.018695869 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:41:50.967 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:50 smithi059 podman[126722]: 2024-05-08 05:41:50.730683169 +0000 UTC m=+0.142812087 container create 870f3c287e7f81e92e56e01e0deedde0967840f285732b527af9ae35600227b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-05-08T05:41:50.967 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:50 smithi059 podman[126722]: 2024-05-08 05:41:50.967263622 +0000 UTC m=+0.379392545 container init 870f3c287e7f81e92e56e01e0deedde0967840f285732b527af9ae35600227b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-05-08T05:41:51.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:50 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:50.619+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:51.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:50.888+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:51.277 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:50 smithi059 podman[126722]: 2024-05-08 05:41:50.971276085 +0000 UTC m=+0.383405004 container start 870f3c287e7f81e92e56e01e0deedde0967840f285732b527af9ae35600227b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507) 2024-05-08T05:41:51.277 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:51 smithi059 podman[126722]: 2024-05-08 05:41:51.066861434 +0000 UTC m=+0.478990372 container attach 870f3c287e7f81e92e56e01e0deedde0967840f285732b527af9ae35600227b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3-deactivate, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:41:51.277 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:51 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:51.050+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:51.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:51 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:51.343+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:51.586 INFO:journalctl@ceph.osd.3.smithi059.stdout:May 08 05:41:51 smithi059 podman[126722]: 2024-05-08 05:41:51.277841548 +0000 UTC m=+0.689970462 container died 870f3c287e7f81e92e56e01e0deedde0967840f285732b527af9ae35600227b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:41:51.743 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:51.446+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:51.896 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.3.service' 2024-05-08T05:41:51.941 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:41:51.941 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-05-08T05:41:51.941 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-05-08T05:41:51.942 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.4 2024-05-08T05:41:51.968 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:51 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:51.649+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:52.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:51.885+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:52.249 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:51 smithi096 systemd[1]: Stopping Ceph osd.4 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:41:52.311 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:52 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:52.035+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:52.311 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:52 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:52.310+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:52.529 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:52 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4[45644]: 2024-05-08T05:41:52.248+0000 7f3ed61ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:41:52.529 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:52 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4[45644]: 2024-05-08T05:41:52.248+0000 7f3ed61ab700 -1 osd.4 106 *** Got signal Terminated *** 2024-05-08T05:41:52.529 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:52 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4[45644]: 2024-05-08T05:41:52.248+0000 7f3ed61ab700 -1 osd.4 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:41:52.743 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:52.443+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:53.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:52 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:52.650+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:53.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:52.844+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:53.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:53 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:53.051+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:53.743 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:53 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:53.446+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:53.836 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:53 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:53.346+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:54.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:53 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:53.662+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:54.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:53 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:53.889+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:54.529 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:54 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4[45644]: 2024-05-08T05:41:54.237+0000 7f3ece797700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.596489+0000 front 2024-05-08T05:41:28.596462+0000 (oldest deadline 2024-05-08T05:41:53.896581+0000) 2024-05-08T05:41:54.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:54 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:54.332+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:54.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:54 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:54.097+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:54.743 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:54 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:54.405+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:55.030 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:54 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:54.664+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:55.030 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:54 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:54.664+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:41:55.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:54 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:54.856+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:55.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:55 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:55.059+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:55.529 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:55 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4[45644]: 2024-05-08T05:41:55.274+0000 7f3ece797700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.596489+0000 front 2024-05-08T05:41:28.596462+0000 (oldest deadline 2024-05-08T05:41:53.896581+0000) 2024-05-08T05:41:55.529 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:41:55 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:41:55.246+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:41:55.638 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:55 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:55.357+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:55.743 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:55 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:55.390+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:56.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:55 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:55.693+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:56.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:55 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:55.693+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:41:56.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:55 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:55.887+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:56.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:56.083+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:56.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:56.326+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:56.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:56.326+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:41:56.529 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4[45644]: 2024-05-08T05:41:56.252+0000 7f3ece797700 -1 osd.4 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.596489+0000 front 2024-05-08T05:41:28.596462+0000 (oldest deadline 2024-05-08T05:41:53.896581+0000) 2024-05-08T05:41:56.530 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:41:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:41:56.275+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:41:56.725 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:56.342+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:56.725 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:56.342+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:41:56.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:56.877+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:56.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:56.877+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:41:56.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:41:56 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:41:56.725+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:41:57.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:56.739+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:57.029 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:56 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:56.739+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:41:57.529 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:41:57 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:41:57.276+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:41:57.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:57 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:57.279+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:57.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:57 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:57.279+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:41:57.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:57 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:57.097+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:57.587 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:57 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:57.097+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:41:57.730 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:57 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:57.295+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:57.730 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:57 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:57.295+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:41:57.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:41:57 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:41:57.730+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:41:57.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:57 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:57.895+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:57.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:57 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:57.895+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:41:58.029 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:57 smithi096 podman[71094]: 2024-05-08 05:41:57.776044442 +0000 UTC m=+5.643320789 container died b576282a60ce5fe05bb01700a3a1fd3be1ee31b4f307bc5bcaaacb52e1810507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, ceph=True, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:41:58.030 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:57 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:57.758+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:58.030 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:57 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:57.758+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:41:58.444 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:41:58 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:41:58.310+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:41:58.444 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:58 smithi096 podman[71094]: 2024-05-08 05:41:58.175529308 +0000 UTC m=+6.042805654 container cleanup b576282a60ce5fe05bb01700a3a1fd3be1ee31b4f307bc5bcaaacb52e1810507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-05-08T05:41:58.444 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:58 smithi096 bash[71094]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4 2024-05-08T05:41:58.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:58 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:58.143+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:58.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:58 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:58.143+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:41:58.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:58 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:58.270+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:58.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:58 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:58.270+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:41:58.743 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:58 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:58.308+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:58.743 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:58 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:58.308+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:41:58.779 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:58 smithi096 podman[71114]: 2024-05-08 05:41:58.444758954 +0000 UTC m=+0.666751543 container remove b576282a60ce5fe05bb01700a3a1fd3be1ee31b4f307bc5bcaaacb52e1810507 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5) 2024-05-08T05:41:59.040 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:58 smithi096 podman[71207]: 2024-05-08 05:41:58.691560921 +0000 UTC m=+0.018108832 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:41:59.041 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:58 smithi096 podman[71207]: 2024-05-08 05:41:58.812383866 +0000 UTC m=+0.138931766 container create b678b6a20bb94ae2e42ceae7477a8928797816fccf0e1a277930395bc572a379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:41:59.041 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:58 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:58.784+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:41:59.041 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:58 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:58.784+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:41:59.243 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:41:58 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:41:58.767+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:41:59.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:58 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:58.871+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:59.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:58 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:58.871+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:41:59.321 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:59 smithi096 podman[71207]: 2024-05-08 05:41:59.040607829 +0000 UTC m=+0.367155729 container init b678b6a20bb94ae2e42ceae7477a8928797816fccf0e1a277930395bc572a379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4-deactivate, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:41:59.321 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:59 smithi096 podman[71207]: 2024-05-08 05:41:59.044672065 +0000 UTC m=+0.371219974 container start b678b6a20bb94ae2e42ceae7477a8928797816fccf0e1a277930395bc572a379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.build-date=20240507) 2024-05-08T05:41:59.321 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:59 smithi096 podman[71207]: 2024-05-08 05:41:59.149164761 +0000 UTC m=+0.475712670 container attach b678b6a20bb94ae2e42ceae7477a8928797816fccf0e1a277930395bc572a379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4-deactivate, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True) 2024-05-08T05:41:59.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:59 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:59.099+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:41:59.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:41:59 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:41:59.099+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:41:59.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:59 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:59.225+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:41:59.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:41:59 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:41:59.225+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:41:59.710 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:41:59 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:41:59.319+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:41:59.710 INFO:journalctl@ceph.osd.4.smithi096.stdout:May 08 05:41:59 smithi096 podman[71207]: 2024-05-08 05:41:59.33862533 +0000 UTC m=+0.665173237 container died b678b6a20bb94ae2e42ceae7477a8928797816fccf0e1a277930395bc572a379 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-4-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-05-08T05:41:59.720 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:59 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:59.268+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:41:59.720 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:41:59 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:41:59.268+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:41:59.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:59 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:59.907+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:41:59.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:41:59 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:41:59.907+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:41:59.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:41:59 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:41:59.720+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:00.011 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:59 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:59.835+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:00.011 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:41:59 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:41:59.835+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:00.044 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.4.service' 2024-05-08T05:42:00.121 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:42:00.121 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-05-08T05:42:00.121 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-05-08T05:42:00.121 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.5 2024-05-08T05:42:00.454 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:00 smithi143 systemd[1]: Stopping Ceph osd.5 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:42:00.454 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:00.243+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:42:00.454 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:00.243+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:42:00.454 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:00.453+0000 7f4224144700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:42:00.455 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:00.453+0000 7f4224144700 -1 osd.5 106 *** Got signal Terminated *** 2024-05-08T05:42:00.455 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:00.453+0000 7f4224144700 -1 osd.5 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:42:00.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:00.106+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:00.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:00.106+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:00.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:00.201+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:00.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:00 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:00.201+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:00.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:00 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:00.357+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:01.240 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:00.745+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:01.240 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:00.878+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:01.241 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:00 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:00.878+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:01.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:00 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:00.795+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:01.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:00 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:00.795+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:01.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:01 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:01.197+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:01.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:01 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:01.197+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:01.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:01 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:01.064+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:01.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:01 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:01.064+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:01.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:01 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:01.239+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:42:01.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:01 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:01.239+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:42:01.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:01 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:01.239+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:37.092917+0000 front 2024-05-08T05:41:37.092940+0000 (oldest deadline 2024-05-08T05:42:01.193055+0000) 2024-05-08T05:42:01.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:01 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:01.321+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:02.201 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:01 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:01.753+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:02.202 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:01 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:01.902+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:02.202 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:01 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:01.902+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:02.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:01 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:01.816+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:02.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:01 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:01.816+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:02.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:02.048+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:02.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:02.048+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:02.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:02.235+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:02.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:02.235+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:02.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:02 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:02.201+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:42:02.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:02 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:02.201+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:42:02.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:02 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:02.201+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:37.092917+0000 front 2024-05-08T05:41:37.092940+0000 (oldest deadline 2024-05-08T05:42:01.193055+0000) 2024-05-08T05:42:02.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:02 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:02.300+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:02.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:02 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:02.718+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:02.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:02 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:02.718+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:02.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:02 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:02.852+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:02.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:02 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:02.852+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:03.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:02 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:02.795+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:03.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:02 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:02.795+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:03.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:03 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:03.052+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:03.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:03 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:03.052+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:03.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:03 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:03.263+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:03.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:03 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:03.263+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:03.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:03 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:03.263+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:03.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:03 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:03.207+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:42:03.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:03 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:03.207+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:42:03.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:03 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:03.207+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:37.092917+0000 front 2024-05-08T05:41:37.092940+0000 (oldest deadline 2024-05-08T05:42:01.193055+0000) 2024-05-08T05:42:03.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:03 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:03.327+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:03.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:03 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:03.682+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:03.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:03 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:03.682+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:03.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:03 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:03.872+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:03.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:03 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:03.872+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:04.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:03 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:03.834+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:04.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:03 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:03.834+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:04.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:04.242+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:04.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:04.242+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:04.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:04.242+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:04.337 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:04.020+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:04.337 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:04.020+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:04.337 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:04.020+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:04.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:04 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:04.193+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:42:04.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:04 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:04.193+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:42:04.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:04 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:04.193+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:37.092917+0000 front 2024-05-08T05:41:37.092940+0000 (oldest deadline 2024-05-08T05:42:01.193055+0000) 2024-05-08T05:42:04.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:04 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:04.288+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:04.913 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:04 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:04.654+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:04.913 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:04 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:04.654+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:05.219 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:04 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:04.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:05.219 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:04 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:04.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:05.219 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:04 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:04.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:05.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:04 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:04.854+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:05.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:04 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:04.854+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:05.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:04 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:04.854+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:05.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:05.240+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:05.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:05.240+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:05.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:05.240+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:05.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:05.012+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:05.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:05.012+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:05.337 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:05.012+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:05.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:05 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:05.219+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:23.690837+0000 front 2024-05-08T05:41:23.690851+0000 (oldest deadline 2024-05-08T05:41:49.590873+0000) 2024-05-08T05:42:05.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:05 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:05.219+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.291874+0000 front 2024-05-08T05:41:31.291959+0000 (oldest deadline 2024-05-08T05:41:55.992032+0000) 2024-05-08T05:42:05.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:05 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5[46417]: 2024-05-08T05:42:05.219+0000 7f421c730700 -1 osd.5 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:37.092917+0000 front 2024-05-08T05:41:37.092940+0000 (oldest deadline 2024-05-08T05:42:01.193055+0000) 2024-05-08T05:42:05.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:05 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:05.330+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:05.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:05 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:05.330+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:05.956 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:05 smithi143 podman[72331]: 2024-05-08 05:42:05.877314778 +0000 UTC m=+5.552525535 container died 8054591672b3e6078a00c0503c8fa7ffc47d20c62913514a9bd68824b0281b70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-08T05:42:05.956 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:05 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:05.657+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:05.956 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:05 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:05.657+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:06.241 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:05.973+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:06.241 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:05.973+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:06.241 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:05 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:05.973+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:06.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:05 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:05.955+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:06.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:05 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:05.955+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:06.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:05 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:05.955+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:06.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:05 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:05.878+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:06.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:05 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:05.878+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:06.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:05 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:05.878+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:06.522 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:06 smithi143 podman[72331]: 2024-05-08 05:42:06.254314665 +0000 UTC m=+5.929525414 container cleanup 8054591672b3e6078a00c0503c8fa7ffc47d20c62913514a9bd68824b0281b70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-08T05:42:06.522 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:06 smithi143 bash[72331]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5 2024-05-08T05:42:06.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:06 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:06.240+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:06.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:06 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:06.240+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:06.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:06 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:06.240+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:06.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:06 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:06.337+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:06.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:06 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:06.337+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:06.874 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:06 smithi143 podman[72350]: 2024-05-08 05:42:06.522442804 +0000 UTC m=+0.642492739 container remove 8054591672b3e6078a00c0503c8fa7ffc47d20c62913514a9bd68824b0281b70 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2) 2024-05-08T05:42:06.874 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:06 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:06.702+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:06.874 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:06 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:06.702+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:07.209 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:06 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:06.947+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:07.209 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:06 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:06.947+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:07.209 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:06 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:06.947+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:07.209 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:06 smithi143 podman[72442]: 2024-05-08 05:42:06.776877957 +0000 UTC m=+0.017815626 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:42:07.209 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:06 smithi143 podman[72442]: 2024-05-08 05:42:06.898395107 +0000 UTC m=+0.139332805 container create 7ce017f9e8702a24ac3dfd0c6f7bb1a7f95e861adf39073731645ee4d928123f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-05-08T05:42:07.209 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:07 smithi143 podman[72442]: 2024-05-08 05:42:07.109032519 +0000 UTC m=+0.349970176 container init 7ce017f9e8702a24ac3dfd0c6f7bb1a7f95e861adf39073731645ee4d928123f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5-deactivate, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-08T05:42:07.209 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:07 smithi143 podman[72442]: 2024-05-08 05:42:07.112112003 +0000 UTC m=+0.353049665 container start 7ce017f9e8702a24ac3dfd0c6f7bb1a7f95e861adf39073731645ee4d928123f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5-deactivate, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240507, ceph=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:42:07.209 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:07 smithi143 podman[72442]: 2024-05-08 05:42:07.209312085 +0000 UTC m=+0.450249757 container attach 7ce017f9e8702a24ac3dfd0c6f7bb1a7f95e861adf39073731645ee4d928123f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-05-08T05:42:07.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:06 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:06.877+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:07.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:06 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:06.877+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:07.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:06 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:06.877+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:07.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:07.264+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:07.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:07.264+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:07.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:07.264+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:07.337 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:07.018+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:07.337 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:07.018+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:07.337 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:07.018+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:07.493 INFO:journalctl@ceph.osd.5.smithi143.stdout:May 08 05:42:07 smithi143 podman[72442]: 2024-05-08 05:42:07.404096405 +0000 UTC m=+0.645034064 container died 7ce017f9e8702a24ac3dfd0c6f7bb1a7f95e861adf39073731645ee4d928123f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-5-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:42:07.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:07 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:07.303+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:07.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:07 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:07.303+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:07.922 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:07 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:07.662+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:07.922 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:07 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:07.662+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:08.047 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.5.service' 2024-05-08T05:42:08.085 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:42:08.085 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-05-08T05:42:08.085 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-05-08T05:42:08.085 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.6 2024-05-08T05:42:08.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:07 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:07.921+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:08.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:07 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:07.921+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:08.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:07 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:07.921+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:08.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:07 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:07.862+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:08.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:07 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:07.862+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:08.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:07 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:07.862+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:08.297 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:07.972+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:08.297 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:07.972+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:08.297 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:07 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:07.972+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:08.297 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:08 smithi059 systemd[1]: Stopping Ceph osd.6 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:42:08.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:08.423+0000 7f418b933700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:42:08.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:08.423+0000 7f418b933700 -1 osd.6 106 *** Got signal Terminated *** 2024-05-08T05:42:08.586 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:08.423+0000 7f418b933700 -1 osd.6 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:42:08.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:08.296+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:08.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:08.296+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:08.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:08.296+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:08.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:08 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:08.293+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:08.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:08 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:08.293+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:08.900 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:08 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:08.647+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:08.900 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:08 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:08.647+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:09.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:08 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:08.900+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:09.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:08 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:08.900+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:09.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:08 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:08.900+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:09.259 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:08.948+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:09.259 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:08.948+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:09.259 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:08 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:08.948+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:09.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:08 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:08.843+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:09.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:08 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:08.843+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:09.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:08 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:08.843+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:09.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:09.259+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:09.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:09.259+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:09.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:09.259+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:09.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:09 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:09.322+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:09.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:09 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:09.322+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:09.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:09 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:09.863+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:09.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:09 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:09.863+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:09.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:09 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:09.863+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:09.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:09 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:09.625+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:09.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:09 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:09.625+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:10.230 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:09.979+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:10.230 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:09.979+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:10.231 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:09.979+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:10.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:09 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:09.828+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:10.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:09 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:09.828+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:10.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:09 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:09.828+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:10.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:10.230+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:10.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:10.230+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:10.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:10.230+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:10.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:10 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:10.318+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:10.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:10 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:10.318+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:10.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:10 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:10.847+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:10.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:10 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:10.847+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:10.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:10 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:10.847+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:10.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:10 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:10.612+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:10.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:10 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:10.612+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:10.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:10 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:10.612+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:11.274 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:10 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:10.848+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:11.274 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:10 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:10.848+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:11.275 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:10 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:10.848+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:11.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:11 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:11.232+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:11.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:11 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:11.232+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:11.336 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:11 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:11.232+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:11.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:10.984+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:11.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:10.984+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:11.337 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:10.984+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:11.337 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:10 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:10.984+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.441208+0000 front 2024-05-08T05:41:47.441234+0000 (oldest deadline 2024-05-08T05:42:10.341404+0000) 2024-05-08T05:42:11.529 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:11 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:11.273+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:11.529 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:11 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:11.273+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:11.529 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:11 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:11.273+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:11.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:11 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:11.879+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:11.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:11 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:11.879+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:11.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:11 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:11.879+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:11.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:11 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:11.661+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:11.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:11 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:11.661+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:11.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:11 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:11.661+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:12.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:11 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:11.802+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:12.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:11 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:11.802+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:12.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:11 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:11.802+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:12.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:12.033+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:12.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:12.033+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:12.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:12.033+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:12.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:12.033+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.441208+0000 front 2024-05-08T05:41:47.441234+0000 (oldest deadline 2024-05-08T05:42:10.341404+0000) 2024-05-08T05:42:12.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:12.185+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:12.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:12.185+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:12.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:12.185+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:12.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:12 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:12.185+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:12.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:12.319+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:12.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:12.319+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:12.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:12.319+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:12.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:12 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:12.841+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:12.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:12 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:12.841+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:12.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:12 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:12.841+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:12.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:12 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:12.670+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:12.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:12 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:12.670+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:12.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:12 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:12.670+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:13.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:12.849+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:13.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:12.849+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:13.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:12.849+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:13.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:13.055+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.837382+0000 front 2024-05-08T05:41:22.837415+0000 (oldest deadline 2024-05-08T05:41:48.137612+0000) 2024-05-08T05:42:13.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:13.055+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:32.238943+0000 front 2024-05-08T05:41:32.238779+0000 (oldest deadline 2024-05-08T05:41:56.338855+0000) 2024-05-08T05:42:13.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:13.055+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.040119+0000 front 2024-05-08T05:41:38.040138+0000 (oldest deadline 2024-05-08T05:42:03.940187+0000) 2024-05-08T05:42:13.336 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6[58776]: 2024-05-08T05:42:13.055+0000 7f4187f27700 -1 osd.6 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.441208+0000 front 2024-05-08T05:41:47.441234+0000 (oldest deadline 2024-05-08T05:42:10.341404+0000) 2024-05-08T05:42:13.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:13.207+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:13.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:13.207+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:13.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:13.207+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:13.337 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:13 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:13.207+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:13.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:13 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:13.337+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:13.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:13 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:13.337+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:13.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:13 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:13.337+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:13.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:13 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:13.864+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:13.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:13 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:13.864+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:13.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:13 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:13.864+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:13.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:13 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:13.670+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:13.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:13 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:13.670+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:13.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:13 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:13.670+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:14.188 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:13 smithi059 podman[126840]: 2024-05-08 05:42:13.888537983 +0000 UTC m=+5.594479623 container died df4dc58411b57112f5bc854f5a71d554026d223badc820ecb988bba27e80a74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:42:14.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:13 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:13.858+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:14.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:13 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:13.858+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:14.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:13 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:13.858+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:14.558 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:14 smithi059 podman[126840]: 2024-05-08 05:42:14.298728036 +0000 UTC m=+6.004669674 container cleanup df4dc58411b57112f5bc854f5a71d554026d223badc820ecb988bba27e80a74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-05-08T05:42:14.558 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:14 smithi059 bash[126840]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6 2024-05-08T05:42:14.558 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:14 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:14.188+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:14.558 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:14 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:14.188+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:14.558 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:14 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:14.188+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:14.559 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:14 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:14.188+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:14.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:14 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:14.368+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:14.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:14 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:14.368+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:14.780 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:14 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:14.368+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:14.836 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:14 smithi059 podman[126858]: 2024-05-08 05:42:14.558266966 +0000 UTC m=+0.663401981 container remove df4dc58411b57112f5bc854f5a71d554026d223badc820ecb988bba27e80a74c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True) 2024-05-08T05:42:14.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:14 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:14.897+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:14.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:14 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:14.897+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:14.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:14 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:14.897+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:14.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:14 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:14.715+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:14.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:14 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:14.715+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:14.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:14 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:14.715+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:15.191 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:14 smithi059 podman[126952]: 2024-05-08 05:42:14.834351893 +0000 UTC m=+0.018741686 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:42:15.191 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:14 smithi059 podman[126952]: 2024-05-08 05:42:14.959228669 +0000 UTC m=+0.143618485 container create 35a68921f9aa5c25e9cd8459b4a3a57b282dab5000578ea5c93244db5eab19f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-08T05:42:15.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:14 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:14.830+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:15.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:14 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:14.831+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:15.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:14 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:14.831+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:15.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:14 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:14.831+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:15.513 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:15 smithi059 podman[126952]: 2024-05-08 05:42:15.194859542 +0000 UTC m=+0.379249315 container init 35a68921f9aa5c25e9cd8459b4a3a57b282dab5000578ea5c93244db5eab19f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6-deactivate, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS) 2024-05-08T05:42:15.513 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:15 smithi059 podman[126952]: 2024-05-08 05:42:15.198228962 +0000 UTC m=+0.382618742 container start 35a68921f9aa5c25e9cd8459b4a3a57b282dab5000578ea5c93244db5eab19f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2) 2024-05-08T05:42:15.513 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:15 smithi059 podman[126952]: 2024-05-08 05:42:15.311727145 +0000 UTC m=+0.496116940 container attach 35a68921f9aa5c25e9cd8459b4a3a57b282dab5000578ea5c93244db5eab19f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD) 2024-05-08T05:42:15.513 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:15 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:15.191+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:15.513 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:15 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:15.191+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:15.514 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:15 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:15.191+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:15.514 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:15 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:15.191+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:15.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:15 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:15.411+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:15.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:15 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:15.411+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:15.780 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:15 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:15.411+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:15.836 INFO:journalctl@ceph.osd.6.smithi059.stdout:May 08 05:42:15 smithi059 podman[126952]: 2024-05-08 05:42:15.513668939 +0000 UTC m=+0.698058718 container died 35a68921f9aa5c25e9cd8459b4a3a57b282dab5000578ea5c93244db5eab19f3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-05-08T05:42:15.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:15.908+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:15.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:15.908+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:15.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:15.908+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:15.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:15.700+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:15.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:15.700+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:15.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:15 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:15.700+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:16.144 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.6.service' 2024-05-08T05:42:16.185 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:42:16.185 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-05-08T05:42:16.186 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-05-08T05:42:16.186 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.7 2024-05-08T05:42:16.212 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:15 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:15.855+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:16.212 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:15 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:15.855+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:16.212 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:15 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:15.855+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:16.212 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:15 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:15.855+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:16.529 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:16 smithi096 systemd[1]: Stopping Ceph osd.7 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:42:16.529 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:16.375+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:16.529 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:16.375+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:16.529 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:16.375+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:16.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:16 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:16.197+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:16.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:16 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:16.198+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:16.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:16 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:16.198+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:16.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:16 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:16.198+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:16.899 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:16.549+0000 7fc6ab4a7700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:42:16.899 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:16.549+0000 7fc6ab4a7700 -1 osd.7 106 *** Got signal Terminated *** 2024-05-08T05:42:16.899 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:16.549+0000 7fc6ab4a7700 -1 osd.7 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:42:16.932 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:16 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:16.671+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:16.933 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:16 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:16.671+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:16.933 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:16 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:16.671+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:17.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:16 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:16.932+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:17.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:16 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:16.932+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:17.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:16 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:16.932+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:17.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:16.898+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:17.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:16.898+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:17.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:16.898+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:17.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:16 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:16.898+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:17.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:17 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:17.157+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:17.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:17 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:17.157+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:17.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:17 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:17.157+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:17.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:17 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:17.157+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:17.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:17.404+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:17.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:17.404+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:17.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:17.404+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:17.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:17.711+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:17.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:17.711+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:17.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:17.711+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:17.993 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:17.887+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:17.994 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:17.887+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:17.994 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:17 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:17.887+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:18.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:17.917+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:18.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:17.917+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:18.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:17.917+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:18.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:17.917+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:18.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:17.917+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:18.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:18 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:18.193+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:18.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:18 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:18.193+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:18.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:18 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:18.193+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:18.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:18 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:18.193+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:18.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:18 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:18.418+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:18.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:18 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:18.418+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:18.780 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:18 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:18.418+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:18.909 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:18 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:18.719+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:18.909 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:18 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:18.719+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:18.909 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:18 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:18.719+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:19.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:18 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:18.908+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:19.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:18 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:18.908+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:19.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:18 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:18.908+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:19.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:18 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:18.943+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:19.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:18 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:18.943+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:19.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:18 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:18.943+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:19.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:18 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:18.943+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:19.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:18 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:18.943+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:19.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:19 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:19.238+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:19.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:19 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:19.238+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:19.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:19 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:19.238+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:19.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:19 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:19.238+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:19.743 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:19.680+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:19.743 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:19.680+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:19.743 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:19.680+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:19.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:19.382+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:19.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:19.382+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:19.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:19.382+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:20.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:19.953+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:20.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:19.953+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:20.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:19.953+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:20.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:19 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:19.953+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:20.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:19.958+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:20.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:19.958+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:20.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:19.958+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:20.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:19.958+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:20.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:19 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:19.958+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:20.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:20 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:20.247+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:20.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:20 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:20.247+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:20.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:20 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:20.247+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:20.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:20 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:20.247+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:20.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:20.361+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:20.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:20.361+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:20.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:20.361+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:20.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:20.361+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:53.037787+0000 front 2024-05-08T05:41:53.037831+0000 (oldest deadline 2024-05-08T05:42:20.038475+0000) 2024-05-08T05:42:20.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:20.727+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:20.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:20.727+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:20.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:20.727+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:21.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:20.980+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:21.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:20.980+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:21.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:20.980+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:21.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:20.980+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:21.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:20 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:20.980+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:21.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:20.992+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:21.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:20.992+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:21.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:20.992+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:21.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:20 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:20.993+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:21.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:21 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:21.208+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:21.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:21 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:21.208+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:21.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:21 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:21.208+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:21.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:21 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:21.208+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:21.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:21 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:21.406+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.934071+0000 front 2024-05-08T05:41:30.934099+0000 (oldest deadline 2024-05-08T05:41:54.434237+0000) 2024-05-08T05:42:21.779 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:21 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:21.406+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:39.035867+0000 front 2024-05-08T05:41:39.035803+0000 (oldest deadline 2024-05-08T05:42:04.935914+0000) 2024-05-08T05:42:21.780 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:21 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:21.406+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.236961+0000 front 2024-05-08T05:41:47.237006+0000 (oldest deadline 2024-05-08T05:42:10.736904+0000) 2024-05-08T05:42:21.780 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:21 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7[52190]: 2024-05-08T05:42:21.406+0000 7fc6a7a9b700 -1 osd.7 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:53.037787+0000 front 2024-05-08T05:41:53.037831+0000 (oldest deadline 2024-05-08T05:42:20.038475+0000) 2024-05-08T05:42:21.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:21.691+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:21.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:21.691+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:21.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:21.691+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:22.279 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:22 smithi096 podman[71312]: 2024-05-08 05:42:22.061675511 +0000 UTC m=+5.641428849 container died c7fc6de95e035441f216e9980d65c119fb416d6643f3d77bc44a3d6c5d4751e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-05-08T05:42:22.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:22.001+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:22.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:22.001+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:22.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:22.001+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:22.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:22.001+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:22.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:22.002+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:22.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:21.998+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:22.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:21.998+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:22.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:21.998+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:22.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:21 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:21.998+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:22.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:22 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:22.226+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:22.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:22 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:22.226+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:22.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:22 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:22.226+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:22.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:22 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:22.226+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:22.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:22 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:22.226+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:22.672 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:22 smithi096 podman[71312]: 2024-05-08 05:42:22.462953368 +0000 UTC m=+6.042706705 container cleanup c7fc6de95e035441f216e9980d65c119fb416d6643f3d77bc44a3d6c5d4751e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:42:22.673 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:22 smithi096 bash[71312]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7 2024-05-08T05:42:22.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:22 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:22.741+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:22.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:22 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:22.741+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:22.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:22 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:22.741+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:22.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:22 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:22.741+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.178460+0000 front 2024-05-08T05:41:56.178476+0000 (oldest deadline 2024-05-08T05:42:22.078510+0000) 2024-05-08T05:42:23.001 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:22 smithi096 podman[71340]: 2024-05-08 05:42:22.672983986 +0000 UTC m=+0.605347051 container remove c7fc6de95e035441f216e9980d65c119fb416d6643f3d77bc44a3d6c5d4751e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:42:23.279 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:23 smithi096 podman[71431]: 2024-05-08 05:42:22.986370328 +0000 UTC m=+0.017659249 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:42:23.279 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:23 smithi096 podman[71431]: 2024-05-08 05:42:23.090568741 +0000 UTC m=+0.121857717 container create 8cb56bf00de4b8f628efdfd2bc5b185e01446eeb51a49ee71a2c95c3033e9a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507) 2024-05-08T05:42:23.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:23 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:23.001+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:23.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:23 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:23.001+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:23.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:23 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:23.001+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:23.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:23 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:23.001+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:23.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:23 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:23.001+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:23.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:23 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:23.010+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:23.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:23 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:23.010+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:23.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:23 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:23.010+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:23.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:23 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:23.010+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:23.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:23 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:23.253+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:23.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:23 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:23.253+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:23.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:23 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:23.253+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:23.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:23 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:23.253+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:23.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:23 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:23.253+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:23.633 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:23 smithi096 podman[71431]: 2024-05-08 05:42:23.328822973 +0000 UTC m=+0.360111884 container init 8cb56bf00de4b8f628efdfd2bc5b185e01446eeb51a49ee71a2c95c3033e9a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-08T05:42:23.633 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:23 smithi096 podman[71431]: 2024-05-08 05:42:23.331841857 +0000 UTC m=+0.363130771 container start 8cb56bf00de4b8f628efdfd2bc5b185e01446eeb51a49ee71a2c95c3033e9a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7-deactivate, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-05-08T05:42:23.633 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:23 smithi096 podman[71431]: 2024-05-08 05:42:23.40958456 +0000 UTC m=+0.440873490 container attach 8cb56bf00de4b8f628efdfd2bc5b185e01446eeb51a49ee71a2c95c3033e9a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507) 2024-05-08T05:42:23.920 INFO:journalctl@ceph.osd.7.smithi096.stdout:May 08 05:42:23 smithi096 podman[71431]: 2024-05-08 05:42:23.63277954 +0000 UTC m=+0.664068453 container died 8cb56bf00de4b8f628efdfd2bc5b185e01446eeb51a49ee71a2c95c3033e9a9a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:42:24.024 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:23 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:23.760+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:24.024 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:23 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:23.760+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:24.024 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:23 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:23.760+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:24.024 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:23 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:23.760+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.178460+0000 front 2024-05-08T05:41:56.178476+0000 (oldest deadline 2024-05-08T05:42:22.078510+0000) 2024-05-08T05:42:24.229 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:24 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:24.017+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:24.230 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:24 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:24.017+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:24.230 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:24 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:24.017+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:24.230 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:24 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:24.017+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:24.230 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:24 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:24.017+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:24.253 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.7.service' 2024-05-08T05:42:24.331 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:42:24.332 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-05-08T05:42:24.332 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-05-08T05:42:24.332 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.8 2024-05-08T05:42:24.358 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:24.023+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:24.358 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:24.023+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:24.358 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:24.023+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:24.358 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:24.023+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:24.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:24 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:24.251+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:24.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:24 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:24.251+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:24.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:24 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:24.251+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:24.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:24 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:24.251+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:24.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:24 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:24.251+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:24.649 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:24 smithi143 systemd[1]: Stopping Ceph osd.8 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:42:24.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:24.648+0000 7f7a4629d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:42:24.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:24.648+0000 7f7a4629d700 -1 osd.8 106 *** Got signal Terminated *** 2024-05-08T05:42:24.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:24.648+0000 7f7a4629d700 -1 osd.8 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:42:24.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:24.751+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:24.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:24.751+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:24.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:24.751+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:24.994 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:24 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:24.751+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.178460+0000 front 2024-05-08T05:41:56.178476+0000 (oldest deadline 2024-05-08T05:42:22.078510+0000) 2024-05-08T05:42:25.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:25 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:25.013+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:25.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:25 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:25.013+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:25.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:25 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:25.013+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:25.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:25 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:25.013+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:25.280 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:25 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:25.013+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:25.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:25.042+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:25.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:25.042+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:25.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:25.042+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:25.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:25.042+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:25.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:25 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:25.278+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:25.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:25 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:25.278+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:25.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:25 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:25.278+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:25.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:25 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:25.278+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:25.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:25 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:25.278+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:26.243 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:25.752+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:26.243 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:25.752+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:26.243 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:25.752+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:26.243 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:25.752+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.178460+0000 front 2024-05-08T05:41:56.178476+0000 (oldest deadline 2024-05-08T05:42:22.078510+0000) 2024-05-08T05:42:26.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:25.997+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:26.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:25.997+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:26.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:25.997+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:26.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:25 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:25.997+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:26.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:26.046+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:26.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:26.046+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:26.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:26.046+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:26.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:26.046+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:26.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:26.046+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:26.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:26.284+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:26.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:26.284+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:26.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:26.284+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:26.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:26.284+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:26.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:26 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:26.284+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:27.012 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:26.751+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:27.012 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:26.751+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:27.012 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:26.751+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:27.012 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:26 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:26.751+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.178460+0000 front 2024-05-08T05:41:56.178476+0000 (oldest deadline 2024-05-08T05:42:22.078510+0000) 2024-05-08T05:42:27.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.011+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:27.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.011+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:27.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.011+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:27.494 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.011+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:27.494 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.011+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:27.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:27.030+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:27.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:27.030+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:27.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:27.030+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:27.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:27.030+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:27.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:27 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:27.030+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:27.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:27 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:27.252+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:27.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:27 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:27.252+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:27.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:27 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:27.252+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:27.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:27 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:27.252+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:27.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:27 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:27.252+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:27.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:27 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:27.252+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:28.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.962+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:28.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.962+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:28.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.962+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:28.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.962+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:28.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:27.962+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:28.244 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:27.762+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:28.244 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:27.762+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:28.244 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:27.762+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:28.244 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:27 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:27.762+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.178460+0000 front 2024-05-08T05:41:56.178476+0000 (oldest deadline 2024-05-08T05:42:22.078510+0000) 2024-05-08T05:42:28.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:28.043+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:28.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:28.043+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:28.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:28.043+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:28.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:28.043+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:28.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:28.043+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:28.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:28 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:28.043+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:28.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:28.230+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:28.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:28.230+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:28.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:28.230+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:28.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:28.230+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:28.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:28.230+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:28.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:28 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:28.230+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:28.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:28.717+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:31.773002+0000 front 2024-05-08T05:41:31.773130+0000 (oldest deadline 2024-05-08T05:41:56.473119+0000) 2024-05-08T05:42:28.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:28.717+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.473733+0000 front 2024-05-08T05:41:36.473823+0000 (oldest deadline 2024-05-08T05:42:01.773896+0000) 2024-05-08T05:42:28.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:28.717+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.075259+0000 front 2024-05-08T05:41:47.075165+0000 (oldest deadline 2024-05-08T05:42:10.575270+0000) 2024-05-08T05:42:28.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:28.717+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.178460+0000 front 2024-05-08T05:41:56.178476+0000 (oldest deadline 2024-05-08T05:42:22.078510+0000) 2024-05-08T05:42:28.993 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8[53111]: 2024-05-08T05:42:28.717+0000 7f7a3e889700 -1 osd.8 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.078902+0000 front 2024-05-08T05:42:02.078882+0000 (oldest deadline 2024-05-08T05:42:27.978873+0000) 2024-05-08T05:42:29.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:28.993+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:29.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:28.993+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:29.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:28.993+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:29.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:28.993+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:29.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:28 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:28.993+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:29.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:29 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:29.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:29.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:29 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:29.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:29.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:29 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:29.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:29.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:29 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:29.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:29.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:29 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:29.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:29.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:29 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:29.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:29.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:29.231+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:29.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:29.231+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:29.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:29.231+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:29.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:29.231+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:29.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:29.231+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:29.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:29 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:29.231+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:30.243 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:30 smithi143 podman[72559]: 2024-05-08 05:42:30.189757276 +0000 UTC m=+5.658311792 container died a15af0860e33d3dd8c23b9174ee65a4bfb45153a02552611e3f0e6b35c0c8d05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-05-08T05:42:30.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:29.983+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:30.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:29.983+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:30.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:29.983+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:30.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:29.983+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:30.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:29 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:29.983+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:30.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:30.081+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:30.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:30.081+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:30.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:30.081+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:30.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:30.081+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:30.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:30.081+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:30.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:30 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:30.081+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:30.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:30.218+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:30.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:30.218+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:30.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:30.218+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:30.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:30.218+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:30.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:30.218+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:30.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:30 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:30.218+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:30.743 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:30 smithi143 podman[72559]: 2024-05-08 05:42:30.599335776 +0000 UTC m=+6.067890284 container cleanup a15af0860e33d3dd8c23b9174ee65a4bfb45153a02552611e3f0e6b35c0c8d05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:42:30.743 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:30 smithi143 bash[72559]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8 2024-05-08T05:42:31.151 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:30 smithi143 podman[72577]: 2024-05-08 05:42:30.842322338 +0000 UTC m=+0.649187165 container remove a15af0860e33d3dd8c23b9174ee65a4bfb45153a02552611e3f0e6b35c0c8d05 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8, RELEASE=wip-57676-3-bee299f, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.build-date=20240507) 2024-05-08T05:42:31.151 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:31.010+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:31.151 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:31.010+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:31.152 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:31.010+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:31.152 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:31.010+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:31.152 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:31 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:31.010+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:31.404 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:31 smithi143 podman[72666]: 2024-05-08 05:42:31.151638312 +0000 UTC m=+0.112299574 container create 6bbd9f3b13f56a54296e9bf44f2ad31f9654e6cfdf682707917cea616d29c123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-05-08T05:42:31.404 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:31 smithi143 podman[72666]: 2024-05-08 05:42:31.056919822 +0000 UTC m=+0.017581040 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:42:31.404 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:31 smithi143 podman[72666]: 2024-05-08 05:42:31.404119153 +0000 UTC m=+0.364780373 container init 6bbd9f3b13f56a54296e9bf44f2ad31f9654e6cfdf682707917cea616d29c123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-05-08T05:42:31.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:31.056+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:31.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:31.056+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:31.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:31.056+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:31.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:31.056+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:31.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:31.056+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:31.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:31 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:31.056+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:31.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:31.242+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:31.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:31.242+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:31.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:31.242+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:31.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:31.242+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:31.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:31.242+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:31.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:31 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:31.242+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:31.697 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:31 smithi143 podman[72666]: 2024-05-08 05:42:31.407227695 +0000 UTC m=+0.367888918 container start 6bbd9f3b13f56a54296e9bf44f2ad31f9654e6cfdf682707917cea616d29c123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-08T05:42:31.698 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:31 smithi143 podman[72666]: 2024-05-08 05:42:31.479123588 +0000 UTC m=+0.439784823 container attach 6bbd9f3b13f56a54296e9bf44f2ad31f9654e6cfdf682707917cea616d29c123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-05-08T05:42:31.990 INFO:journalctl@ceph.osd.8.smithi143.stdout:May 08 05:42:31 smithi143 podman[72666]: 2024-05-08 05:42:31.698163406 +0000 UTC m=+0.658824631 container died 6bbd9f3b13f56a54296e9bf44f2ad31f9654e6cfdf682707917cea616d29c123 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-8-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:42:32.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:32 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:32.028+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:32.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:32 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:32.028+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:32.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:32 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:32.028+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:32.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:32 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:32.028+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:32.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:32 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:32.028+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:32.323 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.8.service' 2024-05-08T05:42:32.404 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:42:32.405 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-05-08T05:42:32.405 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-05-08T05:42:32.405 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.9 2024-05-08T05:42:32.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:32.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:32.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:32.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:32.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:32.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:32.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:32.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:32.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:32.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:32.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:32 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:32.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:32.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:32.255+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:32.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:32.255+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:32.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:32.255+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:32.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:32.255+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:32.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:32.255+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:32.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:32.255+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:32.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 systemd[1]: Stopping Ceph osd.9 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:42:33.086 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:32.708+0000 7fae335f9700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:42:33.086 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:32.708+0000 7fae335f9700 -1 osd.9 106 *** Got signal Terminated *** 2024-05-08T05:42:33.086 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:32 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:32.708+0000 7fae335f9700 -1 osd.9 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:42:33.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.008+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:33.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.008+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:33.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.008+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:33.493 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.008+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:33.494 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.008+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:33.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:33 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:33.092+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:33.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:33 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:33.092+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:33.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:33 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:33.092+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:33.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:33 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:33.092+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:33.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:33 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:33.092+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:33.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:33 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:33.092+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:33.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:33.249+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:33.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:33.249+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:33.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:33.249+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:33.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:33.249+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:33.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:33.249+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:33.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:33 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:33.249+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:34.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.959+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:34.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.959+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:34.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.959+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:34.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.959+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:34.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:33 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:33.959+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:34.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:34 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:34.113+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:34.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:34 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:34.113+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:34.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:34 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:34.113+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:34.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:34 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:34.113+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:34.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:34 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:34.113+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:34.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:34 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:34.113+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:34.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:34 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:34.113+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:34.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:34.243+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:34.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:34.243+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:34.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:34.243+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:34.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:34.243+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:34.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:34.243+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:34.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:34 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:34.243+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:35.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:34 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:34.939+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:35.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:34 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:34.939+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:35.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:34 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:34.939+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:35.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:34 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:34.939+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:35.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:34 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:34.939+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:35.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:35 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:35.115+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:35.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:35 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:35.115+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:35.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:35 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:35.115+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:35.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:35 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:35.115+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:35.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:35 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:35.115+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:35.531 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:35 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:35.115+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:35.531 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:35 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:35.115+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:35.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:35 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:35.263+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:35.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:35 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:35.263+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:35.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:35 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:35.263+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:35.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:35 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:35.263+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:35.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:35 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:35.263+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:35.587 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:35 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:35.263+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:36.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:35.923+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:36.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:35.923+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:36.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:35.923+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:36.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:35.923+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:36.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:35.923+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:36.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:35 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:35.923+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:36.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:36 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:36.090+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:36.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:36 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:36.090+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:36.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:36 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:36.090+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:36.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:36 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:36.090+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:36.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:36 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:36.090+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:36.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:36 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:36.090+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:36.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:36 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:36.090+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:36.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:36 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:36.304+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:36.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:36 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:36.304+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:36.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:36 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:36.304+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:36.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:36 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:36.304+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:36.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:36 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:36.304+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:36.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:36 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:36.304+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:37.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:36 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:36.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:37.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:36 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:36.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:37.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:36 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:36.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:37.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:36 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:36.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:37.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:36 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:36.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:37.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:36 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:36.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:37.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:37 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:37.042+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:37.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:37 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:37.042+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:37.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:37 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:37.042+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:37.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:37 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:37.042+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:37.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:37 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:37.042+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:37.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:37 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:37.042+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:37.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:37 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:37.042+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:37.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:37 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:37.308+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:22.126026+0000 front 2024-05-08T05:41:22.126101+0000 (oldest deadline 2024-05-08T05:41:46.226140+0000) 2024-05-08T05:42:37.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:37 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:37.308+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.227498+0000 front 2024-05-08T05:41:30.227521+0000 (oldest deadline 2024-05-08T05:41:56.127694+0000) 2024-05-08T05:42:37.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:37 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:37.308+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:36.128387+0000 front 2024-05-08T05:41:36.128334+0000 (oldest deadline 2024-05-08T05:42:03.129152+0000) 2024-05-08T05:42:37.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:37 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:37.308+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:45.430129+0000 front 2024-05-08T05:41:45.430247+0000 (oldest deadline 2024-05-08T05:42:11.330165+0000) 2024-05-08T05:42:37.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:37 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:37.308+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:56.531674+0000 front 2024-05-08T05:41:56.531609+0000 (oldest deadline 2024-05-08T05:42:21.231710+0000) 2024-05-08T05:42:37.586 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:37 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9[65633]: 2024-05-08T05:42:37.308+0000 7fae2fbed700 -1 osd.9 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:03.533030+0000 front 2024-05-08T05:42:03.532908+0000 (oldest deadline 2024-05-08T05:42:26.432899+0000) 2024-05-08T05:42:38.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:37 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:37.948+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:38.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:37 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:37.948+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:38.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:37 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:37.948+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:38.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:37 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:37.948+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:38.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:37 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:37.948+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:38.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:37 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:37.948+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:38.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:38 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:38.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:38.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:38 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:38.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:38.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:38 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:38.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:38.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:38 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:38.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:38.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:38 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:38.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:38.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:38 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:38.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:38.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:38 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:38.048+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:38.535 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:38 smithi059 podman[127070]: 2024-05-08 05:42:38.259187558 +0000 UTC m=+5.650556836 container died bfc811b77a45644c2b809e66d71421306c9d475aacc616a1b304ff4bfcfdceb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507) 2024-05-08T05:42:38.836 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:38 smithi059 podman[127070]: 2024-05-08 05:42:38.643280425 +0000 UTC m=+6.034649748 container cleanup bfc811b77a45644c2b809e66d71421306c9d475aacc616a1b304ff4bfcfdceb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0) 2024-05-08T05:42:38.836 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:38 smithi059 bash[127070]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9 2024-05-08T05:42:39.227 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:38 smithi059 podman[127089]: 2024-05-08 05:42:38.88622638 +0000 UTC m=+0.619701150 container remove bfc811b77a45644c2b809e66d71421306c9d475aacc616a1b304ff4bfcfdceb1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-08T05:42:39.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:38 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:38.919+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:39.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:38 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:38.919+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:39.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:38 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:38.919+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:39.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:38 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:38.919+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:39.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:38 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:38.919+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:39.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:38 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:38.919+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:39.488 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:39 smithi059 podman[127180]: 2024-05-08 05:42:39.131214454 +0000 UTC m=+0.018513971 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:42:39.489 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:39 smithi059 podman[127180]: 2024-05-08 05:42:39.236672133 +0000 UTC m=+0.123971689 container create 6248c0b20a6cbc565a4a1446612e39e562c2a5595d88ae7a16ca46f05ef5d8ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-05-08T05:42:39.489 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:39 smithi059 podman[127180]: 2024-05-08 05:42:39.489094795 +0000 UTC m=+0.376394310 container init 6248c0b20a6cbc565a4a1446612e39e562c2a5595d88ae7a16ca46f05ef5d8ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:42:39.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:39 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:39.069+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:39.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:39 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:39.069+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:39.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:39 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:39.069+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:39.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:39 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:39.069+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:39.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:39 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:39.069+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:39.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:39 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:39.069+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:39.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:39 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:39.069+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:39.803 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:39 smithi059 podman[127180]: 2024-05-08 05:42:39.49261765 +0000 UTC m=+0.379917165 container start 6248c0b20a6cbc565a4a1446612e39e562c2a5595d88ae7a16ca46f05ef5d8ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-05-08T05:42:39.803 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:39 smithi059 podman[127180]: 2024-05-08 05:42:39.555629242 +0000 UTC m=+0.442928776 container attach 6248c0b20a6cbc565a4a1446612e39e562c2a5595d88ae7a16ca46f05ef5d8ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-05-08T05:42:40.074 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:39 smithi059 podman[127180]: 2024-05-08 05:42:39.804209267 +0000 UTC m=+0.691508780 container died 6248c0b20a6cbc565a4a1446612e39e562c2a5595d88ae7a16ca46f05ef5d8ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9-deactivate, org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:42:40.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:39 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:39.926+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:40.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:39 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:39.926+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:40.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:39 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:39.926+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:40.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:39 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:39.926+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:40.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:39 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:39.926+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:40.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:39 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:39.926+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:40.325 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:40 smithi059 podman[127206]: 2024-05-08 05:42:40.308709934 +0000 UTC m=+0.498338654 container remove 6248c0b20a6cbc565a4a1446612e39e562c2a5595d88ae7a16ca46f05ef5d8ea (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-9-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux ) 2024-05-08T05:42:40.325 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:40 smithi059 systemd[1]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.9.service: Deactivated successfully. 2024-05-08T05:42:40.325 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:40 smithi059 systemd[1]: Stopped Ceph osd.9 for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:42:40.325 INFO:journalctl@ceph.osd.9.smithi059.stdout:May 08 05:42:40 smithi059 systemd[1]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.9.service: Consumed 2min 5.920s CPU time. 2024-05-08T05:42:40.326 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.9.service' 2024-05-08T05:42:40.367 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:42:40.367 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-05-08T05:42:40.368 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-05-08T05:42:40.368 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.10 2024-05-08T05:42:40.393 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.085+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:40.393 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.085+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:40.393 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.085+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:40.393 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.085+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:40.393 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.085+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:40.393 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.085+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:40.393 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.085+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:40.681 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 systemd[1]: Stopping Ceph osd.10 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:42:40.681 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.680+0000 7f296632c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:42:40.681 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.680+0000 7f296632c700 -1 osd.10 106 *** Got signal Terminated *** 2024-05-08T05:42:40.681 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:40 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:40.680+0000 7f296632c700 -1 osd.10 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:42:41.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:40 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:40.927+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:41.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:40 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:40.927+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:41.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:40 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:40.927+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:41.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:40 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:40.927+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:41.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:40 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:40.927+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:41.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:40 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:40.927+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:41.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:41 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:41.075+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:41.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:41 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:41.075+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:41.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:41 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:41.075+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:41.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:41 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:41.075+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:41.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:41 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:41.075+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:41.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:41 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:41.075+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:41.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:41 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:41.075+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:42.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:41 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:41.950+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:42.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:41 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:41.950+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:42.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:41 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:41.950+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:42.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:41 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:41.950+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:42.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:41 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:41.950+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:42.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:41 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:41.950+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:42.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:42 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:42.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:42.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:42 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:42.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:42.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:42 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:42.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:42.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:42 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:42.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:42.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:42 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:42.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:42.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:42 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:42.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:42.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:42 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:42.063+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:43.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:42.914+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:43.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:42.914+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:43.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:42.914+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:43.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:42.914+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:43.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:42.914+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:43.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:42.914+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:43.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:43 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:43.096+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:43.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:43 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:43.096+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:43.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:43 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:43.096+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:43.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:43 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:43.096+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:43.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:43 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:43.096+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:43.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:43 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:43.096+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:43.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:43 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:43.096+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:44.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:43 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:43.937+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:44.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:43 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:43.937+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:44.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:43 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:43.937+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:44.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:43 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:43.937+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:44.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:43 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:43.937+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:44.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:43 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:43.937+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:44.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:44 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:44.135+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:44.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:44 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:44.135+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:44.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:44 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:44.135+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:44.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:44 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:44.135+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:44.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:44 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:44.135+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:44.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:44 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:44.135+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:44.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:44 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:44.135+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:45.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:44 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:44.907+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:45.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:44 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:44.907+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:45.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:44 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:44.907+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:45.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:44 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:44.907+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:45.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:44 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:44.907+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:45.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:44 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:44.907+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:45.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:45 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:45.137+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:24.079071+0000 front 2024-05-08T05:41:24.079018+0000 (oldest deadline 2024-05-08T05:41:46.279635+0000) 2024-05-08T05:42:45.529 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:45 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:45.137+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:28.580658+0000 front 2024-05-08T05:41:28.580833+0000 (oldest deadline 2024-05-08T05:41:54.480826+0000) 2024-05-08T05:42:45.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:45 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:45.137+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.482480+0000 front 2024-05-08T05:41:38.482464+0000 (oldest deadline 2024-05-08T05:42:04.382460+0000) 2024-05-08T05:42:45.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:45 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:45.137+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6814 osd.3 since back 2024-05-08T05:41:47.883737+0000 front 2024-05-08T05:41:47.883659+0000 (oldest deadline 2024-05-08T05:42:14.284250+0000) 2024-05-08T05:42:45.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:45 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:45.137+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:54.284608+0000 front 2024-05-08T05:41:54.284628+0000 (oldest deadline 2024-05-08T05:42:17.784777+0000) 2024-05-08T05:42:45.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:45 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:45.137+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:02.887153+0000 front 2024-05-08T05:42:02.886987+0000 (oldest deadline 2024-05-08T05:42:27.586845+0000) 2024-05-08T05:42:45.530 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:45 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10[59096]: 2024-05-08T05:42:45.137+0000 7f2962920700 -1 osd.10 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:09.288108+0000 front 2024-05-08T05:42:09.288066+0000 (oldest deadline 2024-05-08T05:42:33.987816+0000) 2024-05-08T05:42:46.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:45 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:45.872+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:46.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:45 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:45.872+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:46.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:45 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:45.872+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:46.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:45 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:45.872+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:46.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:45 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:45.872+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:46.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:45 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:45.872+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:46.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:45 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:45.872+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-08T05:42:19.720577+0000 front 2024-05-08T05:42:19.720472+0000 (oldest deadline 2024-05-08T05:42:45.020372+0000) 2024-05-08T05:42:46.523 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:46 smithi096 podman[71536]: 2024-05-08 05:42:46.257574308 +0000 UTC m=+5.694036666 container stop c11cd9e27599ba1464ddc165b72d6fe4a31561e67141bd52dacf3cc95dbd84ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:42:46.523 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:46 smithi096 podman[71536]: 2024-05-08 05:42:46.283232926 +0000 UTC m=+5.719695281 container died c11cd9e27599ba1464ddc165b72d6fe4a31561e67141bd52dacf3cc95dbd84ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_CLEAN=True) 2024-05-08T05:42:46.779 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:46 smithi096 podman[71536]: 2024-05-08 05:42:46.613875783 +0000 UTC m=+6.050338136 container cleanup c11cd9e27599ba1464ddc165b72d6fe4a31561e67141bd52dacf3cc95dbd84ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-05-08T05:42:46.779 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:46 smithi096 bash[71536]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10 2024-05-08T05:42:47.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:46 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:46.916+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:47.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:46 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:46.916+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:47.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:46 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:46.916+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:47.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:46 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:46.916+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:47.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:46 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:46.916+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:47.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:46 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:46.916+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:47.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:46 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:46.916+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-08T05:42:19.720577+0000 front 2024-05-08T05:42:19.720472+0000 (oldest deadline 2024-05-08T05:42:45.020372+0000) 2024-05-08T05:42:47.279 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:46 smithi096 podman[71554]: 2024-05-08 05:42:46.840337667 +0000 UTC m=+0.576175161 container remove c11cd9e27599ba1464ddc165b72d6fe4a31561e67141bd52dacf3cc95dbd84ee (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5) 2024-05-08T05:42:47.568 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:47 smithi096 podman[71655]: 2024-05-08 05:42:47.215312857 +0000 UTC m=+0.017855000 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:42:47.568 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:47 smithi096 podman[71655]: 2024-05-08 05:42:47.316053863 +0000 UTC m=+0.118596000 container create c9d26b05bebdb585fa3463f1390010f1b8a32db623211d0c22780d1c07c2de10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD) 2024-05-08T05:42:47.568 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:47 smithi096 podman[71655]: 2024-05-08 05:42:47.568840652 +0000 UTC m=+0.371382781 container init c9d26b05bebdb585fa3463f1390010f1b8a32db623211d0c22780d1c07c2de10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD) 2024-05-08T05:42:47.869 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:47 smithi096 podman[71655]: 2024-05-08 05:42:47.572213538 +0000 UTC m=+0.374755669 container start c9d26b05bebdb585fa3463f1390010f1b8a32db623211d0c22780d1c07c2de10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:42:47.869 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:47 smithi096 podman[71655]: 2024-05-08 05:42:47.643892238 +0000 UTC m=+0.446434368 container attach c9d26b05bebdb585fa3463f1390010f1b8a32db623211d0c22780d1c07c2de10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-05-08T05:42:48.154 INFO:journalctl@ceph.osd.10.smithi096.stdout:May 08 05:42:47 smithi096 podman[71655]: 2024-05-08 05:42:47.869534978 +0000 UTC m=+0.672077109 container died c9d26b05bebdb585fa3463f1390010f1b8a32db623211d0c22780d1c07c2de10 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240507) 2024-05-08T05:42:48.158 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:47 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:47.874+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:48.158 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:47 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:47.874+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:48.158 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:47 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:47.874+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:48.158 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:47 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:47.874+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:48.158 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:47 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:47.874+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:48.158 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:47 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:47.874+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:48.158 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:47 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:47.874+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-08T05:42:19.720577+0000 front 2024-05-08T05:42:19.720472+0000 (oldest deadline 2024-05-08T05:42:45.020372+0000) 2024-05-08T05:42:48.425 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.10.service' 2024-05-08T05:42:48.503 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:42:48.504 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-05-08T05:42:48.504 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-05-08T05:42:48.504 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.11 2024-05-08T05:42:48.743 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 systemd[1]: Stopping Ceph osd.11 for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:42:49.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.785+0000 7fc6a37a8700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:42:49.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.785+0000 7fc6a37a8700 -1 osd.11 106 *** Got signal Terminated *** 2024-05-08T05:42:49.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.785+0000 7fc6a37a8700 -1 osd.11 106 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-05-08T05:42:49.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.885+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:49.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.885+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:49.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.885+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:49.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.885+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:49.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.886+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:49.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.886+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:49.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:48 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:48.886+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-08T05:42:19.720577+0000 front 2024-05-08T05:42:19.720472+0000 (oldest deadline 2024-05-08T05:42:45.020372+0000) 2024-05-08T05:42:50.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:49 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:49.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:50.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:49 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:49.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:50.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:49 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:49.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:50.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:49 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:49.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:50.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:49 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:49.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:50.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:49 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:49.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:50.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:49 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:49.913+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-08T05:42:19.720577+0000 front 2024-05-08T05:42:19.720472+0000 (oldest deadline 2024-05-08T05:42:45.020372+0000) 2024-05-08T05:42:51.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:50.876+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:51.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:50.876+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:51.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:50.876+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:51.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:50.876+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:51.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:50.876+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:51.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:50.876+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:51.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:50.876+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-08T05:42:19.720577+0000 front 2024-05-08T05:42:19.720472+0000 (oldest deadline 2024-05-08T05:42:45.020372+0000) 2024-05-08T05:42:51.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:50 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:50.876+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-05-08T05:42:27.122203+0000 front 2024-05-08T05:42:27.122222+0000 (oldest deadline 2024-05-08T05:42:50.622272+0000) 2024-05-08T05:42:52.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:51.848+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:52.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:51.848+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:52.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:51.848+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:52.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:51.848+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:52.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:51.848+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:52.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:51.848+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:52.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:51.848+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-08T05:42:19.720577+0000 front 2024-05-08T05:42:19.720472+0000 (oldest deadline 2024-05-08T05:42:45.020372+0000) 2024-05-08T05:42:52.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:51 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:51.848+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-05-08T05:42:27.122203+0000 front 2024-05-08T05:42:27.122222+0000 (oldest deadline 2024-05-08T05:42:50.622272+0000) 2024-05-08T05:42:53.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:52.830+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6806 osd.0 since back 2024-05-08T05:41:21.813067+0000 front 2024-05-08T05:41:21.813242+0000 (oldest deadline 2024-05-08T05:41:46.513239+0000) 2024-05-08T05:42:53.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:52.830+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6804 osd.1 since back 2024-05-08T05:41:30.014535+0000 front 2024-05-08T05:41:30.014436+0000 (oldest deadline 2024-05-08T05:41:55.914357+0000) 2024-05-08T05:42:53.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:52.830+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6804 osd.2 since back 2024-05-08T05:41:38.815364+0000 front 2024-05-08T05:41:38.815237+0000 (oldest deadline 2024-05-08T05:42:04.715366+0000) 2024-05-08T05:42:53.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:52.830+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6812 osd.4 since back 2024-05-08T05:41:55.217247+0000 front 2024-05-08T05:41:55.217409+0000 (oldest deadline 2024-05-08T05:42:19.317114+0000) 2024-05-08T05:42:53.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:52.830+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6812 osd.5 since back 2024-05-08T05:42:01.618044+0000 front 2024-05-08T05:42:01.618084+0000 (oldest deadline 2024-05-08T05:42:26.318094+0000) 2024-05-08T05:42:53.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:52.830+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.59:6822 osd.6 since back 2024-05-08T05:42:12.119505+0000 front 2024-05-08T05:42:12.119467+0000 (oldest deadline 2024-05-08T05:42:35.019421+0000) 2024-05-08T05:42:53.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:52.830+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.96:6820 osd.7 since back 2024-05-08T05:42:19.720577+0000 front 2024-05-08T05:42:19.720472+0000 (oldest deadline 2024-05-08T05:42:45.020372+0000) 2024-05-08T05:42:53.244 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:52 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11[60001]: 2024-05-08T05:42:52.830+0000 7fc69fd9c700 -1 osd.11 106 heartbeat_check: no reply from 172.21.15.143:6820 osd.8 since back 2024-05-08T05:42:27.122203+0000 front 2024-05-08T05:42:27.122222+0000 (oldest deadline 2024-05-08T05:42:50.622272+0000) 2024-05-08T05:42:54.610 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:54 smithi143 podman[72790]: 2024-05-08 05:42:54.301800982 +0000 UTC m=+5.614166227 container died b383deb01f3d036169e58caca85aa6e6983e0dd29a069e538f9e2098c8c0a734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True) 2024-05-08T05:42:54.919 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:54 smithi143 podman[72790]: 2024-05-08 05:42:54.710462347 +0000 UTC m=+6.022827589 container cleanup b383deb01f3d036169e58caca85aa6e6983e0dd29a069e538f9e2098c8c0a734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=) 2024-05-08T05:42:54.920 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:54 smithi143 bash[72790]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11 2024-05-08T05:42:55.243 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:54 smithi143 podman[72810]: 2024-05-08 05:42:54.91999372 +0000 UTC m=+0.608596108 container remove b383deb01f3d036169e58caca85aa6e6983e0dd29a069e538f9e2098c8c0a734 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux ) 2024-05-08T05:42:55.598 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:55 smithi143 podman[72899]: 2024-05-08 05:42:55.225468596 +0000 UTC m=+0.018313018 image pull 352576a2df49fbab3aeeba1effbcb7a94815e8fc1f120a55ddc340b4ee63b8ce quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c 2024-05-08T05:42:55.598 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:55 smithi143 podman[72899]: 2024-05-08 05:42:55.329245849 +0000 UTC m=+0.122090269 container create 4d7cfe7d7ae6a6c1dcbef295e76458ab5b2bc27a7b2c8e630e70c861c4e064b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-05-08T05:42:55.896 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:55 smithi143 podman[72899]: 2024-05-08 05:42:55.598654389 +0000 UTC m=+0.391498817 container init 4d7cfe7d7ae6a6c1dcbef295e76458ab5b2bc27a7b2c8e630e70c861c4e064b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True) 2024-05-08T05:42:55.896 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:55 smithi143 podman[72899]: 2024-05-08 05:42:55.602312539 +0000 UTC m=+0.395156959 container start 4d7cfe7d7ae6a6c1dcbef295e76458ab5b2bc27a7b2c8e630e70c861c4e064b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-05-08T05:42:55.896 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:55 smithi143 podman[72899]: 2024-05-08 05:42:55.681791162 +0000 UTC m=+0.474635582 container attach 4d7cfe7d7ae6a6c1dcbef295e76458ab5b2bc27a7b2c8e630e70c861c4e064b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-05-08T05:42:56.167 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:55 smithi143 podman[72899]: 2024-05-08 05:42:55.896598584 +0000 UTC m=+0.689443002 container died 4d7cfe7d7ae6a6c1dcbef295e76458ab5b2bc27a7b2c8e630e70c861c4e064b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11-deactivate, RELEASE=wip-57676-3-bee299f, ceph=True, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-05-08T05:42:56.399 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.11.service' 2024-05-08T05:42:56.420 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:56 smithi143 podman[72925]: 2024-05-08 05:42:56.376690897 +0000 UTC m=+0.470268194 container remove 4d7cfe7d7ae6a6c1dcbef295e76458ab5b2bc27a7b2c8e630e70c861c4e064b1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-osd-11-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240507, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-05-08T05:42:56.420 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:56 smithi143 systemd[1]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.11.service: Deactivated successfully. 2024-05-08T05:42:56.420 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:56 smithi143 systemd[1]: Stopped Ceph osd.11 for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:42:56.420 INFO:journalctl@ceph.osd.11.smithi143.stdout:May 08 05:42:56 smithi143 systemd[1]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@osd.11.service: Consumed 1min 28.133s CPU time. 2024-05-08T05:42:56.434 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:42:56.435 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-05-08T05:42:56.435 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-05-08T05:42:56.435 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.a 2024-05-08T05:42:56.737 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:42:56 smithi059 systemd[1]: Stopping Ceph mds.a for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:42:57.086 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:42:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-a[72589]: 2024-05-08T05:42:56.737+0000 7f94a3da6700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:42:57.086 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:42:56 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-a[72589]: 2024-05-08T05:42:56.737+0000 7f94a3da6700 -1 mds.a *** got signal Terminated *** 2024-05-08T05:43:01.029 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:43:00 smithi059 podman[127284]: 2024-05-08 05:43:00.743130478 +0000 UTC m=+4.103636767 container died d02388b60e7bad8c46a4d9fed1aa4a0709696c382a9c9931ed1ab12fd30a5992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-a, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=) 2024-05-08T05:43:01.336 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:43:01 smithi059 podman[127284]: 2024-05-08 05:43:01.111953048 +0000 UTC m=+4.472459339 container cleanup d02388b60e7bad8c46a4d9fed1aa4a0709696c382a9c9931ed1ab12fd30a5992 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-a, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, ceph=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-05-08T05:43:01.336 INFO:journalctl@ceph.mds.a.smithi059.stdout:May 08 05:43:01 smithi059 bash[127284]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-a 2024-05-08T05:43:01.561 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.a.service' 2024-05-08T05:43:01.596 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:01.596 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-05-08T05:43:01.596 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-05-08T05:43:01.597 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.d 2024-05-08T05:43:02.008 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:43:01 smithi059 systemd[1]: Stopping Ceph mds.d for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:02.336 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:43:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-d[71358]: 2024-05-08T05:43:02.007+0000 7f98ebcc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:02.336 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:43:02 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-d[71358]: 2024-05-08T05:43:02.007+0000 7f98ebcc0700 -1 mds.d *** got signal Terminated *** 2024-05-08T05:43:03.683 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:43:03 smithi059 podman[127406]: 2024-05-08 05:43:03.411026426 +0000 UTC m=+1.591881078 container died 16357abb3bc047c131158640104819c73c814dd167a95a5ab4ef29f63700dbf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-d, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-05-08T05:43:04.086 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:43:03 smithi059 podman[127406]: 2024-05-08 05:43:03.782663169 +0000 UTC m=+1.963517775 container cleanup 16357abb3bc047c131158640104819c73c814dd167a95a5ab4ef29f63700dbf8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-d, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-05-08T05:43:04.086 INFO:journalctl@ceph.mds.d.smithi059.stdout:May 08 05:43:03 smithi059 bash[127406]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-d 2024-05-08T05:43:04.232 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.d.service' 2024-05-08T05:43:04.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:04.266 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-05-08T05:43:04.266 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-05-08T05:43:04.266 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.g 2024-05-08T05:43:04.836 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:43:04 smithi059 systemd[1]: Stopping Ceph mds.g for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:04.836 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:43:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g[72165]: 2024-05-08T05:43:04.595+0000 7fb863ed2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:04.836 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:43:04 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g[72165]: 2024-05-08T05:43:04.595+0000 7fb863ed2700 -1 mds.g *** got signal Terminated *** 2024-05-08T05:43:08.902 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:43:08 smithi059 podman[127526]: 2024-05-08 05:43:08.601120311 +0000 UTC m=+4.105842142 container died bbacb2744df2a0696c00a24e3b2bdc97046d60b8083d02b43b8afbb1194c9022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:43:09.222 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:43:08 smithi059 podman[127526]: 2024-05-08 05:43:08.902818389 +0000 UTC m=+4.407540220 container cleanup bbacb2744df2a0696c00a24e3b2bdc97046d60b8083d02b43b8afbb1194c9022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD) 2024-05-08T05:43:09.222 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:43:08 smithi059 bash[127526]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g 2024-05-08T05:43:09.222 INFO:journalctl@ceph.mds.g.smithi059.stdout:May 08 05:43:09 smithi059 podman[127541]: 2024-05-08 05:43:09.104049016 +0000 UTC m=+0.496158274 container remove bbacb2744df2a0696c00a24e3b2bdc97046d60b8083d02b43b8afbb1194c9022 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-g, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-08T05:43:09.240 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.g.service' 2024-05-08T05:43:09.274 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:09.274 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-05-08T05:43:09.274 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-05-08T05:43:09.274 DEBUG:teuthology.orchestra.run.smithi059:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.j 2024-05-08T05:43:09.586 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:09 smithi059 systemd[1]: Stopping Ceph mds.j for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:10.086 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j[70916]: 2024-05-08T05:43:09.608+0000 7ff721cea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:10.086 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:09 smithi059 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j[70916]: 2024-05-08T05:43:09.608+0000 7ff721cea700 -1 mds.j *** got signal Terminated *** 2024-05-08T05:43:11.648 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:11 smithi059 podman[127647]: 2024-05-08 05:43:11.385007035 +0000 UTC m=+1.889320466 container died 283f0a6ceacb0d265b1e61f1777c54e8aad1ad203e11725ae3ee249f5d267d69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:43:11.923 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:11 smithi059 podman[127647]: 2024-05-08 05:43:11.723031464 +0000 UTC m=+2.227344898 container cleanup 283f0a6ceacb0d265b1e61f1777c54e8aad1ad203e11725ae3ee249f5d267d69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, ceph=True, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.license=GPLv2) 2024-05-08T05:43:11.923 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:11 smithi059 bash[127647]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j 2024-05-08T05:43:12.161 DEBUG:teuthology.orchestra.run.smithi059:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.j.service' 2024-05-08T05:43:12.183 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:11 smithi059 podman[127661]: 2024-05-08 05:43:11.923895214 +0000 UTC m=+0.538471598 container remove 283f0a6ceacb0d265b1e61f1777c54e8aad1ad203e11725ae3ee249f5d267d69 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-j, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f) 2024-05-08T05:43:12.183 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:12 smithi059 systemd[1]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.j.service: Deactivated successfully. 2024-05-08T05:43:12.183 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:12 smithi059 systemd[1]: Stopped Ceph mds.j for 0bf01a10-0cfa-11ef-bc97-c7b262605968. 2024-05-08T05:43:12.183 INFO:journalctl@ceph.mds.j.smithi059.stdout:May 08 05:43:12 smithi059 systemd[1]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.j.service: Consumed 1.124s CPU time. 2024-05-08T05:43:12.194 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:12.194 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-05-08T05:43:12.194 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-05-08T05:43:12.194 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.b 2024-05-08T05:43:12.487 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:43:12 smithi096 systemd[1]: Stopping Ceph mds.b for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:12.779 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:43:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b[64457]: 2024-05-08T05:43:12.486+0000 7f36e468f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:12.779 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:43:12 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b[64457]: 2024-05-08T05:43:12.486+0000 7f36e468f700 -1 mds.b *** got signal Terminated *** 2024-05-08T05:43:16.755 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:43:16 smithi096 podman[71761]: 2024-05-08 05:43:16.49248309 +0000 UTC m=+4.101531812 container died 19a46f73e739048daccad6d4777d8b4e3db376e75e0a643bf99f9ec4ecff0704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5) 2024-05-08T05:43:17.029 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:43:16 smithi096 podman[71761]: 2024-05-08 05:43:16.845542772 +0000 UTC m=+4.454591491 container cleanup 19a46f73e739048daccad6d4777d8b4e3db376e75e0a643bf99f9ec4ecff0704 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-05-08T05:43:17.029 INFO:journalctl@ceph.mds.b.smithi096.stdout:May 08 05:43:16 smithi096 bash[71761]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-b 2024-05-08T05:43:17.322 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.b.service' 2024-05-08T05:43:17.354 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:17.355 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-05-08T05:43:17.355 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-05-08T05:43:17.355 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.e 2024-05-08T05:43:17.674 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:43:17 smithi096 systemd[1]: Stopping Ceph mds.e for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:18.029 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:43:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e[63836]: 2024-05-08T05:43:17.673+0000 7f293075d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:18.029 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:43:17 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e[63836]: 2024-05-08T05:43:17.673+0000 7f293075d700 -1 mds.e *** got signal Terminated *** 2024-05-08T05:43:21.279 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:43:20 smithi096 podman[71881]: 2024-05-08 05:43:20.986174556 +0000 UTC m=+3.420548385 container died e2474414a20601912da2a7e4ca99646eb189a3961a78574dc2baa6c462b7faa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, RELEASE=wip-57676-3-bee299f, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True) 2024-05-08T05:43:21.625 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:43:21 smithi096 podman[71881]: 2024-05-08 05:43:21.365801369 +0000 UTC m=+3.800175198 container cleanup e2474414a20601912da2a7e4ca99646eb189a3961a78574dc2baa6c462b7faa9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504) 2024-05-08T05:43:21.625 INFO:journalctl@ceph.mds.e.smithi096.stdout:May 08 05:43:21 smithi096 bash[71881]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-e 2024-05-08T05:43:21.737 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.e.service' 2024-05-08T05:43:21.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:21.771 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-05-08T05:43:21.771 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-05-08T05:43:21.771 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.h 2024-05-08T05:43:22.029 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:43:21 smithi096 systemd[1]: Stopping Ceph mds.h for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:22.529 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:43:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h[64147]: 2024-05-08T05:43:22.119+0000 7f2ebb7ea700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:22.529 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:43:22 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h[64147]: 2024-05-08T05:43:22.119+0000 7f2ebb7ea700 -1 mds.h *** got signal Terminated *** 2024-05-08T05:43:25.253 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:43:24 smithi096 podman[72013]: 2024-05-08 05:43:24.977947344 +0000 UTC m=+2.994547761 container died e5e7468a1174ef77d39f609e00b4427d6cb0bbe205924a7d8d5295096844232c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-05-08T05:43:25.512 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:43:25 smithi096 podman[72013]: 2024-05-08 05:43:25.319747705 +0000 UTC m=+3.336348153 container cleanup e5e7468a1174ef77d39f609e00b4427d6cb0bbe205924a7d8d5295096844232c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-57676-3-bee299f, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240507, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS) 2024-05-08T05:43:25.512 INFO:journalctl@ceph.mds.h.smithi096.stdout:May 08 05:43:25 smithi096 bash[72013]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-h 2024-05-08T05:43:25.738 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.h.service' 2024-05-08T05:43:25.770 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:25.770 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-05-08T05:43:25.770 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-05-08T05:43:25.770 DEBUG:teuthology.orchestra.run.smithi096:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.k 2024-05-08T05:43:26.029 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:43:25 smithi096 systemd[1]: Stopping Ceph mds.k for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:26.529 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:43:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k[63518]: 2024-05-08T05:43:26.114+0000 7f8175b69700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:26.529 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:43:26 smithi096 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k[63518]: 2024-05-08T05:43:26.114+0000 7f8175b69700 -1 mds.k *** got signal Terminated *** 2024-05-08T05:43:29.279 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:43:28 smithi096 podman[72133]: 2024-05-08 05:43:28.995420136 +0000 UTC m=+3.007349109 container died e8fde80f3d5533d5cdd65658667f4a585540e4636585314bb1bb54677ff9f745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, RELEASE=wip-57676-3-bee299f) 2024-05-08T05:43:29.688 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:43:29 smithi096 podman[72133]: 2024-05-08 05:43:29.314844634 +0000 UTC m=+3.326773597 container cleanup e8fde80f3d5533d5cdd65658667f4a585540e4636585314bb1bb54677ff9f745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.build-date=20240507, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:43:29.688 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:43:29 smithi096 bash[72133]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k 2024-05-08T05:43:29.688 INFO:journalctl@ceph.mds.k.smithi096.stdout:May 08 05:43:29 smithi096 podman[72145]: 2024-05-08 05:43:29.532379879 +0000 UTC m=+0.533692250 container remove e8fde80f3d5533d5cdd65658667f4a585540e4636585314bb1bb54677ff9f745 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True) 2024-05-08T05:43:29.757 DEBUG:teuthology.orchestra.run.smithi096:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.k.service' 2024-05-08T05:43:29.787 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:29.787 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-05-08T05:43:29.788 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-05-08T05:43:29.788 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.c 2024-05-08T05:43:30.168 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:43:29 smithi143 systemd[1]: Stopping Ceph mds.c for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:30.429 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:43:30 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c[64997]: 2024-05-08T05:43:30.167+0000 7ff72573f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:30.430 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:43:30 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c[64997]: 2024-05-08T05:43:30.167+0000 7ff72573f700 -1 mds.c *** got signal Terminated *** 2024-05-08T05:43:33.243 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:43:32 smithi143 podman[73003]: 2024-05-08 05:43:32.985203166 +0000 UTC m=+3.006663245 container died 8388243eb8233eaae9dfb70a81d3ee08734c23305c61b7138d86972ab51fd573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c, org.label-schema.build-date=20240507, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-57676-3-bee299f, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-05-08T05:43:33.726 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:43:33 smithi143 podman[73003]: 2024-05-08 05:43:33.450588535 +0000 UTC m=+3.472048624 container cleanup 8388243eb8233eaae9dfb70a81d3ee08734c23305c61b7138d86972ab51fd573 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c, ceph=True, org.label-schema.build-date=20240507, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux ) 2024-05-08T05:43:33.726 INFO:journalctl@ceph.mds.c.smithi143.stdout:May 08 05:43:33 smithi143 bash[73003]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-c 2024-05-08T05:43:33.875 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.c.service' 2024-05-08T05:43:33.910 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:33.910 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-05-08T05:43:33.910 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-05-08T05:43:33.910 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.f 2024-05-08T05:43:34.243 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:43:33 smithi143 systemd[1]: Stopping Ceph mds.f for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:34.743 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:43:34 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f[64685]: 2024-05-08T05:43:34.245+0000 7f728e870700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:34.743 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:43:34 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f[64685]: 2024-05-08T05:43:34.245+0000 7f728e870700 -1 mds.f *** got signal Terminated *** 2024-05-08T05:43:37.243 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:43:36 smithi143 podman[73135]: 2024-05-08 05:43:36.981041701 +0000 UTC m=+2.853160513 container died 33cb737861dada2f2d0525358a3b0dadcb89f4f8efda1a3e1451c2bafcaa36d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240507, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5) 2024-05-08T05:43:37.622 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:43:37 smithi143 podman[73135]: 2024-05-08 05:43:37.362793905 +0000 UTC m=+3.234912714 container cleanup 33cb737861dada2f2d0525358a3b0dadcb89f4f8efda1a3e1451c2bafcaa36d2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f, org.label-schema.vendor=CentOS, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.build-date=20240507, GIT_BRANCH=HEAD, RELEASE=wip-57676-3-bee299f, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:43:37.622 INFO:journalctl@ceph.mds.f.smithi143.stdout:May 08 05:43:37 smithi143 bash[73135]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-f 2024-05-08T05:43:37.723 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.f.service' 2024-05-08T05:43:37.755 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:37.755 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-05-08T05:43:37.755 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-05-08T05:43:37.756 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.i 2024-05-08T05:43:37.992 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:43:37 smithi143 systemd[1]: Stopping Ceph mds.i for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:38.493 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:43:38 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i[65304]: 2024-05-08T05:43:38.090+0000 7f6a3a264700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:38.493 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:43:38 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i[65304]: 2024-05-08T05:43:38.090+0000 7f6a3a264700 -1 mds.i *** got signal Terminated *** 2024-05-08T05:43:41.243 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:43:40 smithi143 podman[73255]: 2024-05-08 05:43:40.98900728 +0000 UTC m=+3.011603398 container died dcd9b13ab8a790385d7260297b33c521e854764ec425fe13df1518e04879b97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-57676-3-bee299f, org.label-schema.build-date=20240507) 2024-05-08T05:43:41.617 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:43:41 smithi143 podman[73255]: 2024-05-08 05:43:41.333169907 +0000 UTC m=+3.355766029 container cleanup dcd9b13ab8a790385d7260297b33c521e854764ec425fe13df1518e04879b97a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, maintainer=Guillaume Abrioux , ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240507, RELEASE=wip-57676-3-bee299f, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image) 2024-05-08T05:43:41.617 INFO:journalctl@ceph.mds.i.smithi143.stdout:May 08 05:43:41 smithi143 bash[73255]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-i 2024-05-08T05:43:41.778 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.i.service' 2024-05-08T05:43:41.809 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:41.810 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-05-08T05:43:41.810 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-05-08T05:43:41.810 DEBUG:teuthology.orchestra.run.smithi143:> sudo systemctl stop ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.l 2024-05-08T05:43:42.120 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:43:41 smithi143 systemd[1]: Stopping Ceph mds.l for 0bf01a10-0cfa-11ef-bc97-c7b262605968... 2024-05-08T05:43:42.493 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:43:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l[64369]: 2024-05-08T05:43:42.119+0000 7fc6268e3700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-05-08T05:43:42.493 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:43:42 smithi143 ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l[64369]: 2024-05-08T05:43:42.119+0000 7fc6268e3700 -1 mds.l *** got signal Terminated *** 2024-05-08T05:43:45.253 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:43:44 smithi143 podman[73376]: 2024-05-08 05:43:44.999654954 +0000 UTC m=+2.973618891 container died d840ca7eb67bfa48c07ed3c6f3291fdf89b41641ced2668be9b151b3750eb5bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=wip-57676-3-bee299f, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240507, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-05-08T05:43:45.545 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:43:45 smithi143 podman[73376]: 2024-05-08 05:43:45.328146806 +0000 UTC m=+3.302110752 container cleanup d840ca7eb67bfa48c07ed3c6f3291fdf89b41641ced2668be9b151b3750eb5bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:5a93a30dfa6b7dac54076fc754f0102ea322f9bbae94f68f29b70eaa3b17b91c, name=ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-57676-3-bee299f, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240507, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=627304247bdbcb1f80af6cad8aac9a1ee2979504, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-05-08T05:43:45.545 INFO:journalctl@ceph.mds.l.smithi143.stdout:May 08 05:43:45 smithi143 bash[73376]: ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968-mds-l 2024-05-08T05:43:45.774 DEBUG:teuthology.orchestra.run.smithi143:> sudo pkill -f 'journalctl -f -n 0 -u ceph-0bf01a10-0cfa-11ef-bc97-c7b262605968@mds.l.service' 2024-05-08T05:43:45.804 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-05-08T05:43:45.804 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-05-08T05:43:45.804 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T05:43:46.008 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:43:48.602 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T05:43:48.792 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:43:51.196 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 --force --keep-logs 2024-05-08T05:43:51.385 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:43:53.655 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:53.689 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:53.720 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-05-08T05:43:53.745 INFO:tasks.cephadm:Archiving crash dumps... 2024-05-08T05:43:53.749 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/remote/smithi059/crash 2024-05-08T05:43:53.750 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/crash -- . 2024-05-08T05:43:53.791 INFO:teuthology.orchestra.run.smithi059.stderr:tar: /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T05:43:53.791 INFO:teuthology.orchestra.run.smithi059.stderr:tar: Error is not recoverable: exiting now 2024-05-08T05:43:53.793 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/remote/smithi096/crash 2024-05-08T05:43:53.794 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/crash -- . 2024-05-08T05:43:53.836 INFO:teuthology.orchestra.run.smithi096.stderr:tar: /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T05:43:53.837 INFO:teuthology.orchestra.run.smithi096.stderr:tar: Error is not recoverable: exiting now 2024-05-08T05:43:53.839 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/crash to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/remote/smithi143/crash 2024-05-08T05:43:53.840 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/crash -- . 2024-05-08T05:43:53.884 INFO:teuthology.orchestra.run.smithi143.stderr:tar: /var/lib/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/crash: Cannot open: No such file or directory 2024-05-08T05:43:53.884 INFO:teuthology.orchestra.run.smithi143.stderr:tar: Error is not recoverable: exiting now 2024-05-08T05:43:53.886 INFO:tasks.cephadm:Checking cluster log for badness... 2024-05-08T05:43:53.886 DEBUG:teuthology.orchestra.run.smithi059:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | head -n 1 2024-05-08T05:43:53.922 INFO:tasks.cephadm:Compressing logs... 2024-05-08T05:43:53.922 DEBUG:teuthology.orchestra.run.smithi059:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T05:43:53.965 DEBUG:teuthology.orchestra.run.smithi096:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T05:43:53.968 DEBUG:teuthology.orchestra.run.smithi143:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-05-08T05:43:53.990 INFO:teuthology.orchestra.run.smithi143.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log 2024-05-08T05:43:53.991 INFO:teuthology.orchestra.run.smithi143.stderr:: No such file or directory 2024-05-08T05:43:53.991 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.2.log 2024-05-08T05:43:53.998 INFO:teuthology.orchestra.run.smithi096.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose --: No such file or directory /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log 2024-05-08T05:43:53.998 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-05-08T05:43:53.998 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T05:43:53.998 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.7.log 2024-05-08T05:43:53.998 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.b.log 2024-05-08T05:43:53.999 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.audit.log: /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.4.log 2024-05-08T05:43:53.999 INFO:teuthology.orchestra.run.smithi096.stderr:gzip/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.b.log: -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.1.log 2024-05-08T05:43:53.999 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mgr.y.log 2024-05-08T05:43:53.999 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.h.log 2024-05-08T05:43:53.999 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.10.log 2024-05-08T05:43:53.999 INFO:teuthology.orchestra.run.smithi096.stderr: 87.2% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T05:43:53.999 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.h.log: 90.3% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T05:43:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log 2024-05-08T05:43:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:: No such file or directory 2024-05-08T05:43:54.000 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.g.log 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log: /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.j.log 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.3.log 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.0.log 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.j.log: /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mgr.x.log 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mon.a.log 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.6.log 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr: 87.2% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mon.a.log: 90.2% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T05:43:54.001 INFO:teuthology.orchestra.run.smithi059.stderr: 94.6% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.j.log.gz 2024-05-08T05:43:54.003 INFO:teuthology.orchestra.run.smithi096.stderr: 92.0% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mgr.y.log.gz 2024-05-08T05:43:54.004 INFO:teuthology.orchestra.run.smithi143.stderr:gzip/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log: -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.f.log 2024-05-08T05:43:54.004 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.audit.log 2024-05-08T05:43:54.004 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.5.log 2024-05-08T05:43:54.004 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.c.log 2024-05-08T05:43:54.004 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mon.c.log 2024-05-08T05:43:54.004 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.c.log: /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mon.c.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mgr.z.log 2024-05-08T05:43:54.004 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.i.log 2024-05-08T05:43:54.004 INFO:teuthology.orchestra.run.smithi143.stderr: 87.2% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.log.gz 2024-05-08T05:43:54.005 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mgr.z.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T05:43:54.005 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.i.log: 90.3% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.audit.log.gz 2024-05-08T05:43:54.005 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.11.log 2024-05-08T05:43:54.005 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.cephadm.log: 83.7% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T05:43:54.005 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.8.log 2024-05-08T05:43:54.007 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.11.log: 92.1% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mgr.z.log.gz 2024-05-08T05:43:54.007 INFO:teuthology.orchestra.run.smithi143.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T05:43:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T05:43:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.a.log 2024-05-08T05:43:54.008 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T05:43:54.008 INFO:teuthology.orchestra.run.smithi059.stderr: 84.0% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T05:43:54.010 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.cephadm.log 2024-05-08T05:43:54.010 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.10.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.e.log 2024-05-08T05:43:54.010 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-volume.log 2024-05-08T05:43:54.010 INFO:teuthology.orchestra.run.smithi096.stderr: 83.8%/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.e.log: -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph.cephadm.log.gz 2024-05-08T05:43:54.021 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mon.b.log 2024-05-08T05:43:54.023 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.d.log 2024-05-08T05:43:54.024 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.l.log 2024-05-08T05:43:54.034 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.k.log 2024-05-08T05:43:54.035 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T05:43:54.037 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T05:43:54.038 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-volume.log: 91.1% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T05:43:54.040 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.9.log 2024-05-08T05:43:54.041 INFO:teuthology.orchestra.run.smithi096.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T05:43:54.046 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.d.log: 94.5% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.d.log.gz 2024-05-08T05:43:54.051 INFO:teuthology.orchestra.run.smithi143.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.l.log: /var/log/ceph/cephadm.log: 91.0% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-volume.log.gz 2024-05-08T05:43:54.053 INFO:teuthology.orchestra.run.smithi059.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-05-08T05:43:54.056 INFO:teuthology.orchestra.run.smithi096.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T05:43:54.060 INFO:teuthology.orchestra.run.smithi143.stderr: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T05:43:54.066 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-05-08T05:43:54.083 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/cephadm.log: 94.0% 94.4% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.a.log.gz 2024-05-08T05:43:54.086 INFO:teuthology.orchestra.run.smithi059.stderr:/var/log/ceph/ceph-client.admin.log: 92.8% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-05-08T05:43:54.086 INFO:teuthology.orchestra.run.smithi059.stderr: -- replaced with /var/log/ceph/cephadm.log.gz 2024-05-08T05:43:54.094 INFO:teuthology.orchestra.run.smithi143.stderr: 93.0% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.c.log.gz 2024-05-08T05:43:54.099 INFO:teuthology.orchestra.run.smithi059.stderr: 94.2% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.g.log.gz 2024-05-08T05:43:54.145 INFO:teuthology.orchestra.run.smithi096.stderr: 93.3% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.h.log.gz 2024-05-08T05:43:54.172 INFO:teuthology.orchestra.run.smithi096.stderr: 94.2% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.k.log.gz 2024-05-08T05:43:54.196 INFO:teuthology.orchestra.run.smithi143.stderr: 94.4% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.l.log.gz 2024-05-08T05:43:54.360 INFO:teuthology.orchestra.run.smithi096.stderr: 91.9% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mon.b.log.gz 2024-05-08T05:43:54.396 INFO:teuthology.orchestra.run.smithi059.stderr: 90.1% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mgr.x.log.gz 2024-05-08T05:43:54.435 INFO:teuthology.orchestra.run.smithi143.stderr: 91.8% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mon.c.log.gz 2024-05-08T05:43:54.512 INFO:teuthology.orchestra.run.smithi096.stderr: 94.5% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.e.log.gz 2024-05-08T05:43:54.935 INFO:teuthology.orchestra.run.smithi143.stderr: 95.1% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.i.log.gz 2024-05-08T05:43:54.951 INFO:teuthology.orchestra.run.smithi059.stderr: 90.3% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mon.a.log.gz 2024-05-08T05:43:56.262 INFO:teuthology.orchestra.run.smithi143.stderr: 95.8% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.f.log.gz 2024-05-08T05:44:06.409 INFO:teuthology.orchestra.run.smithi143.stderr: 91.7% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.11.log.gz 2024-05-08T05:44:07.486 INFO:teuthology.orchestra.run.smithi096.stderr: 91.5% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.7.log.gz 2024-05-08T05:44:08.821 INFO:teuthology.orchestra.run.smithi059.stderr: 91.8% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.9.log.gz 2024-05-08T05:44:09.110 INFO:teuthology.orchestra.run.smithi143.stderr: 91.6% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.2.log.gz 2024-05-08T05:44:12.050 INFO:teuthology.orchestra.run.smithi096.stderr: 91.6% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.10.log.gz 2024-05-08T05:44:13.771 INFO:teuthology.orchestra.run.smithi059.stderr: 91.7% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.0.log.gz 2024-05-08T05:44:14.734 INFO:teuthology.orchestra.run.smithi096.stderr: 91.7% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.1.log.gz 2024-05-08T05:44:15.594 INFO:teuthology.orchestra.run.smithi143.stderr: 91.6% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.5.log.gz 2024-05-08T05:44:16.555 INFO:teuthology.orchestra.run.smithi059.stderr: 91.5% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.6.log.gz 2024-05-08T05:44:20.119 INFO:teuthology.orchestra.run.smithi143.stderr: 91.5% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.8.log.gz 2024-05-08T05:44:20.120 INFO:teuthology.orchestra.run.smithi143.stderr: 2024-05-08T05:44:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:real 0m26.139s 2024-05-08T05:44:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:user 1m16.560s 2024-05-08T05:44:20.121 INFO:teuthology.orchestra.run.smithi143.stderr:sys 0m2.441s 2024-05-08T05:44:22.989 INFO:teuthology.orchestra.run.smithi096.stderr: 91.9% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.4.log.gz 2024-05-08T05:44:24.379 INFO:teuthology.orchestra.run.smithi059.stderr: 91.5% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-osd.3.log.gz 2024-05-08T05:44:24.381 INFO:teuthology.orchestra.run.smithi059.stderr: 2024-05-08T05:44:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:real 0m30.400s 2024-05-08T05:44:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:user 1m26.314s 2024-05-08T05:44:24.381 INFO:teuthology.orchestra.run.smithi059.stderr:sys 0m2.614s 2024-05-08T05:44:29.466 INFO:teuthology.orchestra.run.smithi096.stderr: 93.8% -- replaced with /var/log/ceph/0bf01a10-0cfa-11ef-bc97-c7b262605968/ceph-mds.b.log.gz 2024-05-08T05:44:29.468 INFO:teuthology.orchestra.run.smithi096.stderr: 2024-05-08T05:44:29.468 INFO:teuthology.orchestra.run.smithi096.stderr:real 0m35.487s 2024-05-08T05:44:29.468 INFO:teuthology.orchestra.run.smithi096.stderr:user 1m54.068s 2024-05-08T05:44:29.468 INFO:teuthology.orchestra.run.smithi096.stderr:sys 0m3.410s 2024-05-08T05:44:29.468 INFO:tasks.cephadm:Archiving logs... 2024-05-08T05:44:29.468 DEBUG:teuthology.misc:Transferring archived files from smithi059:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/remote/smithi059/log 2024-05-08T05:44:29.469 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T05:44:44.007 DEBUG:teuthology.misc:Transferring archived files from smithi096:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/remote/smithi096/log 2024-05-08T05:44:44.008 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T05:45:01.378 DEBUG:teuthology.misc:Transferring archived files from smithi143:/var/log/ceph to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/remote/smithi143/log 2024-05-08T05:45:01.379 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /var/log/ceph -- . 2024-05-08T05:45:14.334 INFO:tasks.cephadm:Removing cluster... 2024-05-08T05:45:14.334 DEBUG:teuthology.orchestra.run.smithi059:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 --force 2024-05-08T05:45:14.539 INFO:teuthology.orchestra.run.smithi059.stdout:Deleting cluster with fsid: 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:45:14.942 DEBUG:teuthology.orchestra.run.smithi096:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 --force 2024-05-08T05:45:15.133 INFO:teuthology.orchestra.run.smithi096.stdout:Deleting cluster with fsid: 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:45:15.671 DEBUG:teuthology.orchestra.run.smithi143:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid 0bf01a10-0cfa-11ef-bc97-c7b262605968 --force 2024-05-08T05:45:15.862 INFO:teuthology.orchestra.run.smithi143.stdout:Deleting cluster with fsid: 0bf01a10-0cfa-11ef-bc97-c7b262605968 2024-05-08T05:45:16.234 INFO:tasks.cephadm:Removing cephadm ... 2024-05-08T05:45:16.235 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T05:45:16.255 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T05:45:16.270 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf /home/ubuntu/cephtest/cephadm 2024-05-08T05:45:16.285 INFO:tasks.cephadm:Teardown complete 2024-05-08T05:45:16.286 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-05-08T05:45:16.297 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-05-08T05:45:16.297 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T05:45:16.300 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T05:45:16.312 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-05-08T05:45:16.417 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-05-08T05:45:16.429 INFO:teuthology.task.clock:Checking final clock skew... 2024-05-08T05:45:16.429 DEBUG:teuthology.orchestra.run.smithi059:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:45:16.432 DEBUG:teuthology.orchestra.run.smithi096:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:45:16.434 DEBUG:teuthology.orchestra.run.smithi143:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-05-08T05:45:16.446 INFO:teuthology.orchestra.run.smithi096.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:45:16.447 INFO:teuthology.orchestra.run.smithi059.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:45:16.447 INFO:teuthology.orchestra.run.smithi143.stderr:bash: line 1: ntpq: command not found 2024-05-08T05:45:16.450 INFO:teuthology.orchestra.run.smithi096.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:45:16.450 INFO:teuthology.orchestra.run.smithi096.stdout:=============================================================================== 2024-05-08T05:45:16.450 INFO:teuthology.orchestra.run.smithi096.stdout:^* hv01.front.sepia.ceph.com 3 6 377 53 +205us[ +204us] +/- 54ms 2024-05-08T05:45:16.451 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -139us[ -139us] +/- 61ms 2024-05-08T05:45:16.451 INFO:teuthology.orchestra.run.smithi096.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -98us[ -98us] +/- 75ms 2024-05-08T05:45:16.451 INFO:teuthology.orchestra.run.smithi096.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:45:16.453 INFO:teuthology.orchestra.run.smithi059.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:45:16.453 INFO:teuthology.orchestra.run.smithi059.stdout:=============================================================================== 2024-05-08T05:45:16.453 INFO:teuthology.orchestra.run.smithi059.stdout:^* hv01.front.sepia.ceph.com 3 6 377 54 +215us[ +274us] +/- 54ms 2024-05-08T05:45:16.453 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 51 -128us[ -128us] +/- 62ms 2024-05-08T05:45:16.453 INFO:teuthology.orchestra.run.smithi059.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 54 -81us[ -23us] +/- 75ms 2024-05-08T05:45:16.453 INFO:teuthology.orchestra.run.smithi059.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:45:16.453 INFO:teuthology.orchestra.run.smithi143.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-05-08T05:45:16.453 INFO:teuthology.orchestra.run.smithi143.stdout:=============================================================================== 2024-05-08T05:45:16.454 INFO:teuthology.orchestra.run.smithi143.stdout:^* hv01.front.sepia.ceph.com 3 6 377 55 +225us[ +231us] +/- 54ms 2024-05-08T05:45:16.454 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 53 -135us[ -135us] +/- 61ms 2024-05-08T05:45:16.454 INFO:teuthology.orchestra.run.smithi143.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 52 -88us[ -88us] +/- 75ms 2024-05-08T05:45:16.454 INFO:teuthology.orchestra.run.smithi143.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-05-08T05:45:16.454 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-05-08T05:45:16.466 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-05-08T05:45:16.466 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-05-08T05:45:16.486 DEBUG:teuthology.orchestra.run.smithi059:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T05:45:16.581 DEBUG:teuthology.orchestra.run.smithi096:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T05:45:16.646 DEBUG:teuthology.orchestra.run.smithi143:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-05-08T05:45:16.710 DEBUG:teuthology.orchestra.run.smithi059:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:45:16.743 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:45:16.744 DEBUG:teuthology.orchestra.run.smithi096:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:45:16.771 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:45:16.771 DEBUG:teuthology.orchestra.run.smithi143:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 -e scontext=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-05-08T05:45:16.797 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:45:16.797 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-05-08T05:45:16.808 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-05-08T05:45:16.844 INFO:teuthology.task.internal:Duration was 2186.555333 seconds 2024-05-08T05:45:16.845 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-05-08T05:45:16.861 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-05-08T05:45:16.861 DEBUG:teuthology.orchestra.run.smithi059:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T05:45:16.864 DEBUG:teuthology.orchestra.run.smithi096:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T05:45:16.866 DEBUG:teuthology.orchestra.run.smithi143:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-05-08T05:45:16.900 INFO:teuthology.orchestra.run.smithi096.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:45:16.902 INFO:teuthology.orchestra.run.smithi143.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:45:16.904 INFO:teuthology.orchestra.run.smithi059.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-05-08T05:45:17.317 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-05-08T05:45:17.317 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi059.front.sepia.ceph.com 2024-05-08T05:45:17.317 DEBUG:teuthology.orchestra.run.smithi059:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T05:45:17.339 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi096.front.sepia.ceph.com 2024-05-08T05:45:17.339 DEBUG:teuthology.orchestra.run.smithi096:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T05:45:17.385 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi143.front.sepia.ceph.com 2024-05-08T05:45:17.386 DEBUG:teuthology.orchestra.run.smithi143:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-05-08T05:45:17.407 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-05-08T05:45:17.407 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T05:45:17.410 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T05:45:17.427 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-05-08T05:45:17.472 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-05-08T05:45:17.473 DEBUG:teuthology.orchestra.run.smithi059:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T05:45:17.478 DEBUG:teuthology.orchestra.run.smithi096:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T05:45:17.493 DEBUG:teuthology.orchestra.run.smithi143:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-05-08T05:45:18.264 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-05-08T05:45:18.277 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-05-08T05:45:18.277 DEBUG:teuthology.orchestra.run.smithi059:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T05:45:18.303 DEBUG:teuthology.orchestra.run.smithi096:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T05:45:18.329 DEBUG:teuthology.orchestra.run.smithi143:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-05-08T05:45:18.354 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-05-08T05:45:18.365 DEBUG:teuthology.orchestra.run.smithi059:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:45:18.368 DEBUG:teuthology.orchestra.run.smithi096:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:45:18.372 DEBUG:teuthology.orchestra.run.smithi143:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:45:18.390 INFO:teuthology.orchestra.run.smithi059.stdout:kernel.core_pattern = core 2024-05-08T05:45:18.392 INFO:teuthology.orchestra.run.smithi096.stdout:kernel.core_pattern = core 2024-05-08T05:45:18.416 INFO:teuthology.orchestra.run.smithi143.stdout:kernel.core_pattern = core 2024-05-08T05:45:18.435 DEBUG:teuthology.orchestra.run.smithi059:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:45:18.463 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:45:18.463 DEBUG:teuthology.orchestra.run.smithi096:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:45:18.477 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:45:18.478 DEBUG:teuthology.orchestra.run.smithi143:> test -e /home/ubuntu/cephtest/archive/coredump 2024-05-08T05:45:18.492 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-05-08T05:45:18.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-05-08T05:45:18.504 INFO:teuthology.task.internal:Transferring archived files... 2024-05-08T05:45:18.505 DEBUG:teuthology.misc:Transferring archived files from smithi059:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/remote/smithi059 2024-05-08T05:45:18.505 DEBUG:teuthology.orchestra.run.smithi059:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T05:45:18.561 DEBUG:teuthology.misc:Transferring archived files from smithi096:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/remote/smithi096 2024-05-08T05:45:18.561 DEBUG:teuthology.orchestra.run.smithi096:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T05:45:18.597 DEBUG:teuthology.misc:Transferring archived files from smithi143:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2024-05-08_04:00:39-fs:workload-wip-57676-3-testing-default-smithi/7697230/remote/smithi143 2024-05-08T05:45:18.597 DEBUG:teuthology.orchestra.run.smithi143:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-05-08T05:45:18.632 INFO:teuthology.task.internal:Removing archive directory... 2024-05-08T05:45:18.632 DEBUG:teuthology.orchestra.run.smithi059:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T05:45:18.635 DEBUG:teuthology.orchestra.run.smithi096:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T05:45:18.638 DEBUG:teuthology.orchestra.run.smithi143:> rm -rf -- /home/ubuntu/cephtest/archive 2024-05-08T05:45:18.680 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-05-08T05:45:18.695 INFO:teuthology.task.internal:Not uploading archives. 2024-05-08T05:45:18.695 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-05-08T05:45:18.718 INFO:teuthology.task.internal:Tidying up after the test... 2024-05-08T05:45:18.719 DEBUG:teuthology.orchestra.run.smithi059:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T05:45:18.721 DEBUG:teuthology.orchestra.run.smithi096:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T05:45:18.724 DEBUG:teuthology.orchestra.run.smithi143:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-05-08T05:45:18.734 INFO:teuthology.orchestra.run.smithi059.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 05:45 /home/ubuntu/cephtest 2024-05-08T05:45:18.736 INFO:teuthology.orchestra.run.smithi096.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 05:45 /home/ubuntu/cephtest 2024-05-08T05:45:18.738 INFO:teuthology.orchestra.run.smithi143.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 May 8 05:45 /home/ubuntu/cephtest 2024-05-08T05:45:18.739 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-05-08T05:45:18.751 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-05-08T05:45:18.789 INFO:teuthology.run:Summary data: description: fs:workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/legacy wsync/no} objectstore-ec/bluestore-comp-ec-root omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/random export-check n/5 replication/default} standby-replay tasks/{0-subvolume/{with-namespace-isolated-and-quota} 1-check-counter 2-scrub/no 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/suites/fsx}} duration: 2186.555332660675 flavor: default owner: scheduled_vshankar@teuthology success: true 2024-05-08T05:45:18.790 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-05-08T05:45:18.874 INFO:teuthology.run:pass